Had kind of a weird error, which I eventually pared down to: #lang racket (require parser-tools/lex-sre) (/ 4 2) unsaved editor:3:0: /: illegal use of syntax in: (/ 4 2) value at phase 1: #<lex-trans> #(45 7)

So apparently, parser-tools/lex-sre is incompatible with… math?

That module shadows a lot of names from Racket, including /. It’s common to require it with a prefix to avoid that problem. There’s a note about it on this page: https://docs.racket-lang.org/parser-tools/Lexers.html (search for “must be imported using a prefix”).

I have a deterministic program with no IO. It does a lot a list processing. When running it multiple times, I observe that the running times vary between 4.5s and 9s, with roughly 10% GC. What factors could account for such a difference?

I’m trying hard to make it fast and memory efficient, so that difference matters a lot to me

Can you share your program?

unfortunately not (plus it’s a pile of 2000sloc, undocumented)

It’s compiled and run from the command line btw.

Is it currently possible to hide a frame% and give focus to another window (not belonging to the Racket process)? I’m thinking of behavior similar to how Alfred and Spotlight work on macOS: you hit a key (eg. ESC ), the frame disappears and focus is restored to the window behind the one that disappeared.

Wait… that does seem to work fine if I run my app from the command line. I guess racket-mode must be doing something odd here. nvm!

Any hashing? Do Racket and Racket CS behave the same?

Zero (explicit) hashing. A lot of pairs though. I haven’t tried racket CS yet

Is 4.5s/9s CPU time or wall-clock time?

I guess I don’t have any ideas, yet, except to try Racket CS. If it’s lots of pairs, Racket CS should be both faster and more memory efficient, so maybe whether it is will tell us something.

it is both cpu and wall clock time

measured using both time on the command line and (time ...) in racket

I’ll try racket CS

To elaborate on what Ryan says. Use (require (prefix-in : parser/lex-sre) This will prepend a colon to all the names. That is, you now need to write :/ instead of just / . :* instead of just * etc.

If you lookup position-token, you will see it is a struct. One of the fields are named token, so to get the token stored in a position-token, you need to use position-token-token.

That’s quite surprising. Have you looked at a profile?

The statistical profiler gives no meaningful information anymore. I can give you a dump-memory-stats at high usage if you want.

Can you try the statistical profiler with errortrace on?

ah maybe I forgot that :confused: Will do!

Well that isn’t the normal thing to do, and it will make the program slower, but maybe give better info

The profile (redacted) with --use-errortrace is not meaningful either as far as I can tell. The lines referred to are the main functions, so nothing of interest.

You would need to recompile the program with errortrace enabled

(after re-reading the docs and trying a few things) I’m missing something. Isn’t it: 1. throw away .zo files 2. raco profile --use-errortrace myprog.rkt ?

I think that should work

ok. then it gives the same result as above. I’ll try racket CS

FYI, https://github.com/racket/racket/blob/master/racket/collects/pkg/private/catalog.rkt#L327

this is the code that reads pkgs/… or pkg-all

You could also try increasing the sampling rate

good idea

Wow, on Racket CS it runs in 2.4s with little variance!

Yay, but still surprising for traditional Racket

indeed

I’ll still try with higher sampling freq

@laurent.orseau Another thought, especially given the Racket CS result: does your program involve any deep (non-tail) recursion?

@samth The reason the profiler wasn’t giving anything is because I’m using with-limits which I guess implicitly runs a thread. Using the --all-threads options works

Aha

@mflatt There is possibly a fair amount of non-tail recursion, but it shouldn’t be very deep (say a dozen levels)

That’s a pretty good incentive to start using Racket CS :slightly_smiling_face: Thank you both for your help!

Obviously, let us know if just using Racket CS doesn’t work out. But Racket CS really should be a viable option at this point, especially if you can use snapshots. It’s worrying that we don’t know what went wrong in traditional Racket, but there are all sorts of areas where Racket CS provides more consistent performance than traditional Racket.

Aha, I think I found the culprit! I think I had forgotten to compile a dependency that basically exports a single macro (which is used at a sensitive place)

Now the times are about 4.1s to 4.3s for traditional Racket

Sorry for all the noise then, but that’s been very helpful anyhow

Wait, isn’t raco make supposed to compile the dependencies too?

yes, definitely

uh, then I don’t understand. The high variance results were after raco make myprog.rkt and the faster, low variance ones where after raco make myprog.rkt the-dep.rkt

that suggests a problem with raco make and the relationship between myprog and the-dep.

is it just a regular require?

yes

that’s very odd then

Maybe I’m interpreting too much and I’m overlooking something I did that would explain it

maybe you removed a zo file at some point, or edited the-dep

Hmm, I’m seeing some variance again, this time also with Racket CS (still faster though). I’m starting to suspect that my machine just has ups and downs

This time I’m sure I’m compiling everything. I’m running the same program twice in a row, once it takes 4.3s, the next one it takes 8.8s…

Hm, so far there’s only the one ’/’ I need - could I specify the namespace or re-bind ’/’ to ‘divide’ before the require comes into effect or something?

@samth was there something you were looking for in the profile that could give a hint about the variance?

just what it spent a lot of time doing

ok

DrRacket users who suffer slow/unresponsive scrolling: Does the current snapshot at https://www.cs.utah.edu/plt/snapshots/ behave any better?

Yes. It’s still a little unresponsive but better. Also with DrRacket 7.5 I can scroll at most about 100 lines in a single motion whereas with the current snapshot there is no practical limit. I’m on macOS high sierra by the way.

What’s changed? I looked at https://github.com/racket/drracket/commits/master and saw nothing.

It’s a placebo test :stuck_out_tongue:

my guess would be to look at racket/gui instead maybe

While it’s not a placebo test, I did intentionally refrain from explaining the change to avoid biasing the way you might test scrolling.

Hm, how would I easiest test whether a single pattern in my lexer works? Is there a function where I can plug in that pattern and a target string and see the result?

Oh this makes me excited, I’m gonna try out the snapshot as soon as I can after work

If you use brag , you may want to use https://docs.racket-lang.org/brag/#%28def._%28%28lib._brag%2Fsupport..rkt%29._apply-port-proc%29%29

thanks, I’ll check it out!

Hm, not quite what I’m looking for. I tried to make a testing function here: (define (investigate pattern string) (define-lex-abbrev x pattern) (define lex (lexer [(x) (token 'test lexeme)] )) (lex/src (open-input-string string)) ) Any advice about what I’m doing wrong?

define-lex-abbrev is a macro that needs a literal pattern, not a variable with a pattern in it.

we’re having trouble verifying the changes with raco pkg install and with step 5

Also IIRC most of the patterns in the parser-tools are not runtime values. They get compiled into the lexer.

step5 gives errors that typed-racket & other packages can’t be found

Some examples of errors I’m getting during step 5 include: raco setup: error: during making for <pkgs>/errortrace-lib/errortrace raco setup: open-input-file: cannot open module file raco setup: module path: syntax/source-syntax raco setup: path: /home/racket/racket/racket/collects/syntax/source-syntax.rkt raco setup: system error: no such file or directory; rktio_err=3 raco setup: compiling: <pkgs>/errortrace-lib/errortrace/stacktrace.rkt and: raco setup: error: during making for <pkgs>/source-syntax/typed-racket-test/performance raco setup: standard-module-name-resolver: collection not found raco setup: for module path: typed/racket/base raco setup: collection: "typed/racket" raco setup: in collection directories: raco setup: /home/racket/racket/build/user/7.5.0.11/collects raco setup: /home/racket/racket/racket/collects raco setup: ... [29 additional linked and package directories] raco setup: compiling: <pkgs>/source-syntax/typed-racket-test/performance/function-contract.rkt

okay maybe those step5 issues were our mistake … the urls we put in the catalog were wrong (we had https://.....#commit?path=.... instead of https://....?path=....#commit )

sigh, maybe both fails were because of that

success on 5 and 6!

how do we start an http server?

phew we used python3.7 -m http.server (also tried & failed with plt-web-server)

I’m at the step where I’m following the pkg-build instructions, and I’m getting this error when running run.rkt: >> Getting installer table hash-ref: no value found for key key: "{1} Racket \| {3} Linux \| {4} x64_64 (64-bit), natipkg; built on Debian 7 (Wheezy)" context...: /home/racket/.racket/snapshot/pkgs/pkg-build/main.rkt:123:0: build-pkgs "/home/racket/racket-pkg-build/run.rkt": [running body] temp35_0 for-loop run-module-instance! perform-require!

I usually use python3 -m http.server <port>

whats a good port? do we need to tell pkg-build the port?

the default is 8000 and seems okay; we tried 80 and get permission errors

The default port seems to be 8000, I’m getting permission errors when trying other things like 80

Yes, 8000 is fine

is there table.rktd under the directory build/site?

no, but there is table.rktd in the separate directory where I’m running run.rkt

weird, IIRC, after make site-from-installers, there should be a table.rktd under build/site

The table.rktd in that separate directory where run.rkt is is a hash-table where the first key-value pair is ("{1} Racket \| {1} Windows \| x86 (32-bit)" . "racket-7.5-i386-win32.exe")

I’m not sure about run.rkt ; I do have build/site/installers/table.rktd with 1 entry, localhost -> racket–7.5……..sh

What if I change the run.rkt to use "{1} Racket \| {3} Linux \| {3} x64_64 (64-bit), natipkg; built on Debian 8 (Jessie)" in the #:installer-platform-name?

Update: that seems to be an improvement

I’m a little confused. What is run.rkt?

From the instructions on pkg-build, in https://github.com/racket/pkg-build/blob/master/README.md#running-a-build

I see.

@ben if you start a http server with port 8000 in the directory build/site and open “http://localhost:8000/” in your browser, you should be able to see something similar to https://www.cs.utah.edu/plt/snapshots/current/installers/

or under build/site/installers

then run the run.rkt, see if it works this time

Yes. The scrolling is much smoother than it was before on macOS

It seems to have started working after we changed the string "{1} Racket \| {3} Linux \| {3} x64_64 (64-bit), natipkg; built on Debian 7 (Wheezy)" in run.rkt to use "{1} Racket \| {3} Linux \| {3} x64_64 (64-bit), natipkg; built on Debian 8 (Jessie)" instead

yes I do see the installers page

minutes ago, we both started a run.rkt and it’s doing things (like Alex says)

I gave it the #:host address for my computer though, instead of localhost:8000 so idk if things are going to break later

but we’re about to stop working on this for today. Thanks for the help!

Oh, I forgot to mention that for the #:installer-platform-name, you have to use a name listed in your table.rktd . Sorry about that…..

(well the author of pkg-build forgot too)

#:host "192.168.99.100" for that part, you need to make sure which networking mode you are using for your virtualbox instance

we should try to improve these readmes, and/or write a post for http://blog.racket-lang.org\|blog.racket-lang.org

Usually, I would use bridge mode and then ssh into the guest os and use ifconfig to get the ip address of the guest os (updated: #:host is the ip address of you vm (https://github.com/racket/pkg-build/blob/master/main.rkt#L328) )

then use that address in run.rkt

:cry:(looks like I forgot to mention more things than I thought)

So following the steps in pkg-build running run.rkt, It went for a while, including archiving, downloading, packing, and writing checksums for all the packages from a-to-z, but then failed with this error message: Creating catalog /home/racket/racket-pkg-build/server/archive/catalog >> Starting server at locahost:18333 for /home/racket/racket-pkg-build/server/archive >> Starting VM pkg-build Stopping VirtualBox machine "pkg-build" system*: contract violation expected: path-string? given: #f context...: /usr/racket-7.5.0.10/collects/racket/system.rkt:181:19 /usr/racket-7.5.0.10/collects/racket/system.rkt:174:0: do-system*/exit-code /usr/racket-7.5.0.10/collects/racket/system.rkt:211:0: system* /home/racket/.racket/snapshot/pkgs/remote-shell-lib/vbox.rkt:112:0: stop-vbox-vm /usr/racket-7.5.0.10/collects/racket/contract/private/arrow-val-first.rkt:555:3 /home/racket/.racket/snapshot/pkgs/pkg-build/main.rkt:445:2: install /home/racket/.racket/snapshot/pkgs/pkg-build/main.rkt:515:2: check-and-install /home/racket/.racket/snapshot/pkgs/pkg-build/main.rkt:123:0: build-pkgs "/home/racket/racket-pkg-build/run.rkt": [running body] temp35_0 for-loop run-module-instance! perform-require!

It appears to be the line (system* VBoxManage "controlvm" vbox what) here: https://github.com/racket/remote-shell/blob/71cb7647c90851fac4629523d34983375fc2caa3/remote-shell-lib/vbox.rkt#L52

Where VMoxManage is defined with (define VBoxManage (find-executable-path "VBoxManage")) here: https://github.com/racket/remote-shell/blob/71cb7647c90851fac4629523d34983375fc2caa3/remote-shell-lib/vbox.rkt#L29

ah ok, the pkg-build instructions did ask for VBoxManage to be on your PATH

I think that means we don’t want to run run.rkt in the VM

It is on my path outside the VM, but not inside

Does (find-executable-path "VBoxManage") return #f on the host os? @alexknauth

Depends on DrRacket vs Command-Line. • DrRacket: yes, it returns #f • Command-Line: no, it returns #<path:/usr/local/bin/VBoxManage> outside the VM

so if you run racket run.rkt at command line outside the VM, are you still getting the error message you posted ?

I’m in the middle of trying that now

I accidentally installed the wrong pkg-build package, and then I had a weird extra invisible character to delete, and now I’m running it

It’s going through archiving, downloading, packing, and writing checksums for all the packages again, I’ll update once it gets past that step to see if this error is fixed now

Okay, same place, different error this time: Creating catalog /Users/Alex/racket-pkg-build/server/archive/catalog >> Starting server at locahost:18333 for /Users/Alex/racket-pkg-build/server/archive >> Starting VM pkg-build Stopping VirtualBox machine "pkg-build" VBoxManage: error: Could not find a registered machine named 'pkg-build' VBoxManage: error: Details: code VBOX_E_OBJECT_NOT_FOUND (0x80bb0001), component VirtualBoxWrap, interface IVirtualBox, callee nsISupports VBoxManage: error: Context: "FindMachine(Bstr(a->argv[0]).raw(), machine.asOutParam())" at line 382 of file VBoxManageControlVM.cpp VBoxManage: error: Could not find a registered machine named 'pkg-build' VBoxManage: error: Details: code VBOX_E_OBJECT_NOT_FOUND (0x80bb0001), component VirtualBoxWrap, interface IVirtualBox, callee nsISupports VBoxManage: error: Context: "FindMachine(Bstr(VMNameOrUuid).raw(), machine.asOutParam())" at line 2621 of file VBoxManageInfo.cpp vbox-state: could not get virtual machine status: "pkg-build" context...: /Users/Alex/Library/Racket/snapshot-7.5.0.3--2019-10-21/pkgs/remote-shell-lib/vbox.rkt:112:0: stop-vbox-vm21 /Applications/Racket/2019-10-21/Racket v7.5.0.3/collects/racket/contract/private/arrow-val-first.rkt:555:3 /Users/Alex/Library/Racket/snapshot-7.5.0.3--2019-10-21/pkgs/pkg-build/main.rkt:445:2: install64 /Users/Alex/Library/Racket/snapshot-7.5.0.3--2019-10-21/pkgs/pkg-build/main.rkt:515:2: check-and-install70 /Users/Alex/Library/Racket/snapshot-7.5.0.3--2019-10-21/pkgs/pkg-build/main.rkt:123:0: build-pkgs57 "/Users/Alex/racket-pkg-build/run.rkt": [running body] temp37_0 for-loop run-module-instance!125 perform-require!78

Looks like the error is (error 'vbox-state "could not get virtual machine status: ~s" vbox) from https://github.com/racket/remote-shell/blob/71cb7647c90851fac4629523d34983375fc2caa3/remote-shell-lib/vbox.rkt#L49

Which is triggered by the state not being one of the symbols (\|powered off\| aborted running saved paused restoring) from the case expression https://github.com/racket/remote-shell/blob/71cb7647c90851fac4629523d34983375fc2caa3/remote-shell-lib/vbox.rkt#L43-L49

My probably-wrong-or-incomplete assumption was that the name pkg-build in the error message comes from (vbox-vm #:name "pkg-build" #:host "0.0.0.0:8000"), where the host 0.0.0.0:8000 comes from the IP address and port that the Python web server gave me. So when it says it can’t get the state of that machine, is it saying it can’t get the state from that Python web server?

I’m trying again with a different #:host field value taken from the result of running hostname -I on the VM

Well, even with that, same error: Creating catalog /Users/Alex/racket-pkg-build/server/archive/catalog >> Starting server at locahost:18333 for /Users/Alex/racket-pkg-build/server/archive >> Starting VM pkg-build Stopping VirtualBox machine "pkg-build" VBoxManage: error: Could not find a registered machine named 'pkg-build' VBoxManage: error: Details: code VBOX_E_OBJECT_NOT_FOUND (0x80bb0001), component VirtualBoxWrap, interface IVirtualBox, callee nsISupports VBoxManage: error: Context: "FindMachine(Bstr(a->argv[0]).raw(), machine.asOutParam())" at line 382 of file VBoxManageControlVM.cpp VBoxManage: error: Could not find a registered machine named 'pkg-build' VBoxManage: error: Details: code VBOX_E_OBJECT_NOT_FOUND (0x80bb0001), component VirtualBoxWrap, interface IVirtualBox, callee nsISupports VBoxManage: error: Context: "FindMachine(Bstr(VMNameOrUuid).raw(), machine.asOutParam())" at line 2621 of file VBoxManageInfo.cpp vbox-state: could not get virtual machine status: "pkg-build" context...: /Users/Alex/Library/Racket/snapshot-7.5.0.3--2019-10-21/pkgs/remote-shell-lib/vbox.rkt:112:0: stop-vbox-vm21 /Applications/Racket/2019-10-21/Racket v7.5.0.3/collects/racket/contract/private/arrow-val-first.rkt:555:3 /Users/Alex/Library/Racket/snapshot-7.5.0.3--2019-10-21/pkgs/pkg-build/main.rkt:445:2: install64 /Users/Alex/Library/Racket/snapshot-7.5.0.3--2019-10-21/pkgs/pkg-build/main.rkt:515:2: check-and-install70 /Users/Alex/Library/Racket/snapshot-7.5.0.3--2019-10-21/pkgs/pkg-build/main.rkt:123:0: build-pkgs57 "/Users/Alex/racket-pkg-build/run.rkt": [running body] temp37_0 for-loop run-module-instance!125 perform-require!78

Lookup rename-in