@samth is this the binomial heap benchmark from pycket’s figure 8 (oopsla 2017)? I’m running <collapsible>/racket binomial-heap-tests.rkt and its taking a few minutes (which makes sense, b/c collapsible doesn’t collapse structs yet)

the OOPSLA artifact should have it, but it’s originally from Robby’s paper a while back

where is the oopsla artifact?

I think on the DL

ok … I think I found it, there’s a 1GB zipfile on the dl page

yeah it’s not well-labeled

http://www.eecs.northwestern.edu/~robby/pubs/papers/ifl2007-fgr.pdf is the paper

it’s also in the pycket-benchmarks repo

that paper where I got the binomial I sent you, but seems like its taking too long

taking too long in what system/configuration/etc?

system = collapsible, configuration = N/A

I unzipped the tarfile here, and since Racket took ~40 seconds in the Pycket paper, I expected collapsible to be around that

but it’s been over 10 minutes

how fast is it in regular racket?

I haven’t tried yet

do you know where binomial is in the pycket-bench repo?

on, nvm https://github.com/sabauma/pycket-benchmarks/blob/7bec8f54e4e30c4b558e2c7bd4c5239b624b54a6/struct/binomial-heap-tests.rkt

hm, I don’t think I’ve found the right code

that binomail-heap-test.rkt is the same file I had before, and its very slow with + without collapsible

I let it run for a few minutes on the 6.12 baseline, then killed it

there’s a file pff.rkt in the same pycket-bench folder, but that’s slow too

did you try running the script mentioned in the README?

this readme? https://github.com/sabauma/pycket-benchmarks/blob/7bec8f54e4e30c4b558e2c7bd4c5239b624b54a6/struct/README

yeah

hmm the pycket artifact has a folder pycket-bench/ChaperoneBenchmarks/ but its empty

I’ll try tunning ./run-all

I mean, I can’t find anything “binomial” in the artifact

find . -name "*binomial*" … ag "binomial" … nothing

oh .. its a git submodule

./run-all has been running a few minutes, but hasn’t printed anything

the oopsla paper used racket 6.6, right?

yes

ok, I’m getting the same results on 6.6

does the plain racket (no contracts) one work?

no

well, hang on .. you mean the no-contracts mode in the benchmark

all I know is I haven’t found a way of running the code that prints any output … maybe I can comment things out to only try the no contracts mode

it should be pretty easy to figure out the command line from that shell script

(I don’t see the data for Fig 8 in the pycket artifact either, just timings for the gradual typing benchmarks)

this is probably an infinite loop where the contracts are defined; maybe missing recursive-contract?

I note that the last commit in the struct folder is “remove erroneous code”

where do you see that?

in the pycket-benchmarks log

[samth@huor:~/sw/pycket-benchmarks/struct (master) plt] git log —stat –1 commit 9673ec0e6e87bdc4dfbc4d3b3e0022e47eeb586c (HEAD -> master, origin/master, origin/HEAD) Author: Spenser Bauman <sabauma@gmail.com> Date: Tue Mar 8 00:05:16 2016 –0500

Remove erroneous code

struct/binomial-heap-contract.rkt | 39 ————————————— struct/pff.rkt | 2 +- 2 files changed, 1 insertion(+), 40 deletions(-)

ok, that commit isn’t in pycket/pycket-bench, the submodule isn’t up-to-date

ah

[samth@huor:~/sw/pycket-benchmarks/struct (master) plt] r pff.rkt koala-face.trace none 0 running koala-face.trace.time.0 none [samth@huor:~/sw/pycket-benchmarks/struct (master) plt] r pff.rkt koala-face.trace none 1 running koala-face.trace.time.1 none [samth@huor:~/sw/pycket-benchmarks/struct (master) plt] r pff.rkt koala-face.trace none 2 running koala-face.trace.time.2 none

my guess is that some change to the contract system broke that code a long time ago

which is sad but we were less careful about backwards compatibility back then

unfortunately something seems wrong with pycket’s command line handling for that code

so I can’t run that yet

can I send you a simpler version in a few minutes? a folder where you can unpack & run pycket-c main.rkt and see a time

sure

ok. Dan wants something similar to see how collapsible looks if the structs are replaced with vectors

note that that code is tricky because it has a bunch of old contract system features

opt/c in particular

@samth I think this should run on pycket, if you unzip and run pycket-c main.rkt (haven’t successfully tested it myself yet b/c I need to pull the new pycket commits)

sadly that seems to break for some reason

looking now