~/w/h/o/cmdstan-2.18.1 > ./classify.px ../../input/GmmGibbs/50-10000
Can't exec "./gmm": No such file or directory at ./classify.px line 37, <> line 1.
Died at ./classify.px line 37, <> line 1.

I tried stanc on gmm.stan but that just gives me a cpp file

Hey. Say make gmm instead of anything mentioning stanc. I’m in Luddy 3115 btw.

(I see you’ve downloaded STAN, and please make sure you keep the two files I patched)

Thanks!!!

make gmm works

Yay!!

is it possible to make stan run until 10 seconds and minimum 40 trials?

there is a lot of variation, for plotting we mean across trials but stan is weird

sometimes it runs around 40 trials and sometimes only 10

for example: 0.0824637110000001 1 [0.402976898613895] 0.122636635 2 [0.402976898613895] 0.163189989 3 [0.402976898613895] 0.245272901 4 [0.402976898613895] 0.327615521 5 [0.402976898613895] 0.488534931 6 [0.404432580446774] 0.653543113 7 [0.404480626387967] 0.814336181 8 [0.395560172097108] 0.975607016 9 [0.397165642884747] 21.492779457 10 [0.372059883874268]

gmm 50 – 10000 with stan

Right now it stops at the first sample after 10 seconds. If we need more (why?), we can definitely change it - it’s in one of the two files I patched - there’s an if (sec > 10) that you should add \|\|… to

You’re saying that it sometimes stops before 10 seconds!

no it less number of sweeps for some trials

*it has

if you look at the plot it has a lot of variance at the end

Are you around in Luddy?

no I can be

Call me at

@rjnw it would be good to come in this afternoon

okay I will check in the plots and then come

I just added linear interpolation to classify.px

@ccshan I don’t see any commits

@rjnw Oops sorry, I just pushed

if we run stan for longer, does it catch hakaru?

Good question - can be answered by changing several occurrences of 10 in C++ and Perl code :flushed:

the stan code is a significantly different algorithm than any of the others, right?

Yes

I will try running it for 20

are there other different properties for the algorithm that matter in other contexts/measurements/etc?

Um. There are lots of differences…

I changed the generate_transitions to throw error at 20 and classify to range over 2048 ticks

I wonder if it’d be too much trouble to re-run the other systems over 20 seconds though…

for jags it might take a while

@ccshan are you in luddy?

@rjnw Have you been running STAN GMM experiments? I just remembered that we should measure the (significant) compile time and (negligible but measurable) startup time of STAN.

We still have 2.5 pages of room!

I was at school writing I just got back, I will run it now

STAN I already have for 20seconds I just need to run the rest of them

Ok, so to measure the compile time, it’s literally for ((i=0; i<10; i++)) do time sh -c "rm -f gmm.hpp gmm; make gmm"; done

(you can just email me the output) and let me work out how to measure the startup time…

By the way, when you make the new GMM plots, would you please make sure that the shading is not cut off at the top (like the Hakaru curve currently is in the first plot, n=10000, m=50, above 40% accuracy)?

the new intro is good, and the new portion of section 3 is really good

I just pushed new plots in the benchmarks repo, I also commited the accuracy files so now anyone can build the plots again with:

racket GmmGibbsAccuracyPlot.rkt --x-max 20 --y-min 20 --y-max 45 --height 300 --width 700 50 10000 "gmm-50-10000.pdf"

inside the output folder

That’s great because I easily changed the size and font :slightly_smiling_face: