All benchmarks were executed on a dual-socket 12-core i7–4770 with 32 GB of RAM, running Ubuntu 16.04. We conducted our benchmarking inside a Docker container for reproducibility. We used Racket 6.11, LLVM 5.0.1, Maple 2017, and GHC 8.0.2.
haskell ClinicalTrials: 0.8196798780000001 0.9665
hakaru-benchmarks/runners/hk$ stack exec linearRegression 0.6605350149999996 6.19056602609257e–3
0.1 0.2 0.30000000000000004 0.4 0.5 0.6000000000000001 0.7000000000000001 0.8 0.9 1.0 1.1 1.2000000000000002 1.3 1.4000000000000001 1.5 1.6 1.7000000000000002 1.8 1.9000000000000001 2.0 0.717 0.717 0.717 0.717 0.717 0.717 0.717 0.717 0.717 0.717 0.717 0.717 0.717 0.717 0.717 0.717 0.717 0.717 0.717 0.717 0.522 0.522 0.522 0.522 0.522 0.522 0.522 0.522 0.522 0.522 0.522 0.522 0.522 0.522 0.522 0.522 0.522 0.522 0.522 0.522 0.374 0.374 0.374 0.374 0.374 0.374 0.374 0.374 0.374 0.374 0.374 0.374 0.374 0.374 0.374 0.374 0.374 0.374 0.374 0.374 0.392 0.392 0.392 0.392 0.392 0.392 0.392 0.392 0.392 0.392 0.392 0.392 0.392 0.392 0.392 0.392 0.392 0.392 0.392 0.392 0.397 0.397 0.397 0.397 0.397 0.397 0.397 0.397 0.397 0.397 0.397 0.397 0.397 0.397 0.397 0.397 0.397 0.397 0.397 0.397 0.566 0.566 0.566 0.566 0.566 0.566 0.566 0.566 0.566 0.566 0.566 0.566 0.566 0.566 0.566 0.566 0.566 0.566 0.566 0.566 0.711 0.711 0.711 0.711 0.711 0.711 0.711 0.711 0.711 0.711 0.711 0.711 0.711 0.711 0.711 0.711 0.711 0.711 0.711 0.711 0.552 0.552 0.552 0.552 0.552 0.552 0.552 0.552 0.552 0.552 0.552 0.552 0.552 0.552 0.552 0.552 0.552 0.552 0.552 0.552 0.675 0.675 0.675 0.675 0.675 0.675 0.675 0.675 0.675 0.675 0.675 0.675 0.675 0.675 0.675 0.675 0.675 0.675 0.675 0.675 0.636 0.636 0.636 0.636 0.636 0.636 0.636 0.636 0.636 0.636 0.636 0.636 0.636 0.636 0.636 0.636 0.636 0.636 0.636 0.636 0.45 0.45 0.45 0.45 0.45 0.45 0.45 0.45 0.45 0.45 0.45 0.45 0.45 0.45 0.45 0.45 0.45 0.45 0.45 0.45 0.624 0.624 0.624 0.624 0.624 0.624 0.624 0.624 0.624 0.624 0.624 0.624 0.624 0.624 0.624 0.624 0.624 0.624 0.624 0.624
[samth@huor:~/…/output/accuracies/GmmGibbs (master) plt] gist hk/9–1000 https://gist.github.com/cbdfd4bb7cf8b072966faba0999a778b [samth@huor:~/…/output/accuracies/GmmGibbs (master) plt] gist jags/9–1000 https://gist.github.com/aa424297324da71ec0bfeb6df3752ce4 [samth@huor:~/…/output/accuracies/GmmGibbs (master) plt] gist rkt/9–1000 https://gist.github.com/ccbc1fef76f5b7ec521299f3abf2d995
perl -pe ‘s,(^|\t)(\S+),$1 . ($2/1000),ge’ 9–1000
@pravnar @rjnw I have your water bottles
@pravnar did you manage to produce a fixed plot?
@samth Can I get them on Monday?
yes, that’s fine
just letting you know
Thank you, I found out when I came back.
This is after fixing the accuracies code and using 9–1000-rescaled for rkt.
I don’t know why sham is less accurate. I don’t know why hakaru’s accuracy declines.
@pravnar the numbers there seem to be smaller than in the 9–1000 accuracy file
for example, this file, which I think has sensible data, all the numbers are bigger than the ones plotted: https://github.com/rjnw/hakaru-benchmarks/blob/4da51afa586451a16a87ff4dc2c289a0adbbcfc8/output/accuracies/GmmGibbs/rkt/9-1000
I agree. When I run gmmAccuracy (which I did for the most recent plot above), I get much smaller numbers.
Which is weird.
The reason I decided to run gmmAccuracy again is so that I resample accuracies at correct time intervals. In the file you just sent, I am not sure what times were used for resampling the accuracies in columns beyond “2”.
how is it that I get such different numbers?
I would be interested to see if you “still” get different numbers. If you could run gmmAccuracy again, it would be telling.
trying
btw, can you fix the repo so that stack build works?
ok
it still fails on several missing files
Ok pushed the missing files
Please let me know if stack build works inside runners/hk
now builds properly, thanks!
btw, @pravnar and I now get the same (bad) accuracy results for rkt GmmGibbs