I copied everything from our pldi submission

That second plot might be better on a log scale

disregard the plot for naivebayes, sorry it had wrong holdouts in racket

@samth do you mean logscale on x-axis?

Yes

okay I am running racket again I will post that when its done

The accuracy we expect is around 80%, not 97% or 100%.

@ccshan are you in luddy?

Yeah that plot is wrong I will update it shortly

No but I’m heading there

I added topic tag “compile-time optimization” alongside “run-time optimization”. @samth please feel free to look at and change the submission metadata. BTW there’s the “option to attach an annotated copy of the reviews of their previous submission(s), explaining how they have addressed these previous reviews in the present submission”

@ccshan done for my conflicts

I think adding the previous reviews would be good, but only if we have time

So what is the plan for the evaluation? I see very little in the paper itself. I know @rjnw has run tons of stuff - but if we don’t have something substantive here, we’ll likely get rejected again.

Hmm, I guess I should go in and see if I need to add people wrt conflicts. Will do now.

I know Rajan is running Naive Bayes experiments that would end up comparing our two backends, JAGS, and MALLET. So if Rajan is not working on writing, one thing Jacques can do is to write the evaluation, probably with help from Rajan’s writing and speaking. The punch line, which should come first and be supported hierarchically, is that we’re faster and more accurate.

There’s a bunch in the paper, in particular the GmmGibbs plot and the table of benchmark results for different optimizations

What else than Naive Bayes? We were hit with ‘one trick pony’, and the reviewers weren’t wrong, given the paper as written.

the plan is to add NaiveBayes as well

the reviewers were unhappy that we only talked about exact inference, which is no longer the case

No new conflicts.

I’ve dug into section 6 now. What I was expecting to see was a table with all the benchmarks together — or, at worst, one in 6.1 and one in 6.2. I was hoping to see ~5 benchmarks of rather different kinds in 6.2, with ~5 plots, and the table in 6.3 to have ~5 ‘Time’ columns, one for each benchmark.

So it looks like we have 1 (GMM) right now. And we may have 2 (GMM+NaiveBayes) by tomorrow. Is that right?

Right. I hope sharing your hopes helps you make your hopes come true.

@carette I will generate for clinical trial and linear regression

but that is still only haskell and rkt comparison

when you say table what do you want in that table? what should be the rows and columns?

@ccshan I do not posses a “dual-socket 12-core i7–4770 with 32GB of RAM, running 16.04”, nor have access to a similar machine with all the non-trivial setup required to run benchmarks. Nor the knowledge of ML practice to come up with decent benchmarks. I thought that that was one of the great things about having multiple authors with different expertise all working together. </snark>

shrug

@rjnw In 6.1, I was hoping to see something like Figure 15. Similar rows, but with columns for Benchmark, time for clinical trial, and time for linear regression.

Also, should we perhaps put section 6.2 first, as that is going to be more interesting to ML people than exact inference?

I love exact inference myself, but that’s possibly somewhat niche compared to the applicability of approximate inference.

I will work on a table for exact inference

but do we need to show optimization techniques as rows?

if the rows are the same I can just add more columns to fig15

@ccshan I am trying to run activation from bin folder, it just exits without any info

for activating maple

Have you installed lsb-core and x11-utils?

no

let me try again

I understand it can be tricky to install Maple in Docker

that works

oh I am using native right now, I will change the evaluation to match what I am doing

Obviously the bulk of our remaining work is in Sections 5 and 6. Section 5 now starts well, so the rest of it is ready for anyone to revise. Section 6 has Naive Bayes measurements in the comments — thanks Rajan — so it is ready for revision as well as plotting.