laurent.orseau
2022-5-6 14:46:14

After refactoring to use a long flvector instead of a hash, I can now efficiently spread the n-proc futures, each future taking care of n-jobs/n-proc jobs in serial, which avoids most of the cost of creating n-jobs futures with a bare for/async . The gain is substantial as it now runs in 68ms (instead of >1s in full serial).


samth
2022-5-6 20:33:20

Originally my student Sarah wrote an implementation of futures in Racket CS that used work stealing and adapted dynamically which would have worked better for your use case, but Matthew wanted to stick to the model that Racket BC had


laurent.orseau
2022-5-6 20:38:01

Oh that’s too bad, that would have been perfect for me! Any chance this can be restored as a package maybe?


laurent.orseau
2022-5-6 20:38:42

(not that I need it anymore though)


samth
2022-5-6 20:39:14

No, that definitely needs to be built in. And it wasn’t a complete implementation, and was a while ago, so it couldn’t just be restored as is.


samth
2022-5-6 20:39:45

But since you’re using futures seriously, if that model would be better for you it’s worth discussing.


laurent.orseau
2022-5-6 20:43:27

It would have avoided me the one-day cost of refactoring for sure, but now I don’t see a new need for the adaptive version for my work for now. It could certainly be useful to others though, and I’m considering adding a notice to the for/async docs explaining the caveat