
After refactoring to use a long flvector instead of a hash, I can now efficiently spread the n-proc futures, each future taking care of n-jobs/n-proc jobs in serial, which avoids most of the cost of creating n-jobs futures with a bare for/async
. The gain is substantial as it now runs in 68ms (instead of >1s in full serial).

Originally my student Sarah wrote an implementation of futures in Racket CS that used work stealing and adapted dynamically which would have worked better for your use case, but Matthew wanted to stick to the model that Racket BC had

Oh that’s too bad, that would have been perfect for me! Any chance this can be restored as a package maybe?

(not that I need it anymore though)

No, that definitely needs to be built in. And it wasn’t a complete implementation, and was a while ago, so it couldn’t just be restored as is.

But since you’re using futures seriously, if that model would be better for you it’s worth discussing.

It would have avoided me the one-day cost of refactoring for sure, but now I don’t see a new need for the adaptive version for my work for now. It could certainly be useful to others though, and I’m considering adding a notice to the for/async docs explaining the caveat