I did some benchmarking today and found that one of the fastest ways to support iteration is surprisingly just to have the equivalent of for-each. i.e., a procedure which takes in a thunk to call on each element of the iteration. It was particularly efficient if said for-each function was inlined by defining it using define-inline or begin-encourage-inline. However, even without inlining, it was still much faster than a generator and about as fast as as a custom sequence using make-do-sequence.

This is actually a reasonable approach because the main thing missing with for-each iteration is early termination, but that is easily solved in Racket by an escape continuation which is pretty fast since you only call the continuation once for the entire for-each loop.

I initially thought this was an interesting concept because writing a for-each style function is incredibly easy, arguably even easier than a generator since you don’t need to use any special forms. The main disadvantage of this approach outside of Racket is that you can’t pause iteration, but if you really want to do that in Racket you can just use a full delimited continuation.

Note that the reason you want inlining of for-each is so that the thunk you pass it can in turn be inlined. This was with Racket CS, but there still is a measurable cost for dynamically calling a function.

That’s precisely what for loops are for :)

It has lots of features like #:break and variants like for/fold

(i realize i may be at the wrong level of answers, sorry : D )

Buy I found in my old benchmarks that for loops are often at least as fast as custom named let loops (which are supposed to be ‘lower’ level)

@laurent.orseau yeah, for loops are the best option for built in data structure. My methodology mostly makes sense if you need to write some new iteration strategy like iterating over a custom tree or something. The reason for loops are fast is because of custom support from the for macro at expansion time for stuff like lists, vectors, ranges, etc. I think you can add support for new iterators to the for macro, but it looks very complicated to do and may not be worth the effort in a lot of cases.

For example, maybe you want to church encode some data which requires enumerating the prime numbers. You could just write that as a custom for-each/fold function which would seemingly give you most of the performance of the custom for loop while being much simpler to write.

For such cases wouldn’t a named let loop would be just as simple, while directly avoiding the lambda?

Ah, or you mean that the inlined lambda is what you provide the user with, and they just have to write the for-each?