I tried to use for/async with in-hash and that didn’t work (at all). The body of the for seems to be future-friendly (fold/fold, in-flvector, flvector-set! and fixnum and flonum operations). Does in-hash being the culprit sound plausible? If so, can there be a way around it that doesn’t involve converting the hash?

Worst case, I’m still interested to know whether converting the hash would likely solve my issue actually—but this is not a trivial change as it impacts other things

Didn’t work as in there was no speed up?

My recommendation would be a non-equal hash if that works

I’m using a hasheq with fixnums and there was a large slowdown compared to the serial version

Can you try using in-hasheq?

Wait, is that a thing? It doesn’t exist in the docs

Oh sorry, I thought it was

too bad, that would have been a good candidate :slightly_smiling_face:

There is in-immutable-hash

That would require a conversion I suppose. If I go down the conversion path, I’d better turn my hash into two large flvectors. Maybe I should do that.

Oh it’s a mutable hash? Then you could also try in-immutable-hash

interesting—also, suprising!

Or do you mean in-mutable-hash instead?

Argh it autocorrected me

(test running, it takes several minutes to complete)

Nope, doesn’t work: cpu: 234435 real: 88593 gc: 62230

That’s a 4x speed up, but I assume slower than serial

3x

Except that the serial version actually runs in ~1s. (Sorry for the misleading timings above)

Yikes

Also how many futures is that creating?

Have you used the future visualizer?

(processor-count) reports 96

I haven’t tried the vizualizer yet (trying now)

On a smaller example the visualizer says 0 blocks and 0 syncs, but the execution is rather spread out

Oh it could be because each iteration of the loop is actually way too fast, and the cost of instantiating futures is larger than that.

Ideally, for/async would know how many iterations N are to be done, and perform N/(processor-count) of them serially in each future, rather than instanciating one future per iteration

It doesn’t seem easy to do that with a hash and manual futures, but that should be fairly straightforward with a long indexable vector instead. Thanks for your help!

1. Study the implementation of the following data science abstractions in the code base of https://github.com/n3mo/data-science and describe their different levels of data and/or procedure abstraction layers. Provide a diagram showing the different abstractions levels per abstraction. (a) linear-model (b) list->sentiment (c) read-csv (d) qq-plot* (e) hist The data science abstractions offer support for sentiment analysis, a technique that is commonly used to quickly determine the mood, or emotional valence of a body of text.

For example, using the system to analyze the mood of words for textual novels provided by Stories and Fiction http://textfiles.com/stories/ yields the following results.

hey everyone, kindly need help with this… any help rendered will be highly appreciated

Hi @tibajoel90 Welcome to the Racket Slack:slightly_smiling_face:

Lots of people ask for help and it is often not clear exactly what they are asking…so we put together some guidance ; > > <https://github.com/racket/racket/wiki/How-to-get-help#how-to-ask-for-help|How to ask for help> > > Here’s a checklist to make things easier when asking for help, in order: > > > 1. What is the task? > 2. How far have you come? (Provide the smallest working example of your problem if you can) > 3. What exactly are you struggling with? > 4. What approaches have you tried to solve it?

Try answer each of the four questions when asking - and please ask in #beginners

Do you mind if I ask what school you attend and who is your professor? It is always nice to hear of another course using Racket :heart:

hey @spdegabrielle thanks for the welcome and guidance on how to ask questions. its makerere university in Uganda

let me formulate and use #beginners channel, thanks

@capfredf any updates on this issue?

I haven’t got time to look into it yet

I see. It seems to be something related to provide all-defined-out.

Once I removed it, Racket finishes very quickly.