Out of curiosity: is there a noticeable performance difference between using set! and using a box?

Make a benchmark :slightly_smiling_face:

https://docs.racket-lang.org/atomichron/index.html

And then “Advanced”, I think?

With the caveat that my benchmarks may have been bad, it appears that using set! is about 4x faster than using boxes, in a very trivial case. However, in practice, the difference is so small as to be visible only in the case of a massive number of operations.

By, “massive”, I mean hundreds of millions of operations.

I’m not really surprised. Now, it will also depend on the context of use, since various optimizations may be unabled or disabled by either.

Also, these operations are so fast that even with a massive sample size, the benchmarks vary widely in their results, so the better part of the reported times is probably noise.

Given that the performance difference is negligible, I think that I’ll forgo set!, in favor of boxes, for my lang.

So what are the advantages of boxes compared to simply using set!? I have seen them used as output arguments to some methods in racket/draw, but I cannot think of any other use case for them, except perhaps when you need a compare-and-swap operation, since box-cas! exists…

Boxes can be easier to reason about in a statically-typed setting, which will eventually be relevant to my lang. Also, in my lang struct/record types are immutable, so to have a “mutable” field, you’d need to use a box anyway.

Isn’t an immutable struct with boxes just a mutable struct?

Essentially, yes.

However, the underlying implementation of my lang’s structs is strictly immutable, so it’s necessary to use the box technique to have a mutable field.

And, my lang discourages (but certainly doesn’t forbid) mutation, so mutable struct fields are not intended to be common.

I also strongly prefer that bindings be immutable, which requires removing set!.

In the context of #lang racket code, I don’t think there’s a big difference between set! and boxes. I’m just being overly opinionated, as amateur language designers are prone to be.

I ran into this today. I can’t write a subtraction the way I intended. (when (> (- (- λ' λ)) 1e-12) ; because I can't write (- λ λ') (converge))

Maybe use U+2019 ? (or some other symbol that looks like a quote)

It was weird, because λ’ seemed to be fine everywhere else, just not before )

This was fine: (let … [λ' #f] …)

Well, 'λ is not going to raise a read error, and won’t raise an exception as long as it’s not evaluated

ah

but … ') will raise a reader error

This was fine: (set! λ' λ)

and '#f = #f

Then λ = 'λ :wink:

For some reason I thought we could use ’ for prime at the end of a symbol.

nope :slightly_smiling_face:

you can use . though

Makes sense, just thought I saw that somewhere, and thought “great, I’ll use it!” Guess I won’t.

Or maybe they did use a different quote symbol that looks like a normal quote?

(and didn’t tell you just to mess with the reader)

That’s too tricky. I’ll probably just use λprime even if it’s ugly. It only appears in a few places.

Thanks.

But thanks, λ’ definitely would not have worked:

Also keep in mind that you’re shadowing Racket’s λ :wink:

Oh, hadn’t thought of that, but yeah. Fortunately this is iterative code dealing with spherical coordinates, nothing lambda-like about it, λ is longitude.

@alexharsanyi One example comes to mind. A bit involved though.

A lambda expression creates a closure. A closure consists of two things: 1) The code to execute when then closure is invoked and 2) Some representation of the values of the free variables at the time the closure is created. One representation (called a flat closure) stores the values in a simple vector.

When the body is evaluated, a variable reference becomes (vector-ref free-variables variable-index). This works fine as long as all variables are immutable. If a variable is assigned to after the closure is created, then the vector reference gets the old value.

Compilers fix this problem with a pass called “assignment conversion”. A previous pass has determined all variables that are assigned to. All definitions of the form (define x 42) is then rewritten to (define x (box 42)) . All assignments (set! x 43) are rewritten to (box-set! x 43). This effectively makes all variables immutable. The flat closure will now contain the box, and now a variable reference to such a variable becomes (box-ref (vector-ref free-variables variable-index))`.

This solution allows a simple representation of closure (fast creation) at the cost of making each variable reference of assignable variables a little slower.

In short, compilers can introduce boxes in order to remove assignments from a program.

> All definitions of the form  (define x 42) is then rewritten to (define x (box 42)) . All assignments (set! x 43) are rewritten to (box-set! x 43). This effectively makes all variables immutable. I have no experience in compiler implementation, but I fail to see how this makes the variable immutable — to me x in your example above is just a mutable as it was before boxing, just there is an extra level of indirection now…

I think “more details”

It just means that variables themselves always point to the same thing

Which makes it simpler to think about them

In a language with assignable variables and mutable vectors/structs, boxes aren’t strictly needed. That might explain why I couldn’t think of other examples (than the implementation of assignable variables). :slightly_smiling_face:

So a compiler might rewrite code using set! and introduce box , unbox and box-set! for variables which are mutated. This would make the bindings themselves (e.g. let or define) always immutable, allowing the compiler to perform some optimizations on them?

Exactly.

> Every problem can be solved by adding one more level of indirection :wink:

Certainly boxes aren’t necessary, but it’s nice to express a single box instead of a 1 element vector, and to have a first class value instead of encodings with closures

But maybe remove compiled files first? badkins@create src % time (raco make chess.rkt) ( raco make chess.rkt; ) 6.36s user 0.81s system 93% cpu 7.647 total badkins@create src % time (raco make chess.rkt) ( raco make chess.rkt; ) 0.48s user 0.24s system 96% cpu 0.757 total cc: @me1890

And if that level of indirection itself causes a problem? You just add another level of indirection. :stuck_out_tongue:

The best part is that these abstractions never, ever leak.

And having multiple notions of a thing, in no way gets into the usual complications of identity and equality. :slightly_smiling_face:

These and more tips from my upcoming book, Sarcastic Recommendations from a Bitter Senior Software Engineer

Did you try using unsafe-unbox and unsafe-set-box! in comparison to set!? In theory those could be as fast as set! due to constant propagation by Racket.

Actually, probably better CPU wise to do mpair? and unsafe-set-mcar!. These do not seem to actually check their types.

@dyllongagnier you can use unsafe-unbox* to avoid that check

Hi! A question about an idea I had: parinfer &co. allow structural editing for Lisp languages. Since all #lang expand to racket, wouldn’t it be possible to add structural editing to any #lang by “following the macro”? Would that be possible for DrRacket through a configurable Quickscript rules engine?

Not any #lang, since they can have an arbitrary parser.

Hm. Indeed, I hadn’t thought about that… :pensive:

It’s not impossible

but it will require cooperation from the #lang

you can implement structural editing for syntax objects and use the #lang’s reader to turn text into syntax objects

the only missing piece is a #lang-specific mechanism for turning syntax objects back into text

I can imagine a lot of uses for #lang implementations including a writer submodule so that programs can turn syntax objects into formatted code

looks like registration for Racket Package server is broken Exception The application raised an exception with the message: string::1: string->jsexpr: bad input starting #"<html><head><title>Servlet Error</title><link rel=\"stylesheet\" href=\"/error.css\"/></head><body><div class=\"section\"><div class=\"title\">Exception</div><p>The application raised an exception with the message:<pre>subprocess: process creation fai... Stack trace: -raise-read-error at: line 15, column 2, in file /home/pkgserver/racket/collects/syntax/readerr.rkt <unknown procedure> at: line 17, column 0, in file /home/pkgserver/racket-pkg-website/src/json-rpc.rkt notify-of-emailing at: line 471, column 0, in file /home/pkgserver/racket-pkg-website/src/site.rkt login-or-register-flow* at: line 316, column 0, in file /home/pkgserver/racket-pkg-website/src/site.rkt <unknown procedure> at: line 375, column 33, in file /home/pkgserver/racket/collects/racket/contract/private/arrow-higher-order.rkt Whom could I send a report?

That would be Jay (@jeapostrophe), I think

@sorawee Thanks, I forwarded it to them. :thumbsup:

I think it’s the same problem as https://www.reddit.com/r/Racket/comments/leoths/cant_register_on_pkgsracketlangorg/

ah yes, so it’s not only me :slightly_smiling_face:

OK, I made https://github.com/racket/racket-pkg-website/issues/72, since Jay doesn’t seem to be active on Slack

Thanks! I’m following the issue now.

I was under the impression that Chez performs assignment conversion (i.e., transforms all set!-ted variables to boxes) early on in the compilation process, so I’m surprised to hear this. Maybe Racket’s boxes are a bit more complicated? Or maybe my knowledge of Chez is out of date.

To be fair, if I run the benchmarks a few times, the gap between them closes, which leads me to believe that most of the difference them is noise.

Hmmm :thinking_face: (define (merge-input a b [limit 4096]) (or (input-port? a) (raise-argument-error 'merge-input "input-port?" a)) (or (input-port? b) (raise-argument-error 'merge-input "input-port?" b)) ... actual function body here ...) (spotted in the implementation of racket/port)

would it be worthwhile to suggest rewriting that to use unless? or would that be too nitpicky

@notjack coming back to this “writer”: such a thing would enable common tooling for #langs (not only structural editing), right? This sounds exciting to me. In my understanding, the “writer” part would amount to reversing the #lang parser+lexer. Perhaps by providing a #datalog equivalent to parser-tools?