Was taking a look at the inner workings of Pycket and initial attempts to load a racket file return exn:fail : "can't find primitive make-ephemeron-hasheq"

does this mean there’s some new primitive used in the racket expander that hasn’t been added to pycket yet?

probably not in the expander (pycket has a specific version committed), but in the dependencies of racket/base

you might try running a file written in '#%kernel

well, i did make expander regexp fasl, which created a couple of files.

ah that might also have caused the problem

and I can see the use of that primitive in expander.rktl

it should be pretty easy to add make-ephemeron-hasheq; I’ll take a look

could you point me to where pycket is actually expanding a program into a linklet? I was expecting pycket to be interpreting expander.rktl on the source program to expand it, but expander.rktl exports an expand function which afaiu will not create a linklet but a fully expanded program. How/where does it go from here to a linklet?

Another interesting thing I found was this line: https://github.com/pycket/pycket/blob/05ebd9885efa3a0ae54e77c1a1f07ea441b445c6/pycket/racket_entry.py#L630

Apparently this commented line has been there for like half a decade. I am curious if there’s a specific reason for it to persist there - i.e. not removing it.

And this does mean that we are somehow not expanding the original code (maybe instead doing something else)?

First, that commented line is in a little mini-repl that is only for experiments.

Second, the expander goes all the way to linklets. The API that the expander relies on is the linklet API described here: https://docs.racket-lang.org/reference/linklets.html

ah, that explains why the line hasn’t been removed.

Also, regarding the expander, does that mean that the expand function exported by the linklet is different than the racket expand (which doesn’t generate a linklet) ?

The expand function exported by the linklet in expander.rktl is the same as the one from racket/base, but that isn’t used by the runtime.

I think the big thing you’re missing is that the calls go the other way.

If you do racket -l racket/base, then the runtime (in the racket_entry function) calls the namespace-require function exported by the expander with an appropriate s-expression, and then it’s done.

ah - now i got it.. and the expander linklet calls the runtime!!!

that’s why you said it goes the other way.

The expander then calls functions like compile-linklet which are provided by the runtime

But compile-linklet does very little — it just turns an s-expression into an AST.

The expander then calls instantiate-linklet, which runs the evaluation, which is what the runtime knows how to do.

OK. yes. I know what you mean. And that AST is dependent on what the runtime created at the compile-linklet call and independent of the expander linklet.

Yes, exactly.

suddenly - and maybe as a delayed realization - this seems very similar to the interface we have in JS to compile and instantiate wasm modules.

:bulb:

Thanks for explaining this. I honestly have been gathering partial pieces of knowledge about this for years and I think I only just pieced this together. :face_palm:

Yes, although I don’t think the callbacks to JS exist (because WASM doesn’t have macros)

Yes, that’s correct. You can call from Wasm to JS through wasm module imports but that’s different.