:blush:

@ryanc I’m trying to fix https://github.com/racket/racket/issues/1909, but I have a question or two if you have a moment

@lexi.lambda yes?

@ryanc I actually just opened a PR with a hack that resolves the issue, but I don’t know if it’s a good/acceptable solution https://github.com/racket/racket/pull/1911

I’d appreciate your review

I originally tried to implement it with splicing-let-syntax instead of the cheap switch on definition?… but that broke Typed Racket because TR does a surprising thing.

@ryanc I think the alternative solution would be to attach a syntax property to the expansion that has the original identifier in it, then adjust each of the macros exported by syntax/parse to inspect the syntax property.

that would be a little more work, but it might cause fewer problems.

@lexi.lambda I’d rather avoid the extra indirection if possible. Another approach I wanted to try is basically making lazy-require work on macro transformers. That is, the setup is similar, but the proxy macro, instead of constructing a new term with the result of (get-id), calls syntax-local-value on the result of (get-id) to get the original macro’s transformer, then applies the transformer directly.

oh, that’s an interesting idea. I could try that.

If that works, it might be possible to generate the auxiliary modules automatically as submodules and wrap up the whole thing as lazy-require-syntax :slightly_smiling_face:

I’m trying it right now, but I don’t see why it wouldn’t

all the tests pass, at least :)

Yet another thing to try is the procedure variant of prop:rename-transformer… but that might interact badly with provide, since it tries to bypass normal rename-transformers unless ordered not to.

and the only way to order it not to is to make the rename transformer not free-identifier=?, right?

yes, as a syntax-property on the identifier, iirc

the syntax-local-value approach seems to work well, and it’s much simpler. I could try and implement lazy-require-syntax, but I don’t think I fully understand what needs to be done to preserve the laziness.

I remember you explaining this in the inside racket stream you did on syntax/parse, but I didn’t fully understand all the mechanics

thanks, if you update or replace the PR I’ll merge that

just pushed

Getting the old lazy-require solution to work for syntax-parse was a lot of trial and error and running into random details about things like how dynamic-require interacts with module-path-index (de?)serialization.

I think I also just don’t really understand all the details of when Racket maintains a module dependency from a macro use. I understand that if a macro expanded to a top-level define-syntax or begin-for-syntax, some of that phase 1 code will stick around in the compiled module… is that the only detail? so syntax/parse ensures the only thing left behind after expansion comes from the residual module? why can’t Racket detect that only the residual things are left behind and eliminate the dependency on syntax/parse/private/sc?

Typically, Racket just preserves the module dependencies declared through require. Macros typically only insert references from required modules. So if Racket instantiates all of a module’s dependencies before instantiating the module itself, all of the residual references will have been defined. Using lazy-require to trim the set of instantiated modules can interfere with Racket’s bookkeeping, runs two risks. First, a needed module can get left out. Second, a residual reference points to a definition site with a module-path-index that can contain a chain of relative module paths. If some of those relative requires aren’t executed, Racket can’t figure out what (absolute) module path the reference is supposed to be linked to.

See also https://macrologist.blogspot.com/2011/10/lazy-module-loading.html, which was written when some of the details were fresher in my mind.

@ryanc Thanks, that’s helpful. I think the issue is that I’m grappling with my intuition of macros-as-compilers, where the macro spits out some code as a byproduct, but all the macros go away by the time the module is compiled. Therefore, shouldn’t the macro be an exclusively compile-time dependency? And shouldn’t Racket understand that and not instantiate phase 1 things after the module has already been compiled?

Racket doesn’t instantiate phase–1 things at run time – as long as you use something like dynamic-require, which racket does when given a module on the command line. But modules carry all dependencies in case you use eval in a way that needs to trigger phase–1 instantiations. If you know that eval and variants won’t be used (because you know more than the compiler and run-time system), you can use a tool like the demodularizer to discard phase 1 and up.

@mflatt is there a way for a module to declare it’s “un-eval-able” that would let rackets compilation process do that without users having to mess with extra command line tools? something similar to how cross phase persistence is declared?

by “is there a way” I think I mean “is it possible to add that to Racket with reasonable semantics” rather than “can I do it myself with the latest Racket right now”

@notjack I think that would be technically possible, but it’s not obvious to me how to make it work reasonably with even basic tools, such as a REPL.

I think I’d only use it as a way to write a tiny “main entrypoint” module for a webserver to keep build script complexity down, so simply not offering a REPL (like #lang info) in that case is fine by me

It’s not clear to me what that would buy you compared to running the main entrypoint with racket, assuming that there really aren’t any evals.

as in, I would want raco setup to “automagically” make demodularized forms of these sorts of modules so when running that sort of entrypoint with racket the compiled stuff it uses is demodularized already

but I don’t really know enough about how those tools work to know if that makes sense

@ryanc I also just saw your #:undo addition! That’s exciting.

@mflatt When using the handin-server this semester, we (@ccshan and @rrose1 and I) saw occasional instances of tcp-write erroring with “Broken pipe”, causing submissions to be unsuccessful. From reading online, this seems like the result of the socket being closed by the client at an inopportune moment. Have you seen this? Is there some way we can adjust handin-server to handle this situtation? We can certainly do the implementation if there’s a place we should look.

@samth I’m not sure what you mean. Broken pipe errors are a fact of life with TCP on a real network, so the handin server and clients will see them occasionally, and a user on the client side will have to try again occasionally. Is there some more serious consequence that you’re seeing?

Yes, we’ve gotten multiple reports that students see the “Handin successful” message and then nothing is there on the server.

Certainly we’ve seen this error in the log without hearing about any corresponding error messages displayed to students (and I expect we would have heard if they got them)

If you run a server for a while and don’t get “Broken pipe” messages, it means that your network cable is unplugged.

I don’t think you want to chase network errors. If the server can report success before it actually commits to success, then that’s certainly a problem. I haven’t looked at this code in a long time, so it’s taking me a while to work out how and when success is reported.

@samth I don’t see a way for a client to show “Handin successful” without getting the final 'ok message from the server, and I don’t see a way for the server to send 'ok until it has renamed an attempt to “SUCCESS-<n>”. I certainly could be overlooking something, but I doubt that the issue is an ill-timed broken connection.

What we’ve seen is several students at various times emailing to say that they submitted and didn’t get an error (they say they got the success message but that isn’t totally trustworthy), but that the file isn’t on the server. In almost all of these cases, the log shows their most recent submission to the relevant assignment errored with a broken pipe

I don’t remember what an error other than a test failure looks like on the client side, so maybe an error isn’t reported prominently enough. In any case, if there’s a bug here, it can’t be in mishandling network errors, and it must be in a report of success before first committing to success.

It seems like pkgd may have fallen over again…