Are there any gotchas I should be aware of when using the Racket logo on a commercial website to link to the racket webpage? Can’t seem to find any ‘logo’ webpage with usage, etc. Something like: https://llvm.org/Logo.html

please, anything BUT the npm model

Does anyone have any experience with scripting docker-machine commands in racket?

QQ: is there a simple require we can use in ISL+ to include hash-maps?

@mflatt Can I throw some questions your way about local-expand and first-class definition contexts? And if so, would the mailing list be a better place for you than slack?

Here is fine

Okay, great. I’ve been spending some time over the past few days abusing local-expand, so in the process I’ve run into some things I don’t really understand. :) I think the biggest thing I’m confused about is the relationship (if one exists) between using a list for the context argument to local-expand and the internal definition context argument.

After reading the section on first-class definition contexts in the sets of scopes paper again a few times, I picked up that internal-definition-context-introduce essentially corresponds to an outside-edge scope. But what corresponds to the inside-edge scope?

Offhand, I would characterize it more as an inside-edge scope. An outside-edge could simply be applied to the body up front.

There is a kind of redundancy between the context as a list and the definition-context argument. The context as a list is meant for public visibility through syntax-local-context, though, while the provided definition context is not exposed that way. Macros might use the result of syntax-local-context – for example, to determine that a previously lifted definition definition can still be accessed, as the contract system does. But macros should not be able to get the definition-context value and directly add bindings or otherwise abuse it.

A drawback of separate arguments is that there’s no enforcement of the intended correlation between the context list and definition contexts.

Re: inside- vs outside-edge, yes, you’re right; I just checked the section of the paper again and discovered I’d misremembered which one it was. Wrt a list context as an argument to local-expand, what exactly does it do? Is it only relevant to macros that look at syntax-local-context? I guess maybe that’s always true for the context argument, since I suppose things like define just signal errors when syntax-local-context is 'expression, but I’m wondering what effect setting the context to, say, (list (gensym)) actually has on forms in racket/base.

Yes, it only affects the result of syntax-local-context. Well, the distinction between top-level, expression, etc., affects some built-in forms, but the built-in forms don’t care about the list content as a context; all lists are treated the same as just “a definition context”.

I see, interesting. That helps to clarify some of my confusion. What I’m essentially doing is abusing local-expand to reimplement parts of the macroexpander in userspace code so that I can inject custom behavior at certain places in the expansion process (for Hackett, that means defining my own “core forms” for the type/value language, and for the mutation testing project I’ve just started working on with Christos, it means creating mutant variants of forms that are discovered as forms are expanded). Due to the way local-expand adds identifiers to the stop list, that means I’m trying to emulate what the expander does when it sees core forms like #%plain-lambda and letrec-syntaxes+values.

What I’ve been doing is creating a first-class definition context to hold the bindings, then recursively calling local-expand on the body of each form until the program is fully-expanded. Then I replace the binding identifiers with equivalents that have the definition context’s scope added to them.

This seems to work okay when I control the language (in the case of Hackett), but I’ve been running into some situations in which the scopes don’t seem quite right when I’m expanding some racket/base forms (in the case of mutation testing), so I’m trying to ensure I’m jumping through all the appropriate hoops.

I’ve also been running into confusion about where to insert syntax-disarms and syntax-rearms, as well as whether or not what I’m doing is actually safe with regard to the syntax taint system, but maybe I can postpone that question for another day.

Would it be easier/better/relevant to revisit the question of automatically add to the stop list? I think it’s probably not always necessary with scope sets, so maybe we should add a way to disable the additions.

After spending a lot of time over the past few days thinking about this, I’m not sure it would work, but I’m also not sure it wouldn’t—I don’t really know enough about the expander to say for sure. But from what I think I understand, the problem is that partial expansion doesn’t interact well with let-syntax bindings, since if the body of a let-syntax is partially expanded, residual uses of the syntax bindings will be out of context when the form is subsequently expanded. Is that accurate?

Hm, yes… the primitive letrec-syntaxes+values form currently doesn’t support expansion that keeps the letrec-syntaxes+values; and if it did, I guess there would have to be some way of avoiding multiple evaluation of its RHSs.

@robby FWIW, DrRacket background expansion got stuck again on my machine (for the first time in a while), but I don’t think I have the relevant instrumentation inserted for that to be helpful. If it would be helpful, I can insert the instrumentation if you tell me what to do in the case that it happens again.

I think the fundamental issue makes sense, and I think it’s actually mostly okay for what I’m doing. With Hackett, the only issue I’ve actually run into with my solution is the interaction with forms that care about recursive expansion, like the new implementation of syntax-parameterize. That problem is a bit trickier to solve, though, since it essentially means I want the expander to operate using a different set of core forms. From syntax-parameterize’s POV, it needs its bodies to be fully-expanded, so just propagating the stop list wouldn’t be enough. Rather, syntax-parameterize would actually have to yield to my custom expander code so that I can handle those forms properly.

env PLTSTDERR=info@drracket-background-compilation drracket

to see if it is working, put this program in the definitions window and wait more than a second or two, but less than 100 seconds:

#lang racket
(begin-for-syntax
  (let hi-spencer ()
    (void
     (let hi-shu-hung ()
       (void
        (let dan ()
          (void
           (sleep 100))))))))

There are plenty of other ways to solve the problem, though, if we permit modifying the macroexpander. I guess one might be enabling users to define new forms that are treated similarly to Racket’s existing core syntactic forms, in the sense that the macroexpander would yield control to them when seeing one but wouldn’t expand its expansion. That would allow me to use ordinary recursive expansion instead of doing hacky things with the stop list to emulate recursive expansion with a different set of core forms. I don’t really know how to enforce the appropriate protocol there, though (since it would be easy to construct misbehaving “custom core forms”).

and then add some whitespace somewhere

You should see someone that looks like a stacktrace

I haven’t thought enough about the problem, but assuming that we can’t solve it by adjusting local-expand, I wonder if syntax-parameterize needs to support some way for macros to cooperate with it

the output from drracket-background-compilation: expanding-place.rkt: 00 starting monitors to the end of the stack trace is hopefully helpful for the real bug.

Okay, turning the logger on should be sufficient? I don’t need to patch DrRacket’s code in any way, right?

Yes. Right.

Okay, thanks, I’ll do that.

I think the problem is not specific to syntax-parameterize but rather any form that performs recursive expansion.

It works for Hackett because I control the whole language. I could get away with reimplementing syntax-parameterize to use my custom expansion code, but I’d rather not. It wouldn’t work if I didn’t control the whole language.

It sort of works with mutation testing because I’m not adding any new forms to the stop list. So if a form forces expansion using local-expand, the program will still work, I just might miss some opportunities to insert mutations.

But I think @michael.ballantyne’s point in the GH issue I opened about syntax-parameterize and @notjack’s thoughts here in Slack are on the right track: if people are interested in using local-expand in this way, then uses of local-expand probably need to affect nested uses of local-expand in some way. But that raises questions about both safety and API design wrt backwards compatibility, I think.

(The “custom core forms” idea I mentioned earlier would work for Hackett, but not mutation testing, which to be fair is meaningful. Hackett is doing what I think is a reasonable thing to be doing, but the thing I’m doing for mutation testing is kind of evil, since I’m intentionally screwing with code other macros are supposed to control. So maybe it isn’t a good idea to try and come up with a single solution for both problems just because I’m currently solving them in similar, albeit hacky ways.)

It makes sense that a way for macros to cooperate with syntax-parameterize could generalize to a way for macros to cooperate more with macros that use local-expand. I’m open to experiments, but I certainly don’t know the answer offhand.

Yes, I didn’t by any means expect you to. :) I was mostly just hoping to get an answer to my first question; the rest of this is just handwaving. I think I may be unblocked for now on the mutation testing thing I’m tinkering with, which is good enough. I’m not totally sure what to do about Hackett right now, but I can probably do a different hack in the immediate future (by avoiding syntax-parameterize using either syntax-local-get-shadower or just breaking hygiene with datum->syntax).

@lexi.lambda I think adding support for core forms to the expander is definitely a better way to go than trying to emulate the expander in userspace, given that you can’t wrest control of what a call to local-expand in a macro does. There are lots of uses for custom core forms.

Certainly Typed Racket would find “custom core forms” useful as well

@mflatt @michael.ballantyne @samth From your perspectives, what would the API for a “custom core form” binding look like? Obviously it would be neither safe nor very user-friendly to provide the expander’s add-core-form! API as it is, but maybe it can be done in a way that feels similar to writing an ordinary transformer, with the exception that the result isn’t re-expanded by the expander to avoid infinite loops?

I think my only real question is how users would instruct which parts of a core form ought to be expanded by the expander. Would it be enough for them to just call local-expand on those pieces?

(define-syntax my-core-form (make-core-form-expander (lambda (stx) …)))

Then inside use local-expand and the first-class definition context API.

Yes, that’s pretty much exactly what I had in mind, too. If you think that would work, it doesn’t sound too difficult to add to the expander.

I guess you’d have to handle some error case when a core form appears in a fully-expanded program (but, of course, not under quote-syntax).

If there are custom core forms it is no longer the case that the result of expansion is necessarily a racket core form or that it can be be represented by the expander in its “parsed” AST form.

When/where are the parsed AST forms used?

Having a little difficulty tracking it down exactly, but my intuition is that it expands directly to parsed AST forms rather than syntax objects when the result of the expansion is going to be directly compiled or evaluated.

So eval would do that, whereas a user call to expand wouldn’t.

Last I talked to matthew he seemed to feel that the standard top-level expand function should error if it expands to something other than a racket core form, and that there should be an alternate API for expansions where an extended target language is OK.

I don’t think that would be a problem, but it’s less clear to me whether local-expand should have a variant or the existing local-expand should be okay returning new core forms.

I’m curious to the reason why define-struct/contract does not support the #:methods keyword?

I’m trying to add gen:custom-write to a struct I have protected with contracts via define-struct/contract but alas not possible. I can simple revert to using struct but is there any other way to attach contracts to a struct? So far it appears I have only two options: define-struct/contract or (provide (contract-out (struct ...))). I don’t want the provide option since I’m not exporting the struct outside the module but I still want to protect it with contracts because it’s handling critical data.

@abmclin I think define-struct and define-struct/contract ought to be considered deprecated (in the context of #lang racket and #lang racket/base at least)

You can put the struct in a submodule

ah, that would work

you could also use with-contract, but you’d have to list every identifier bound by the struct individually and I think it would shadow the static struct info binding (not sure on that one)

ok