I’ve upgraded all of the native libraries to new versions, so watch out for problems – either package/build problems or regressions due to the actual upgrades

One problem: the package manager will install “draw-<platform>–3”, which replaces the dependency on “draw-<platform>–2” for “draw-lib”, but it won’t automatically uninstall “draw-<platform>–2”. If you’re updating an existing development build, you may need raco pkg remove --auto to clean out “draw-<platform>–2”. (The package manager doesn’t currently detect that those two packages conflict by installing libraries with the same names. It’s likely that "…–3" libraries will overwrite "…–2" libraries, but it’s probably not guaranteed.)

@alexknauth I noticed you forked Hackett this morning. I’m not sure if you care, but fwiw, my (currently very broken) work in progress branch replacing the type representation is wip/syntax-typerep.

I see it

Good! I figured you would, but just making sure. I’m a little curious why you forked it in the first place. :)

If I end up using my macrotypes-nonstx repo to make an extension to Hackett, I want the prefab-struct type representation to be an option, since that’s what most of macrotypes-nonstx was designed for.

I think I see. I’m not sure the prefab representation is unworkable, and if you manage to come up with something, certainly let me know. I’m feeling a bit better about the way I’m planning on using syntax objects here, though, since I’m still in control of the type language (unlike the way turnstile currently does it, which doesn’t use a stop list).

@lexi.lambda congratulations on your new PLT@Northwestern position!

Not that I am selfish but this is great news for Racket (and me by extension ;) I found it to be extreme misallocation of resources that people like Alexis couldn’t work on Racket full-time. Glad this is being fixed! Congratulations Alexis, Northwestern and Racket!

I am having some issues with understanding how I can introduce temporary variables into my macros. I am basically compiling my own language to Java and want the output Java to not have any compound expressions.

For example: let’s say I have (def x (= (+ 2 4) (+ 3 8))) and want to expand it to something like (list (def t1 (+ 2 4)) (def t2 (+ 3 8)) (def x (= t1 t2))).

The catch, though, is that I don’t actually know the variable name that I will need when handling an arbitrary depth of expressions.

I attempted to capture the temp symbol holding the final value of the whole expression in a syntax property, but this is no longer available (as far as I can tell) once the macro has expanded. In other words, the outer macro where I need the value has already expanded its template.

So, is there some better way to reference this syntax property?

If not, then would the best alternative to be to introduce generate temp values, then inject them into an intermediate form along with the original expressions, then use this to generate the subexpressions? I.e., pass the temps forward through the templates instead of trying to retrieve them backwards through syntax properties?

Or, is there a better approach all together for sharing data in this manner?

A better example. perhaps:

(define-syntax (constraint-def stx) (syntax-case stx () [(_ ID EXP) #’(list EXP (define-constraint ID ???))]))

Here, I put the expression before the definition so that it will be broken down into sub-expressions before the final assignment. In the ??? I need the name of the last temporary variable used.

The expression gets expanded, more or less, by this:

(define-syntax (comp-expr stx) (define ltemp (new-temp-symbol)) (define rtemp (new-temp-symbol)) (define exptemp (new-temp-symbol)) (syntax-case stx (=) [(_ LEFT "=" RIGHT) #`(list (val-comp #,ltemp LEFT) (val-comp #,rtemp RIGHT) (define-value #,exptemp (= #,ltemp #,rtemp)))]))

So, if I need to either access the exptemp value in the constraint-def macro to fill in the ??? or I need to create a temp variable (or use the ID, I guess) in the constraint-def macro and make it available to use in place of the exptemp value in the comp-expr macro.

Should I just generate the temp symbol in constraint-def and add it to a syntax property of the EXP?

The first thing you described should work

(expand subexpression, subexpression stores its name as a syntax property, parent expression gets the syntax property)

but you’d need to call local-expand to expand the subexpression

Thanks. I’ll look up local-expand and see what I can do.

@ben That worked wonderfully. Is there some kind of cookbook style website for this kind of thing? I don’t mind digging in and learning this stuff, but it seems rather difficult to even know where to look. If such a website existed, that would be great. If not, would there be much benefit in me writing such a website?

submit a pull request here? http://docs.racket-lang.org/syntax-parse-example/index.html

I’ll take a look

Thanks

Is there a cleverer way to unit test a single macro which may expand to something which contains unbound identifiers than to do the following: create a test file, import the macro I need to test, implement a dummy macro that will match the result and quote everything?

Also, what circumstances can lead to a single syntax property being returned as a pair?

Racket was fun, but I’m starting to question whether or not this was a good idea…

Is my problem that I am trying to do this stuff without contracts? Would using typed racket be better?

@racket588 not sure if it’s right for your exact use case, but have you seen the syntax/macro-testing module?

I have not.

Let me take a look at that real quick.

@notjack I’ll have to give it a shot, but it sounds like what I’m trying to do.

@racket588 also see the expand-once function and similar forms, that’s what I used to build the macro testing utilities of my expect package

That also sounds like it might be good for what I’m doing…

@notjack Have you read “Programming Languages: Application and Interpretation”?

No, but I’m vaguely aware of it

I worked through “Beautiful Racket,” and am finding that it’s coverage is not even remotely sufficient for what I’ve gotten into (and the creation of custom macros that have to be reverse engineered when you need to remove the training wheels has caused some problems). I was just wondering if that would provide a better basis for learning these types of things.

Or if there is a better resources.

My searches so far seem to indicate that my options are: papers, documentation and read other people’s code.

Which is fine, but very time consuming.

I can’t speak to PLAI but these are the resources I personally have found helpful for learning how to structure nontrivial macros:

• The examples section in the syntax/parse documentation
• The “Fortifying Macros” paper
• The “Keeping It Clean With Syntax Parameters” paper
• Some of Ryan Culpepper’s blog posts and miscellaneous articles on macros
• The turnstile library and the paper on it

And the mailing lists

More approachable resources would be wonderful

The set-of-scopes documentation was also very helpful

also hackett

Yeah, I was digging through hackett the other day to try and figure some things out

@racket588 re: properties becoming pairs, see the documentation at the top of this page on how properties are merged by the expander http://docs.racket-lang.org/reference/stxprops.html

Is this the set-of-scopes stuff you meant? https://www.cs.utah.edu/plt/scope-sets/

yes

re: Hackett, is it possible to define macros, etc. using Hackett? Like, would that basically allow me to get the magic of hygienic macros, etc. that make Racket nice while allowing me to basically use Haskell?

Not yet, but eventually. But there are lots of problems to solve first.

darn.

There is not really any theoretical reason you couldn’t (require (for-syntax hackett)) and do business as usual, but you’d be stuck with nothing but syntax-e, datum->syntax, and quote-syntax, which are sort of a Turing tarpit when it comes to macros.

Figuring out how to properly typecheck all the nice things Racket has for writing macros probably isn’t trivial.

I was poking around the other day to see if I could do racket style macros in Haskell and hoped maybe I was just asking the question backwards.

I think the answer to that question is that it’s impossible. At least this way it’s possible, just not there yet. ;)

That would be nice.

re: pairs, I sort of suspected something like that when I inspected the syntax objects and saw the “source(?)” property, but I wasn’t able to reproduce it.

I’ll have to figure out where exactly the two properties are coming from

In the meantime I’ll just use car and feel guilty

the basic idea is that syntax properties are kind of version controlled by the macro expander

macros can’t destroy information about what the syntax property was before they were invoked

The fact that the merging is not controllable by the programmer is absolutely unfortunate.

It’s hard to know how to give more control with a pleasant interface that is backwards-compatible with the existing implementation, though.

¯_(ツ)_/¯

The weird part for me is that I am handling the properties in the exact same way in my macros, but I get pairs in one place and not the other. So something outside of those two macros is introducing this and I have to get my head wrapped around things a bit to figure out that.

it’s the expander itself that introduces that

as in, the racket code that recursively crawls the syntax tree and calls your macros

So it’s not weird to get a pair of, for example (’temp1 . ’temp1)

That means the input syntax object to your macro and the output syntax from your macro both had 'temp1 for that property’s value.

I.e., the exact same property from the same location

Yeah, that was my understanding, but I don’t see how (yet)

Could it be because I generate a list of expressions from a single expression at some point…

I don’t do that in the macro in question, but maybe something before or after that macro does

• your macro is a function from syntax object to syntax object
• expander sees a use of a macro in the syntax object it’s currently expanding
• expander notes what syntax properties exist on that syntax object
• expander calls your macro
• expander see what properties exist on the syntax object returned by your macro
• expander alters those properties so they’re now a pair, where the first item of the pair is the value of the property on the returned syntax object and the second item of the pair is what the value of the property was before calling your macro
• tada! now macros that erase syntax properties (because they construct syntax objects without them) don’t accidentally destroy useful information needed by later macros

(I might have gotten some details wrong on how the pair / list is constructed during that “merge” operation but that’s the spirit of it)

“expander notes what syntax properties exist on that syntax object”

the duplication you see in a property value '(temp1 . temp1) happens because your macro likely directly returns a syntax object it was given as input

by “notes” I just mean “remembers in some local variable somewhere in the expander’s implementation”, I don’t mean that it alters the syntax object in any way there

Ah, so, for instance: (syntax-case [(_ EXP) #’EXP)])

where the EXP then gets expanded and receives a property in another macro?

@racket588 Here’s a tiny program that demonstrates the behavior: #lang racket (require syntax/parse/define) (define-syntax-parser m1 [(_) (syntax-property #'(m1 (+ 1 2)) 'x #t)] [(_ e) (syntax-property #'e 'x #t)]) (m1) If you look in the macro stepper, the resulting (+ 1 2) expression has '(#t . #t) for the 'x property.

in spirit, but I think that might not do that sometimes? I forget the details of how pattern variables affect properties

Alternatively, change the first #t to 1 and the second #t to 2 to get '(2 . 1).

Yeah, I just don’t see where it’s happening in my code (yet). Could it be related to the use of local-expand?

local-expand will do this yes

Depends on your code.

because it calls the expander

but it depends on how you’re using syntax properties and how exactly your macros construct syntax objects

also, there’s a special property called 'origin that works like a normal property except the expander adds it automatically, it tracks the history of what macros expanded a piece of syntax

@racket588 the temp symbols thing you’re doing makes me suspicious of whether adding static info to an identifier would be simpler than using syntax properties in your case

or something with syntax parameters

I need to work on finishing this band-aided prototype, then in a couple of days I’ll see about doing how I should have done it :slightly_smiling_face:

entirely reasonable :)

Once I have more than a couple weeks of Racket under my belt

And read some papers, etc.

Can you provide an MCVE? https://stackoverflow.com/help/mcve

@lexi.lambda re the discussion on your syntax-parameterize issue: the fact that local-expand doesn’t propagate stop lists is massively surprising to me

@lexi.lambda when, I get a chance I definitely will. I was unsuccessful in reducing the code while still reproducing previously and gave up.

@notjack Based on the current API, what would it mean to propagate stop lists? What happens when a local-expand inside another local-expand provides its own stop list?

“should the expander stop” seems pretty monoidal to me - nested uses mean the expander stops more often

when would that not be what you want?

“more often” meaning “when either use says expansion should stop”

> ryanc is typing uh oh

@notjack the point of calling local-expand is to turn a form into something from a known grammar; if the context can add things to the stop list, the macro doesn’t know the grammar of the result, and it has no hope of interpreting the result of local-expand in the general case

yes, I’d consider that a contract violation between the macro calling the outer local expand and the macro calling the inner one - the former gave the latter an expansion context where it wasn’t allowed to acquire the information it needs

For a concrete example, suppose the macro does head-expansion (stop list = #f) and assumes that only things starting with define-values or define-syntaxes can be a definition. If the macro’s context can add define, or struct, or whatever to the stop list, it breaks that macro’s assumptions.

right

a macro calling local-expand with a stop list can also assume that some things aren’t expanded because of the stop list (e.g. macros contained within a use of #%expression), and that’s kind of the opposite sort of problem

I think my general train of thought is uses of local-expand where you only want to expand something into a partially known grammar

All of the uses of local-expand I can think of pretty much break down into three categories: full local-expand (empty stop list), head expansion (which yields a grammar with unknowns but where the outermost structure is known), and head expansion with extra subforms (eg class and init/public/private/field/etc, unit and import/export (?)). And I guess there are is a fourth class where there’s a protocol between the expansion context and the macros (eg all primitive forms expand to quoted expressions) to trick the macro expander into doing things it wasn’t really designed for originally.

It sounds like you have in mind a protocol or agreement between outer macros and inner macros, but I don’t think that really fits the use cases the local expansion mechanism was designed for. I tend to think of the “stop list” as a bit of a red herring: it invites people to think there’s more flexibility there than there is in practice.

On the other hand, that’s my perspective from “classic” macros. You might discover that stretching the macro expander beyond its original intentions to do new and cool things with languages can make using these mechanisms (local-expand, the stop list) in unusual ways make sense.

@ryanc yes I think that’s exactly what I have in mind, this helps clarify things for me

@ryanc was #2033 merged?

@ryanc awesome, looks like it was.

I know this has probably been asked a thousand times, but how do I write a callback for EnumWindows?