Sad. Something about a recent commit to the expander has broken nanopass.

Also, trivial and rosette seem to have broken as well.

that could explain why I couldn’t install css-expr (also built with nanopass) with a recent snapshot

@stchang @alexknauth I think I’m switching back to syntax objects for my type representation…

@lexi.lambda what happened?

oh, sorry for not pinging you as well… basically, I think I sometimes need to be able to embed expanded types in syntax, if, for example, a macro expands to a type annotation form.

but I don’t think I’m going to do it the way turnstile does it. :)

this is the prototype I’ve put together if you’re interested: https://gist.github.com/lexi-lambda/8c05c255542329cc91624a846b200b48

it produces #<syntax:expand-type.rkt:75:13 (#%type:app (#%type:app (#%type:con ->) (#%type:con Integer)) (#%type:con Integer))> as output.

ok, thanks

would prefab structs work too?

they might… I’m currently using prefab structs. but I’m currently just stuffing prefab structs in syntax properties. IIRC, prefab structs containing syntax objects cause problems when embedded in syntax directly.

though I think that’s only true if you’re evaluating the generated prefab structures, actually. so they might work okay.

hmm. now I can’t decide if I should just allow embedding the existing prefab structures wherever types are expected. :)

that sounds convenient :)

let me try swapping this prototype to use prefab structures instead of syntax objects…

@ben the weird thing about embedding prefab structures in syntax objects is that you get… prefab structures wrapped in syntax objects. > (syntax-e (datum->syntax #f (type:app (type:con #'Maybe) (type:con #'Integer)))) '#s(type:app #<syntax #s(type:con Maybe)> #<syntax #s(type:con Integer)>)

I guess that could be fine, but it’s a bit odd.

ok, so might as well just use syntax objects

maybe. now you’ve gone and made me indecisive. :)

well here’s my deal, sometimes at Northeastern I hear the question "why doesn’t turnstile use structs instead of syntax objects? like hackett

“seems we’d get better performance”

and today I see, wow Alexis is thinking about going back to syntax objects!

the embedding problem makes sense

see, the mistake you made is assuming I know what I’m doing

and performance, IMO, should be a second-order problem

but yeah, I have been wondering about the performance thing, too.

I’m worried about both the performance of the pattern-matching and the need to constantly call into the expander to make sure the type is normalized.

I think I want something like syntax-local-expand-expression but with a stop-list… maybe I can emulate something like that using syntax taints.

though maybe not… taints probably wouldn’t do what I want. but I could probably change the expander to give me something that would give better performance if absolutely necessary.

hm .. I think @michael.ballantyne has been adding stop lists to turnstile, successfully (no taints)

the main issue is that I don’t want to have to constantly call local-expand if I’ve already called local-expand on a given piece of syntax.

(pinging him to get a proper answer, but it won’t happen immediately)

since it would be constantly re-traversing the same piece of syntax.

right

but I can’t attach a syntax property saying “this piece of syntax has already been traversed” because a user could end up copying that property onto their own syntax (intentionally or unintentionally).

that said, that seems like a problem to worry about later, as you yourself said…

yeah for structs vs. syntax objects I’d just pick the one that is less work for right now

I’ve been grappling with this problem for the past two months or so. The problem is that types are just phase 1 (POD) values from a certain POV, but they’re syntax from another POV.

They obey a very specific structure, and I want to enforce that structure. But I also need macros to be able to mess with them, put them back together, and put them back into syntax objects as a subform to another macro.

One potential advantage of using prefab structures is that they could be parsed out of the syntax objects by a syntax class, so phase 1 code could still think about them as POD instead of syntax. But all the marshaling/unmarshaling surely has a cost.

I wouldn’t worry about the marshalling cost yet

everything else, I agree it’s not clear what’s the best thing to do

@ben I’ve added a version of the same gist that uses prefab structs https://gist.github.com/lexi-lambda/8c05c255542329cc91624a846b200b48#file-expand-type-prefab-rkt

@mflatt @ryanc I’m interested in making a change to racket/private/template, and I’m trying to understand how it works… specifically, I’m not sure what the purpose of the t-resyntax guide is.

At first blush, it seems similar to t-relocate, but it’s used in a lot more places, and it has a whole optimize-resyntax function that I don’t understand.

@githree Oh, okay. I’ll see if I can figure out what’s up now then.

@lexi.lambda If you interpret a guide as a program that builds a syntax object, t-resyntax marks where the actual syntax boundaries are. For example, suppose x and y are pattern variables, and the template is (x y). That gets turned into the guide (t-resyntax #f tstx (t-list (t-var x-val) (t-var y-val))) meaning: first build a list using the x and y pattern variable bindings, then turn the list into a syntax object with the same lexical context and source as the original template syntax itself (which I’m assuming here is represented by tstx).

If the template were (x (y)) or (x . (y)) then there would be two occurrences of t-resyntax.

It turns out, though, that compiling templates that way makes for .zo files that are larger than we’d like. It sometimes produces more compact code to produce a quote-syntaxed term and then replace parts of it at runtime. For example, if the template is (0 1 x 3 y 5), it’s more compact to have a single syntax constant as the starting point and emit code to update positions 2 and 4: (t-subst (quote-syntax (0 1 _ 3 _ 5)) '(2 4) x-val y-val). That’s what optimize-resyntax does.

I see. I was wondering exactly what the purpose of t-subst was, myself… if I understand correctly, it is designed to do a bit more of the work at runtime to make .zo files smaller?

right

Okay. For context, what I’m interested in doing is effectively making it possible to control where syntax properties come from on the resulting syntax object, like how syntax/loc allows controlling where the srcloc comes from.

I’m not completely sure what needs to be changed, though… it seems like I could adjust t-relocate to accept an additional argument including a syntax object to copy properties from, but it doesn’t look like that would be enough.

For the sake of argument, let’s say there’s a form (syntax/prop prop-stx template) (and forget about source location for now)

If x is a pattern variable, what does (syntax/prop pstx x) mean?

I would imagine it would have the same behavior as (syntax/loc stx x); that is, it’s just #'x.

(btw, is the intent only to apply syntax properties to the outermost syntax object?)

Yes. I’ve basically just written a syntax/loc/props macro a bunch of times (including one that has ended up inside racket/splicing), and I figured it would make sense to put it somewhere.

(The one in racket/splicing does it the “wrong” way, though; it just does (datum->syntax stx (syntax-e stx) pstx pstx).)

The relevant guide forms are t-resyntax, t-relocate, and t-subst. There are two options. One is to just add an extra argument to each of them for the property source syntax. The other is (for .zo-size paranoia mode) is to create separate variants for setting both source and properties (since presumably uses of syntax/loc/props will be less common than syntax/loc for the near future). The other place you’ll need to change is relocate-guide, but it sounds like the main cases are still same.

The API I think I actually want is to add #:srcloc and #:props keyword arguments to syntax to avoid needing syntax/loc, syntax/props, and syntax/loc/props.

But that can expand into whatever under the hood.

@mflatt Is there a way to check that two identifiers are never free-identifier=? in any phase?