Could anyone give me a hand with a confusion about requiring & phases?

:wave:

I have a macro defined by make-require-transformer in one module, and a certain special definition in a second. In a third module I want to require this definition with the special require modifier during phase 1. I’m using (require <module1> (for-syntax (<special require thing> <module2>))), but I get <special require thing>: not a require sub-form

It seems like nesting the require modifier in for-syntax breaks it?

@brendan btw, once you’re using make-require-transformer, then you shouldn’t feel like you need to use the #beginners channel :slightly_smiling_face:

just checking: by “a macro defined by make-require-transformer” I assume you did (define-syntax id <make-require-transformer-call>) instead of (define id <make-require-transformer-call> right?

@samth well that code was written by lexi.lambda

@notjack yes

@brendan require transformers are a little weird in that you can’t do (require <module1> (<require-transformer-defined-in-module1> stuff ...)), you have to import the transformer with a whole separate require statement

is that causing your problem maybe?

Weird, because later on I do exactly that, just not in for-syntax

(at least, I think you can’t… I know you can’t do something similar with provide transformers)

hmm. maybe you can then and I’m just wrong :p

@notjack you used to not be able to, but I fixed it

my next guess is you need to import the transformer for-syntax in order to use it for-syntax

@samth ah gotcha

I was importing it for-syntax

And when I extract the requiring of that module to another require statement it continues to fail

could you share a small snippet of the failing code?

So here’s the failing require

(require (for-syntax racket/base
                     racket/match
                     syntax/kerncase
                     threading
                     hackett/private/typecheck
                     hackett/private/type-reqprov
                     (unmangle-types-in #:no-introduce hackett/private/prim/type))
         racket/promise
         syntax/parse/define
         hackett/private/type-reqprov
         (only-in hackett/private/base τ⇒! τ⇐!)
         (only-in (unmangle-types-in #:no-introduce hackett/private/kernel) String [#%app @%app])
         (only-in hackett/private/prim/base show unsafe-run-io!))

unmangle-types-in is defined in which module?

If I strip out the irrelevant stuff then it’s (require (for-syntax hackett/private/type-reqprov (unmangle-types-in #:no-introduce hackett/private/prim/type)) hackett/private/type-reqprov ...)

ah

@lexi.lambda might have useful thoughts on this question

that seems to me like it should work

It fails with hackett-lib/hackett/private/toplevel.rkt:68:21: unmangle-types-in: not a require sub-form

I changed it to (require hackett/private/type-reqprov (for-syntax (unmangle-types-in #:no-introduce hackett/private/prim/type)) hackett/private/type-reqprov ...)

And now I get a more helpful error

what happens if you do:

(require hackett/private/type-reqprov
         (unmangle-types-in #:no-introduce hackett/private/prim/type (for-syntax hackett/private/prim/type)))

that is, move the for-syntax bit inside the types in so you only have one non-for-syntax use of the transformer?

module: identifier already imported for syntax from a different source in:
  Unit
  (rename hackett/private/prim/type Unit #%hackett-type:Unit)

Unit is defined in that module, but no other modules I require have it

whoops I used hackett/private/prim/type as the imported module in both normal and for-syntax, but your snippet imports hackett/private/kernel in normal phase (and with only-in)

Oh sorry I assumed that was a typo

This occurred with (unmangle-types-in #:no-introduce (for-syntax hackett/private/prim/type))

ah

I’m lost ¯_(ツ)_/¯

I think the issue is hackett-specific

Probably something like that

I tried to just rename the identifier explicitly rather than using lexi.lambda’s magic require form, but now the identifier is unbound

/facedesk I mixed up the orig-id and bind-id

Okay, thanks for the help @notjack! I rewrote it and now my problem is now entirely hackett specific

glad to help!

Is the issue resolved? I’m not at my desk right now, but I can take a look in an hour or two if you’re still having trouble.

@brendan ping

The issue is not resolved, a friend asked me to help her with her code (what a coincidence!)

It seems like it boils down to me not being sure how to import a type (IO) for syntax

@brendan I’m curious why you want to import a type for-syntax

To give better error reporting if somebody does (#:<- x {1 :: Nil}). In my implementation I get the type of the expression given to #:<- and match it against (τ:app τ_fn τ_arg), but then I want to check τ=? τ_fn IO

It works without this check, but the error messages are user unfriendly

so, type bindings are not actually phase 1 bindings. there’s a chance you want (parse-type #'IO).

Oh, right. I saw you do that with String at some point but didn’t realize that’s what you were doing

So should IO be required for phase 1 still?

no, IO is a phase 0 binding. are you familiar with syntax-local-value?

Nope. Sorry

The basic idea is that there are two sorts of bindings in Racket: runtime bindings and syntax bindings. These correspond to define and define-syntax, respectively.

There are also various other binding forms, each with runtime and syntax variants. define-values / define-syntaxes, let / let-syntax, let-values / let-syntaxes, etc.

Each binding additionally has a phase. Phase 0 is runtime, phase 1 is compile-time for phase 0, phase 2 is compile-time for phase 1, etc. But then here’s the question: when you define a macro with define-syntax, are you defining a binding at phase 0 or phase 1?

The answer is subtle. Consider a normal definition. In (define <id> <rhs>), you define a phase 0 binding named <id>, and it’s bound to the result of <rhs>, evaluated at phase 0.

You can shift this up to phase 1 by wrapping the whole thing in begin-for-syntax, so (begin-for-syntax (define <id> <rhs>)) defines <id> at phase 1, and it binds it to the result of <rhs>, evaluated at phase 1.

But what about (define-syntax <id> <rhs>)? Well, this is a bit interesting: it defines <id> at phase 0, but it binds it to the result of <rhs>, evaluated at phase 1.

So in a syntax binding, the binding is at phase 0, but the binding’s value is at phase 1.

Why is any of this relevant? Well, syntax-local-value lets you get the phase 1 value associated with a phase 0 binding.

Is this at runtime or compile time/what phase does this happen at?

It’s at phase 1. So you can write (define-syntax x 'hello), and evaluating (syntax-local-value #'x) produces 'hello at phase 1.

Isn’t the binding only available at phase 0?

In a sense, it is. syntax-local-value feels a bit like a reflective operation. But remember that the RHS really is a phase 1 value, evaluated at compile-time, so the value is available at phase 1.

Okay, that makes sense

This is really important because it means define-syntax essentially bridges the gap between phase 0 and phase 1. It means you can bind things in the phase 0 namespace, which is what people usually expect, but get information out of them at phase 1.

You can think of this as how the macroexpander works, in a sense… it recursively walks the program and finds all the identifiers where syntax-local-value produces a function, then it calls that function with some syntax. That’s a macro transformation.

Just to be clear, I can do (define-syntax x 'hello) (print x) and it will print 'hello at runtime?

No, it won’t. You’ll get bad syntax.

Why is that? Well, when the macroexpander encounters a syntax binding used as an expression, it assumes it is a macro and expects it to be bound to a phase 1 function.

In this case, however, x isn’t bound to a phase 1 function, it’s bound to a symbol. So the macroexpander just barfs.

Oh! I think it just clicked

But that doesn’t mean such a definition is useless! Why? Well, you could write a macro that is given x as a subform, then it could call syntax-local-value on the provided identifier, and it could get the information out.

Like this: (define-syntax x 3) (define-syntax (mac stx) (syntax-parse stx [(_ id:id) #`(quote #,(syntax-local-value #'x))])) (mac x) ; => 3

(I think that should work; I didn’t test it.)

Oops, I forgot a #'.

Okay, fixed. Now it should work.

_: wildcard not allowed as expression?

That means you need to import syntax/parse. :)

The idea seems to make sense though

oh, right

(require (for-syntax syntax/parse))

Yes, it works

It’s a bad error message. I wonder if there’s a way to make it better.

Anyway, this is used all the time in Racket. For example, do you know about match expanders?

Like, the implementation of match?

Like define-match-expander.

no

Hmm… what about syntax classes? Are you familiar with those?

Yes

Checking the docs, match expanders seem to expand the functionality of match

Yeah. Both of them are implemented by using define-syntax to store some phase 1 information, and then match and syntax-parse use syntax-local-value to look up the associated information at expansion-time.

You’ll note that if you use define-syntax-class to define some syntax class, then try and use that syntax class outside of a syntax-parse pattern, you’ll get illegal use of syntax, the same error as if you try and use x as an expression after (define-syntax x 3).

I think I understand

Consider playing with define-syntax and syntax-local-value a bit. One cool, simple use of it that I found in one of Robby’s talks is using it to define two forms: define-enum and enum-case. You should be able to use them like this: (define-enum animal [cat dog fish]) (enum-case animal 'cat [cat 'meow] [dog 'woof] [fish 'blub]) …and enum-case should raise a compile-time error if not all the enum cases are covered.

I should add that to my Racket macro exercises repo. :)

oh dang you have a Racket macro exercises repo?

There’s only 3 exercises currently, haha. I’m trying to collect more!

I should go through that/an analogue

https://lexi-lambda.github.io/racket-macro-exercises/

So I changed my code to use parse-type, but the type-inferer seems to infer a different type than (parse-type #'IO) for (println "Hello, world!") The definition I’m using is [(_ . (#:<- ~! id:id expr:expr)) (match-let-values ([(e- τ_e) (τ⇒! #'expr)]) (match τ_e [(τ:app τ_fn τ_arg) #:when (τ=? τ_fn (parse-type #'IO)) (let ([e-/exec (τ⇐! (quasisyntax/loc this-syntax (@%app unsafe-run-io! #,e-)) τ_arg)]) #`(force #,e-/exec))] [_ #`(format "error: ~a is not an (IO _)" #,(τ->string (apply-current-subst τ_e)))]))] Am I misunderstanding how τ=? works?

Probably. τ=? is not very smart. It does literal type comparison, so if something is (IO Unit), it will not be τ=? to IO. Furthermore, and more insidiously, some unsolved type variable will not be τ=? to IO, even if they could unify. Even worse, (forall [a] a) and (forall [b] b) are not τ=?.

Well the weird thing is what gets printed out is ; <pkgs>/hackett-lib/hackett/private/toplevel.rkt:96.60: parse-type: expected ; type ; at: IO ; in: IO

Have you imported IO at phase 0?

I believe so

Hmm.

So, I thought I had imported it because when I tried to explicitly require it from the module it’s defined in I got the error module: identifier already imported from a different source in: IO (rename hackett/private/prim/type IO #%hackett-type:IO) (rename hackett/private/prim/type IO IO) but now I realize this might be because one of the modules required for syntax could be exporting it without me realizing it

it doesn’t seem like it though

In fact, I can’t find where I’ve apparently already imported it from, but when I try to require it I get that error

Oh! You might be accidentally trying to use the value constructor IO. :)

How could I distinguish those?

IO is defined as (data (IO a) (IO {Real-World -> (Tuple Real-World a)})).

I recently made Hackett multi-namespace with a bunch of terrible hacks. So you can probably get at the type using #%hackett-type:IO… but I wouldn’t recommend it. The #%hackett-type: prefix is supposed to be an implementation detail. unmangle-types-in is the preferred interface.

You probably want to (require (unmangle-types-in #:prefix t: hackett/private/prim/type)), then use (parse-type #'t:IO) or something like that.

That imports all the type-level stuff with a t: prefix.

Working with the two namespaces in single-namespaced Racket kind of sucks.

It might be nice to have punning be unidiomatic in Hackett?

It confuses beginners sometimes

I think Hackett is, currently, somewhat unabashedly not for beginners. :)

That’s entirely true

So, the error has changed to ; <pkgs>/hackett-lib/hackett/private/toplevel.rkt:97.60: parse-type: expected ; type ; at: t:IO ; in: t:IO

Which makes me think that punning was not the issue

Could you post all your requires?

(require (for-syntax racket/base
                     racket/match
                     syntax/kerncase
                     threading
                     hackett/private/typecheck)
         racket/promise
         syntax/parse/define
         hackett/private/type-reqprov
         (only-in hackett/private/base τ⇒! τ⇐!)
         (only-in (unmangle-types-in #:no-introduce hackett/private/kernel) String [#%app @%app])
         (only-in (unmangle-types-in #:prefix t: hackett/private/prim/type) IO)
         (only-in hackett/private/prim/base show unsafe-run-io!))

The (only-in .... IO) is the problem. That’s importing only IO, not t:IO.

wow, that’s obvious in hindsight

Yes, but only in hindsight. :)

Working with the two namespaces in normal Racket is legitimately confusing, and I hope it isn’t something normal users ever have to do.