Thank you :grinning:

Hello. I have some questions about writing my own language(s).

I have created a “core” language that is basically just an s-exp language.

I have created another “dsl” language which I want to expand to the core language, and have that language process further.

So, in other words, I want to chain the languages together like passes in a compiler.

I am not really understanding how to do this, though.

When I try the above, it says all of my macros are unbound. I suspect it’s because the macro expansion takes place in the context of the module some-mod which will, therefore, use “core” which is ignorant of my macros.

So, (if that’s correct), what is the proper way to indicate I want to expand all of the “dsl” macros, then treat the result as a module in the “core” language?

If that assumption is wrong, then what in the world am I doing wrong?

@alama what’s the best way to report typos, layout issues with Server Racket?

@racket588 I don’t think I fully understand your question, but I think you would normally just make forms in your DSL language macros that expand into forms in your core language, then let the expander handle the desugaring.

Imagine it like this: I type…

x = 3 and that parses to…

(define-new-var x 3) which I want to then use macros to expand to…

(define-var x 3) which is valid in the core language, but should expand to…

(define x 3) so that it executes in racket

I have a language extension that gets the last two steps, which is my “core” language

I have another language extension that does the first two steps, which is my “dsl” defined in main.rkt

Once I expand from (define-new-var x 3) to (define-var x 3), my main language will no longer expand and I get errors because define-var is not defined in Racket

So, the crux of the problem is how do I do what you said so that the core language macros will be used to further expand the language?

And for context, I want to be able to have multiple backends which can a) output and compute racket, b) generate equivalent java, c) output analysis information about the code. Also, I want to have multiple DSLs that can be used in the beginning.

Which is why I am trying to do this in separate stages.

I know that selecting the frontend and backend is another problem, but just chaining them together would be a big step forward at this point.

@lexi.lambda I hope that makes more sense…

I think you would want to require the module that provides your core module forms where you define your dsl language forms so that they hygienically reference the core forms and the identifiers aren’t unbound.

I think I tried that, but I can try again.

You mean put the require into the template for the dsl macro, right? In other words, it should be in the module that I output from my dsl?

No, the require should be in the module that defines the DSL macro, not inside the macro’s template.

ah, ok

Are you familiar with racket’s hygiene model?

vaguely

To a first approximation, the idea is that scope should “just work”, even for macros.

So if you use an identifier in a macro template, it should be in scope where you define the macro, and whether it’s in scope where the macro is used is irrelevant.

Ok, that got it. Thanks!

I guess I am just thinking of it in a different manner. Instead of just making all the macros available, I was looking at it as more of a pipleine. Take input, get an AST, compile to the core language, then compile to racket.

I think I need to think about the next problem a bit now (swapping out backends and frontends) to make sure I can do it this way, but this will at least help me get ahead with my prototype. Thank you very much! @lexi.lambda

Swapping out backends is significantly harder, unfortunately.

That’s what I’m thinking…

It can be done, but it involves a bit more trickery.

I could have my dsls expand to a module containing my macros, but with the require parameterized, though, right?

So, my actual dsls would be templated themselves

and a macro would plug in the proper required lib

(I’ll worry about that when I need to, haha)

@alexknauth I think I remember why I rejected the solution of just sticking a type in a syntax property… sometimes the type is not self-contained. For example, imagine I have the type (Foo x), where x is intended to be a type variable bound by some forall. I want to arrange for the type to be under a forall [x] so that the final piece of syntax is (forall [x] (Foo x)). Now, when this expands, forall will introduce fresh bound variables to end up with something like (let-syntax ([x (type-var-transformer (type:bound-var #'x1))]) (Foo x)), and the whole thing eventually becomes (type:forall #'x1 (type:app (type:con #'Foo) (type:bound-var #'x1))).

But this doesn’t work if I don’t have (Foo x) as syntax, but instead only as a prefab structure, since it means I actually have something like (type:app (type:con #'Foo) (type:bound-var #'x)). So if I stick that on a syntax property, I’ll end up with the let-syntax wrapped around a piece of syntax with that syntax property… and the resulting type I’ll get is (type:forall #'x1 (type:app (type:con #'Foo) (type:bound-var #'x))) (note the final #'x instead of #'x1).

I really do need that use of #'x to be expanded, but the prefab structures are already fully expanded by definition. The problem here is that I essentially want to embed an unexpanded type, in this case the identifier x, inside of a type that has already been expanded, in this case (type:app (type:con #'Foo) _), where _ represents a hole to be filled in by my macro.

But wait, there’s more: the type my macro receives is actually (type:app (type:con #'Foo) (type:bound-var #'a1)), and my macro essentially wishes to transform that into #'(#t(type:con #'Foo) x), where #t represents the embedding of a prefab type into syntax by attaching a syntax property. But it doesn’t know what shape the input type will be, only that it needs to substitute all bound #'a1 variables with x. So while I could theoretically provide a way to convert prefab types into syntax objects, including a protocol that could properly encode all the various different types that can show up in the prefab structures, that would end up producing something like (#%app (#%type:con Foo) (#%type:bound-var a1)), and the macro author would then need to reimplement bound variable substitution on syntax objects (since the typechecker’s inst function only works on prefab structures).

So, as far as I can tell, this dualism in the type representation becomes a problem as soon as you want to embed an unparsed/unexpanded type inside of a parsed/expanded one. It would be possible to create some hacks around this issue, like allowing an escape hatch in the prefab type language that allows for “3D types” (which permit arbitrary values in the type language in the same way 3D syntax allows arbitrary values in the syntax language), then a type->syntax function that uses syntax values inside 3D types directly… but that’s a lot of hacks to preserve the prefab structure representation.

Now, to be fair, I currently only have one use case for this technique of embedding an unexpanded type inside an expanded one (the use case is typeclass deriving), but I get a lot of value out of the prefab representation everywhere else. So maybe it will turn out that those tradeoffs are worth it. It’s hard to know without actually doing any real programming in Hackett.

(@ben, you might be interested in the above as well.)