I have another beginners’ question: How do Racket macros differ from Common Lisp macros? Any links where I can find their difference described?

I suppose Racket macros are more advanced, but would like to understand how more / what are the differences.

Thanks

1. Common Lisp uses (bare) s-expressions to represent syntax. Racket uses syntax-objects.

Syntax objects carry information on lexical context and source locations (line and column numbers) on each piece of syntax.

1. The “default” in Racket is to respect lexical/static scope - macro introduced variables does not shadow local variables.

The property of 2. is called “referentially transparent”

Actually I think I am confusing names.

http://scheme.dk/blog/archive/2006_05_01_archive.html

@soegaard2 Isn’t property two called hygiene ?

My interest in Racket is as a tool for meta-programming on other platforms, like for the JVM or the Beam (Erlang). What do you think about this? Is Racket suited for this task? Is anyone using Racket like this? What is your opinion?

So the idea is to use Racket for language-oriented programming on other industrial-strength platforms which do not have any or any good meta-programming functionality. Can Racket fulfill this task? Do you see any limitations?

For example, I work on Elixir these days, which has a very similar macro system to that of Racket, but Elixir seems only for meta-programming on Erlang.

I see Racket for meta-programming for anything. Is that so, or am I thinking too much about Racket?

Standard Racket macros will eventually expand into “Core Racket” aka Fully Expanded Racket.

The compiler/JIT then compiles to the current Racket backend.

If you want to run the result on the JVM, you can’t reuse the Racket compiler/JIT.

However what you describe sounds to me somewhat similar to what I tried to with Urlang.

Urlang let’s you write JavaScript with Racket syntax. This allows the user to define new forms (macros) via the standard Racket macro machinery.

As an example:

(define-urlang-macro cond (λ (stx) (syntax-parse stx [(_cond [else e0:Expr e:Expr ...]) #'(begin e0 e ...)] [(_cond [e0 e1 e2 ...] clause ...) (syntax/loc stx (if e0 (begin e1 e2 ...) (cond clause ...)))] [(_cond) (raise-syntax-error 'cond "expected an else clause" stx)])))

Here cond is being defined in terms of (JavaScript) begin and if.

When the Urlang macro is defined, the identifier cond is associated with the above transformer.

Later when the macro is applied, the “mark-transform-mark” method is used.

(define (parse-macro-application ma) (debug (list 'parse-macro-application (syntax->datum ma))) (syntax-parse ma #:literal-sets (keywords) [ma:MacroApplication (define mark (make-syntax-introducer)) (let ((transform (attribute ma.transformer))) (mark (transform (mark #'ma))))]))

@mario.luis.guimaraes If you want to know how Scheme macros can be implemented. A good start is: https://www.cs.indiana.edu/~dyb/pubs/bc-syntax-case.pdf

“Syntactic Abstraction: The syntax-case expander” by R. Kent Dybvig

another example of source-to-source compilation on top of Racket: https://docs.racket-lang.org/magnolisp/index.html

@soegaard2 @githree The idea is to have implement the target language’s “forms” directly in Racket, and then when a program created with only those forms gets to run it generates the equivalent program in the target language. Don’t no if Urlang or Magnolisp work like this, but basically this is my idea to use Racket for meta-programming on top of other platforms. The idea is also to map compile-time errors in the target language back to the same forms in the source written in Racket.

So I do not need “Core Racket” or Racket’s compiler/JIT per se, other than to generate/interpret the Racket program that will generate the target platform’s program

Then, having this translation base, I believe the rest of Racket’s macro/lang machinery could be used on top of this base to generate new langs / DSLs that would generate to this base, and then this base to the target platform. That is, the target platforms would be new “backends”, but not backends in the traditional sense for the “Core Racket”.

Is this possible? Do you see any issues?

I think the idea is sound.

Not sure it is the simplest approach.

No way to find out, but trying it.

So is this different from what you did in Urlang? What is Urlang’s approach?

Do you or others use Urlang for real world (i.e., non-trivial outside academia) projects?

If yes, or not, what is / was your experience using Urlang in such projects?

If you no longer use Urlang, why?

I simply have a macro called urlang used like this: (urlang ..some JavaScript written with s-expressions..). The transformer thus gets the entire piece of JavaScript as a single syntax object. Then a traditional compiler (written using nanopass) compiles it to JavaScript.

Next school year I am taking a sabbatical in order to write a math website. I intend to use Urlang for this.

Ah, ok, it is clear now; that is a different approach then.

Thanks for your responses :thumbsup:

Is is there anyone from Racket’s core team that could add something to this discussion of using Racket to target other platforms?

One issue with your approach: when a user writes a macro that produces form in your “target language in Racket” he needs to be careful, not to use “non-target-language” forms.

Maybe you need to define your own version of define-syntax, that checks for this.

Yep. But what if all the source files start with “#lang <target_platform>“? Does that issue still apply?

Yes.

I thought that defining “#lang <some-lang>” would automatically only accept only forms defined in some-lang … :thinking_face:

Another issue: it will be difficult for you to perform any static analysis on compile time. If a macro inserts a variable reference to, say, foo and foo is unbound, then the problem will be detected by the compiler for the target language.

About "#lang <some-lang>". I am assuming that you allow macros to be written in Racket.

So you need to think about compile-time too.

That is still compile time, but there are two compilation phases then: the one on Racket-side, the other on Target-side

Yes the idea is to allow macros to be written in Racket

The idea is for some-lang to “import” all of Racket stuff useful to define macros / other langs, on top of the base-lang forms

So some-lang would be macros-lang + base-lang-for-target-platform

Consider this example, the user wants to write: (define-syntax (target-cond stx) #'(target-if ...yada yada...)) but writes by accident: (define-syntax (target-cond stx) #'(if ...yada yada...))

Pardon me, I may not understand some of Racket’s code, but I will try to figure your examples out ….

so in this case I suppose you are saying that target-if is on the base-lang but the user types if which is well known from Racket, right? In that case if is not in the base-lang-for-target-platform and so is rejected by the definition of #lang base-lang-for-target-platform; or if is understood as a call to a target’s platform function that does not exist, and the compiler of the target’s platform will complain with an error, which gets mapped to the line where the form target-cond appears in the sources.

Does this makes sense, or responds to your doubt?

“In that case if is not in the base-lang-for-target-platform and so is rejected by the definition of #lang base-lang-for-target-platform;”

The question is what scope if has where #'(if ...yada yada...) occurs.

If if is in scope you will not see an “unbound identifier” error.

Well the if is either a macro / function in scope the macros-lang at Racket-side, or it is a “target form” whose scope is defined by the semantics of the target platform. Does this makes sense? Perhaps I am not seeing where you are getting …

Since the “target forms” are implemented as Racket macros - the target forms will be expanded into normal Racket code eventually.

Therefore it will be difficult to check the output of a macro transformer.

But maybe it is a non-issue.

As long as the writer of a macro is aware that he must produce forms in the target language all is well.

The point is anything that the Racket sources generate is a program that once run generates a program in the target platform. So the Racket’s program can only generate forms of the target platform, otherwise the target compiler will complain with its own errors, which the idea is to map back to the source Racket program where are the forms that generated the problematic target forms

> As long as the writer of a macro is aware that he must produce forms in the target language all is well. Yep, for example if he or she tries to write forms that are not accepted in the target language, this is either caught by the definition of #lang some-lang or target-lang, or by the target compiler

I would say my preference would be for those cases to get caught by the #lang some-lang or target-lang, but I am not aware yet if this can be done at Racket-side.

It is the “get caught by the #lang some-lang” I see as a challenge.

> Since the “target forms” are implemented as Racket macros - the target forms will be expanded into normal Racket code eventually. The idea is for this “Racket code eventually” to be the program that once run generates the target program

For example, Elixir is all about meta-programming: in fact, people write a program that when interpreted generates another program

Yes - and that means you can’t check whether the output of a macro transformer is “code that generates a target form” or “just some Racket code”.

Every def... form in Elixir is a macro that generates calls to functions that created the modules, their functions, etc, all in bytecodes at the end

Regarding > It is the “get caught by the #lang some-lang” I see as a challenge and > Yes - and that means you can’t check whether the output of a macro transformer is “code that generates a target form” or “just some Racket code”. I think (thought?) that the #lang LANG machinery in Racket only allows the programmer to use forms defined in LANG :thinking_face:

and that would detect as errors the use of forms not defined in LANG (all this without ever reaching the target platform compiler)

The other way around, that is, if a file starting with #lang LANG allows the use of forms outside of LANG, does not seem to make sense … or perhaps I am considering #lang to be capable of something it cannot …

As a first approximation - yes. It holds for programs that doesn’t use macros.

ok, I have to learn Racket’s macro and lang machinery to know about how feasible what I want to do is possible or not.

A true strong language-oriented environment, in my view, should make this possible

A true strong language-oriented environment, in my view, should make it possible to program to any target platform with ease.

It seems I have to try doing this with Racket, and judge by myself.

There are other attempts at compiling Racket to a different backend https://github.com/vishesh/racketscript

Anyway, from what I have seem, Racket seems the closest around to that “true strong language-oriented environment”, that I am aware of

@d_run My idea is not “compiling Racket to a different backend”, but to use Racket machinery to generate a target platform’s program

https://docs.racket-lang.org/asi64/index.html?q=asi64 ?

Well, I don’t see how an algorithm can distinguish between “code in language S producing forms in language T when code is run” and “some code in language S”.

But again - it might be a non-issue.

@soegaard2 I don’t know if I got what you said … but having some Tool such that S -> Tool -> T is different from having a program of kind S -> T. The Tool box there makes the difference, for example, mapping back errors from T to S, and forbidding the generation of unacceptable forms in T.

@mario.luis.guimaraes it is a different backend, even if the target language is a high level one

Is there any thing out there better than Racket to create such Tool?

@andreiformiga didn’t caught what you mean

Note that the idea is not to translate any Racket program to some other platform, which is what a “new Racket backend” for me means

Then I misunderstood, sorry

and which I believe is what Racketscript is trying to do @d_run

The idea is to use Racket’s language-orietented machinery to generate programs of other platforms

Say, write for the JVM but not in Java …

You said “a program that when run generates a program in the target platform”. To me, this is a compiler :grinning:

write for the Erlang Beam, but not using Erlang

isn’t that Elixir?

@andreiformiga A compiler takes a source and processes it into another result. What I meant was to write a program, whose result is a program that once run produces a program in another platform. This is quite different

@d_run Yes, I have said that previously, it is something much like what Elixir does for the Erlang VM. The idea is to do in Racket more or less similarly but for any other VM

For example, I do not understand why there is LFE (Lisp-Flavored Erlang). LFE has its own compiler. Couldn’t LFE be done only using Racket’s #lang and macros machinery, instead as its author did, writing yet another parser and compiler?

What’s the input to your program that generates programs?

Elixir has a compiler

@andreiformiga I have already said that before (this been a long thread, and you came later). But anyway, my thinking is having a #lang target-lang and to program in Racket for #lang target-lang, where the target-lang maps one-to-one the forms in the real target platform. I suggest anyone interested in this topic to read this thread from the beginning, if we want to have a more productive discussion.

Racket seems the best tool for this job I am aware of, hence I came to this channel to try to understand if that is true or not.

Right, sorry

Still sounds like a compiler to me, but I’m probably missing something

@andreiformiga there is no parsing involved, it is only generating target forms to be sent to the target compiler. So Tool (see above) is different from a typical compiler

As for your original #beginners question, @mario.luis.guimaraes you’d asked: https://racket.slack.com/archives/C09L257PY/p1549968907016000

This blog post is maybe helpful: https://blog.racket-lang.org/2011/04/writing-syntax-case-macros.html

That mainly explores one difference: Racket macros aren’t functions from s-expression to s-expressions. Instead they’re functions from “syntax objects” to “syntax objects”. Where syntax objects are sexprs + srcloc + lexical scope info.

There are other big differences, such as phases for deterministic separate compilation.

The blog post is a little dated (2011) — e.g. these days prefer syntax-parse to syntax-case — but it’s still maybe one helpful thing to read if coming from Common Lisp macros.

Thanks for the useful links @greg

@mario.luis.guimaraes you might also be interested in https://dl.acm.org/citation.cfm?id=3005736

@samth Is that link useful for the Racket vs CL macros initial question, or for the use case I am trying to achieve with Racket (the long previous thread mostly with @soegaard2) ?

the latter

ok, I know you @samth are from the core Racket team. Do you think I can use Racket’s lang-oriented machinery to accomplish my “programming to other platforms in Racket” goal?

Do you see any limitations / difficulties using Racket to do so?

I think that paper is a good start, but I don’t know how the details would work out for your use case

ok, thanks

you might also be interested in http://matt.might.net/papers/ballantyne2014metameta.pdf

How do you provide types from a module in typed/racket?

Well, now provideing types seems to be working straightforwardly. Not sure what I was doing wrong before.