I’ve worked through the JSONIC language from Beautiful Racket. I then gave myself the task of replacing the delimiters @$ $@ with just parentheses. I’ve managed to build parse-trees properly. For example, (parse-to-datum (apply-tokenizer-maker make-tokenizer "(/ 3 5.0)")) produces (jsonic-program (jsonic-sexp (/ 3 5.0))), which produces expander.rkt> (jsonic-program (jsonic-sexp (/ 3 5.0))) "0.6" as expected. However, when I run this program from a file with #lang jsonic4 #lang jsonic4 (/ 3 5.0) I get ; c:\sys\tolcom\current\little-languages\jsonic4\test7.rkt::-1: /: unbound identifier ; in: / That makes me think the /-binding isn’t available in my language. With Beautiful Racket’s version, I didn’t face this problem and I think it’s because the parser produced string-s-exp as in "(+ 1 1)" which was then converted to an s-exp at compile-time [using with-pattern]. I don’t need to do this because my tokenizer is producing the s-exp [using read]. [That’s essentially the change I made.] I can’t see how this changes anything relative to racket/base bindings. In other words, I’m lost. Any help is appreciated!

It’s worth checking each layer separately to see where your expectations aren’t met. Step one must be to check the tokeneizer. Can you make it print each token as it is produced?

The tokenizer does what I expect. For instance, #lang br (require jsonic4/parser jsonic4/tokenizer brag/support) (parse-to-datum (apply-tokenizer-maker make-tokenizer "(/ 3 5.0)")) produces '(jsonic-program (jsonic-sexp (/ 3 5.0))) I think the expander also does what I expect. If I enter the expander now and execute this on REPL it works because the REPL itself computes (/ 3 5.0) and so jsonic-exp sees 0.6, it converts it to a string and jsonic-program concatenate all strings and trims the resulting one. All of this is seen in the REPL. But when I run #lang jsonic4 (/ 3 5.0) in a file by itself [called test7.rkt], I get ; c:\sys\tolcom\current\little-languages\jsonic4\test7.rkt::-1: /: unbound identifier ; in: / So it’s this last mile that fails. (If I replace (/ 3 5.0) by, say, (list 1 2 3), I get list: unbound identifier.)

Maybe this is too subtle for Slack-message and I’m too clueless to present it properly. I will produce a working-example for whoever has the time to look at this. Don’t worry about this, anyone.

If you have the time only. Thank you! %wget -q -Ojsonic4.zip <https://bit.ly/2PxAuG5> %unzip jsonic4.zip Archive: jsonic4.zip inflating: jsonic4/main.rkt inflating: jsonic4/expander.rkt inflating: jsonic4/parser-test1.rkt inflating: jsonic4/parser.rkt inflating: jsonic4/tokenizer.rkt extracting: jsonic4/test7.rkt inflating: jsonic4/reader.rkt %cd jsonic4 %racket parser-test1.rkt '(jsonic-program (jsonic-sexp (/ 3 5.0))) %racket test7.rkt test7.rkt::-1: /: unbound identifier in: / location...: test7.rkt::-1 %

I actually think you’d need to raco pkg install before. I am probably running jsonic4 from other directory, but the files are the same so the behavior must be the same here too.

@anything I think, “reader.rkt” is missing?

Omg, so sorry about that!

#lang br/quicklang (require "tokenizer.rkt" "parser.rkt") (define (read-syntax path port) (define parse-tree (parse path (make-tokenizer port))) (define module-datum `(module jsonic-module jsonic4/expander ,parse-tree)) (datum->syntax #f module-datum)) (provide read-syntax)

Updated the URL above as well. File reader.rkt included.

I am confused by this: (define-macro (jsonic-sexp SEXP) (syntax (jsexpr->string SEXP))) When I try (jsexpr->string '(/ 4 3)) I get an error in the repl.

@soegaard2 This might very well be where the problem is because this is a core of the new code. The original from Beautiful Racket is (define-macro (jsonic-sexp SEXP-STR) (with-pattern ([SEXP-DATUM (format-datum '~a #'SEXP-STR)]) #'(jsexpr->string SEXP-DATUM))) Beautiful Racket gets a string "(/ 3 5.0)" while this new version is getting a real s-exp, not a string. What I’m trying to do with jsonic-sexp is to let Racket itself compute the s-exp (which should produce a valid jsexpr such as a number) and then convert it to a string.

The reason you get an error is because you’re passing in a list to jsexpr->string. If you remove the quote, Racket computes the division and jsexpr->string gets just a number, which is what’s suppose to happen using the language jsonic4. (But Racket doesn’t compute the division saying / is not bound, which actually confirms me what’s happening — I just didn’t expect / not to be available.)

Okay, you want to allow a general expression (and not just a datum).

Precisely.

The SEXP is a syntax object. It comes from directly from the reader. That means no binding information has been attached. The expander must attach relevant binding information. Since it is a general Racket expression, consider using expand-syntax and then insert the result.

Hm, that does match with the lectures I have been getting… Let me investigate it further now with this in mind. Thanks a bunch!

Datums are easier - they don’t need any binding information.

You didn’t mean this, did you? (define-macro (jsonic-sexp SEXP) (syntax (jsexpr->string (expand-syntax SEXP)))) (provide jsonic-sexp) I get expander.rkt> (jsonic-sexp '(/ 3 5.0)) ; expand-syntax: contract violation ; expected: syntax? ; given: '(/ 3 5.0)

I was thinking of something like (define-syntax (jsonic-sexp stx) (syntax-parse stx [(_ SEXP) (with-syntax ([SEXP (expand #'SEXP)]) #'(jsexpr->string SEXP))])) It doesn’t work though.

If you can’t do it, I should not even be trying. :slightly_smiling_face: (Clearly I can’t tell what’s easy and what’s hard.)

Well, I haven’t read BR yet, so I am missing some context. (define-macro (jsonic-sexp SEXP) (with-syntax ([SEXP (datum->syntax #'here (syntax->datum #'SEXP))]) #'(jsexpr->string SEXP))) This is close.

Now I get the correct result - but twice?!?

And I am still confusing by define-macro - why not use standard define-syntax.

Ah. It works, I forgot I haded added something to jsonic-mb .

You rock. Also getting correct results here. This builds pressure for me to really understand how to use with-syntax, with-pattern. I look at your (syntax here) and I’m like… What? Lol. What’s that “here” doing there?

The problem is that SEXP has no attached bindings.

(datum->syntax stx datum) attaches the bindings from stx to datum.

So #’here is a way to get the bindings available where jsonic-sexp is defined.

That’s making some sense. Here’s what I’m thinking. When jsonic-sexp runs, we have racket/base available, but jsonic-sexp has a s-exp coming from the user, which does not have any bindings at all in it. To put racket/base bindings in the user code, we need to enrich its syntax object, which is what you’re doing.

Sounds right.

Are you using (syntax here) merely to tap into the compile-time phase used by jsonic-sexp and get its bindings? The procedure datum->syntax gives “the lexical context information of (syntax here)_”._ What’s the context information of (syntax here)? It must be the same context of jsonic-exp, which includes all of racket/base’s binding — I say.

Yes.

Thanks for taking the time to look into this. This is priceless. It takes knowledge and time and other things that are also even more priceless. Thank you!

You’re quite right. I was taught that by Soegaard’s tutorial itself, but I had forgotten by then. Beautiful Racket taught me the same, but I had also forgotten it by then. It’s been weeks I’ve read both. I have been slowly working out exercises for myself and by the time I tried this one I forgot the lesson already. That’s the beginner-thing: at first we aren’t yet used to all the little parts that compose the world, so we focus on one of them and some other we already forgot part trips us up. Thanks for your insight here!

Preaching to the choir there! I’m still a racket beginner, i just remember the bindings stuff tripping me up. Otoh i was sure this was a source of confusion (what gets evaluated when), so wanted to chime in on that