Is there an easy way, in a syntax-parse macro, to accumulate named elements of some body, like this?

@bkovitz I think it depends on what you want to do with each variant

there’s some examples in the syntax parse docs that seem relevant: http://docs.racket-lang.org/syntax/varied-meanings.html\|docs.racket-lang.org/syntax/varied-meanings.html

@bkovitz You can do that using ~alt, like this: #lang racket (require syntax/parse/define) (define-syntax-parser gather #:datum-literals [is-a archetype] [(_ {~alt (is-a ~! parent-expr:expr ...+) (archetype ~! archetype-expr:expr ...+)} ...) #'(hash 'is-a (list parent-expr ... ...) 'archetype (list archetype-expr ... ...))])

> (gather
   (is-a 'number)
   (archetype 42)
   (is-a 'given)
   (archetype 'number 'target))
'#hash((archetype . (42 number target)) (is-a . (number given)))

Each variant needs to be handled somewhat differently, but the main idea is to accumulate them all into a hash table for use elsewhere in the program. Most of the values in the hash table would be lambdas whose arguments are specified right after nodeclass.

Hmmm! This looks good!

I spent a while looking at that page and didn’t figure out how to put the parts together to make exactly this little “gather” test. So that’s what ~! is for.

The ~! isn’t actually necessary there; it will just give you better error reporting.

The example I demonstrated will do the same thing without the ~!s, but it will affect error messages produced by misuses of gather.

Oh, OK. And does #:datum-literals make it so you don’t have to bind the literals somewhere else?

Yes, but in this case, you should probably use #:literals; I was just being lazy.

Ah, OK.

#:datum-literals recognizes the literals by their symbolic name, just by doing a check like (eq? (syntax-e #'some-id) 'some-symbol).

#:literals recognizes the literals by their binding, by doing a check like (free-identifier=? #'some-id #'another-id).

I still don’t get the difference between them. Why would there be literals bound to something?

The advantage of recognizing literals by their binding is that users can rename-in the literals, and the macro will do the right thing!

If you’re open to using a different surface syntax, you could consider not using literals and instead use keywords. Something like (nodeclass (target a) #:is-a number #:archetype n #:is-a given #:archetype number target). Partly a matter of taste, but also the matching is simpler, and you avoid needing to think about literals and bindings etc.

oh, clever trick

For example, you can rename-in else from racket/base and cond will still treat it like else. It also means that if a user shadows else with a local variable binding, they can use it inside of cond, and it won’t be treated like a keyword.

@lexi.lambda So that’s why! That needs to go into the documentation. :slightly_smiling_face:

@lexi.lambda that’s nice

I never realized it was possible to do that

Sometimes you really do want #:datum-literals, if there’s no ambiguity with expressions and if users would never want to rename the names, but in the case that you’re using a literal to distinguish between a keyword and an expression (like cond does with else), you should use the binding, not the symbolic value.

doing it by binding also means drracket’s binding arrows recognize the literals

When I tried making a little test like gather, I’d run into problems because parent-expr and archetype-expr would be undefined in an invocation without an is-a or archetype clause. Why does your gather predefine them (correctly!) to empty lists?

The key is the ... after the {~alt } clause.

Oh. Now I see.

This is documented here: http://docs.racket-lang.org/syntax/More_Keyword_Arguments.html

Do the curly braces have special significance?

No, I am just weird.

Ah, OK. I still haven’t gone over the entire syntax-parse language; still learning all the various pieces…

It’s really big! But you can learn it as you go, so you don’t have to learn it all at once.

(I think I learned syntax/parse over the course of multiple years.)

I can confirm I learn new things about syntax/parse pretty often

OK, good. I’m glad to hear it can be learned incrementally. A valuable property of a programming language!

My fault for typing it too soon, and it scrolled by, but I want to suggest again that a “more Rackety” way of handling keywords is to use #:keywords. :slightly_smiling_face: Like, in hindsight, the else in cond probably should have been #:else.

For a while, when I first found syntax-parse, I basically just used it as a slightly nicer pattern language for syntax-case. I used with-syntax forms in the body to create local pattern bindings… which, as it turns out, is totally unnecessary! You can just use syntax/parse #:with clauses! There’s a lot there to discover.

Whew, this is good news that the ... after {~alt } predefines all the attributes with ellipses to '(). Is there a page that explains that—and whatever other handy predefinitions occur?

I went through exactly the same experience :stuck_out_tongue:

@greg Now looking up #:keywords…

It’s not so much that ... magically makes unspecified attributes the empty list as it is that (foo ...) defaults foo to '() if no foos are given.

That is, a repetition with no elements is just an empty repetition.

I recon there is an ongoing and everlasting war in the mailing list about whether keywords are Good™ or Bad™, and it cycles back every year or so.

@bkovitz If you prefer the surface syntax you have, that’s great, and everything @lexi.lambda explained is helpful for that and important to know in general.

I’m not saying your surface syntax “ought to be” “more Rackety”. That’s your choice.

@lexi.lambda OK, good. That clarifies the “default” attribute values. I don’t have my original version anymore, but somehow I kept getting undefined attributes. Problem solved!

@greg Do you know where #:keywords is documented?

I would probably use keywords like #:is-a because it’s simple to match on those, and you don’t need to worry about how they’re bound (they can’t be). Also, I would probably write a macro where #:is-a can only appear once, and takes zero or more elements. And then maybe write another macro that allows #:is-a to appear multiple times, with one element each — and that macro’s only job is to do the “gathering” and then call the more basic macro. I like to break things into steps like that.

To clarify: I don’t think @greg was talking about a specific #:keywords option to anything, but rather just suggesting to use keywords instead of literal identifiers.

@bkovitz https://docs.racket-lang.org/guide/keywords.html#(tech._keyword)

@bkovitz keywords are basic elements in racket, except in macros, where they have to be “emulated” by using (~seq #:my-keyword value)

I think @bkovitz knows what keywords are and just misinterpreted @greg’s suggestion as talking about a specific option to something named “#:keywords”.

Yes: https://racket.slack.com/archives/C09L257PY/p1541523178068900

Interesting. I’d certainly prefer to go “more Rackety”, just on the general principle that it’s usually better to go with the grain of the language. The current working version of this only uses macros a little bit. It just assembles a bunch of structs, and you can even make the structs and bind them to identifiers outside of a farg-model-spec and incorporate them within a farg-model-spec. I like the readability and simplicity, but I’ll rethink it based on what you’re saying here.

I don’t mean to distract or overwhelm you with too many choices!

@greg Ohh, so you’re saying that maybe farg-model-spec could be a plain ol’ function that takes keyword arguments rather than a macro?

I find choosing between s-expressions and keyword options for small DSLs like that to usually be a matter of whether I’m dealing with a list of options or an actual tree structure

I guess it’s a matter of style, yes

Do you choose between s-exprs and kws though? You can freely mix them

@lexi.lambda Oops, yes, that’s right. I had misinterpreted, and you straightened it out.

specifically, when choosing between this:

(nodeclass (target n)
    (is-a 'number)
    (archetype n)
    (is-a 'given)
    (archetype 'number 'target))

or this:

(nodeclass (target n)
    #:is-a 'number
    #:archetype n
    #:is-a 'given
    #:archetype 'number 'target)

I usually choose to use keyword options for any clause type that doesn’t allow other nested clauses inside it

@notjack Interesting. Indeed the elements here form a list, not a tree structure.

You can also do #:is-a (number) or (#:is-a number) or whatever, @notjack

I can :) but macro readability is important

I agree. It’s context-dependent. What you want as a delimiter matters too.

if I were to use s-expressions in your use case @bkovitz, I’d probably collapse multiple instances of the same clause type into a single clause:

(nodeclass (target n)
    (is-a 'number 'given)
    (archetype n ['number 'target]))

also I’m not sure what semantics your nodeclasses have, but it looks odd to me that things like number and target are quoted - the macro seems to use them like bound names of some sort (like how struct uses the field names to determine accessors)

@notjack Do you mean to write a macro that rewrites the original farg-model-spec so that there is only one instance of each clause, or to restrict the syntax so that each clause can be provided only once?

I wonder about this, recently: Should I make my macros look like s-expressions so that they can be easily used without breaking the consistency of the code around them, or should I ensure that they look “macro-y”? For example, when I write html as s-expr, and I want to include a special element in the page, I’d like my macro to look exactly like if I was writing HTML s-expr, so that it doesn’t break the flow. (div ([class "row"]) (div ([class "col"]) (my-super-calendar #:min-date "yesterday" #:max-date "tomorrow")) (div ([class "col"]))) versus (div ([class "row"]) (div ([class "col"]) (my-super-calendar ([min-date "yesterday"] [max-date "tomorrow"])) (div ([class "col"])))

(What I mentioned above was — do both. Write the easy thing first. Then write the desired thing using the easy thing. :slightly_smiling_face: Here the easy thing is probably what @notjack is suggesting.)

@bkovitz I would probably not disallow multiple uses of the same clause, but document for users that using multiple instances unnecessarily is not recommended so there isn’t unnecessary style deviation. And this is in a hypothetical world where I’m defining the nodeclass syntax, not you :)

I also find it “wrong” that number and target are quoted. I’m hoping to remove that in this next version. It’s a little tricky because those refer to other nodeclasses, which might not be defined yet—and might not even be defined in the present spec. The present spec might need to be combined with another spec to get definitions. (For now, I’m just going with whatever is simplest to implement.)

@jerome.martin.dev wouldn’t you anyway use ` and , in this case to make it clear where you invoke a macro?

@bkovitz you can quote the identifiers inside the macro instead, if required

@macocio That’s what I’m currently struggling with. Should I hide quoting/unquoting or keeping it obvious?

@macocio Ah, that sounds like a good simple solution.

@jerome.martin.dev I’d make it as simple and obvious as possible. See the macro in its own right basically. What’s the best interface without thinking of the context it’s used in?

@jerome.martin.dev because html attributes are arbitrary key-value pairs so there’s no use to having a key like class be a bound identifier, I’d probably just use keywords for everything there:

(div #:class "row"
  (div #:class "col"
    (my-super-calendar
      #:min-date "yesterday"
      #:max-date "tomorrow"))
  (div #:class "col"))

non-obvious -> obvious haha

also that way I’d be able to make tags be plain functions

@notjack Yeah, I’d like a syntax like that, but then I need to define every possible HTML entity as a macro :confused:

Or you can create an interpreter and have it use a list of valid elems @jerome.martin.dev

But there’s already a builtin sexpr->html thingy in the webserver library isnt there?

@jerome.martin.dev how many are there? if it’s under a thousand, defining a macro that takes a list of html entity names and defines each one wouldn’t be infeasible

yeah, right now the only thing in the webserver library is xexpr, which is plain s-expr HTML without keywords, but I’d like to add some kind of entity system like what we’re discussing here

entity system? Can’t you add that in using ` and ,? or is that undesirable?

I’d like to prevent people from having to quote and unquote every time they want a custom element in their HTML markup, so that is feels easy to extend HTML with your own “entities”

hmmm

Aha. Right

if they were functions, that would be doable by them defining a function

Tbh I prefer the xexpr route, makes it obvious due to coloring what’s plain HTML and where I generate/insert other stuff

(define (my-super-calender #:min-date min-date-str #:max-date max-date-str)
  (section #:class "my-super-calendar"
    (p #:class "min-date-text" min-date-str)
    (p #:class "max-date-text" max-date-str)
    …))

Thanks, all, for the info and suggestions! I and a fellow grad student are now about to start implementing the next version of this thing. :slightly_smiling_face:

@notjack yeah, something like that, except I didn’t want to generate all the base elements myself and kept the xexpr stuff. I guess you’re right, I need a macro that generates all the p, div, body functions

What if you invert the idea

Turn unknown funcs into plain html tags. Turn known funcs (whose IDs are bound) into fncalls :stuck_out_tongue:

oh, yeah, why not!

D:

(this is quite dangerous if u import a div func for some reason though and it all goes to hell lol)

ahah

the road to hell is paved with dynamic scope

why not use parameterize too while we’re at it

emacslisp here we go

:skull:

I’m starting to think quotes are fine, in the end xD

(parameterize ([current-tag-resolver (lambda (ignored-tag) 'marquee)])
  (div
    (p "no matter how hard you try")
    (p "you will never be rid of the <marquee> tag")))

@jerome.martin.dev reminds me, every racket programmer implements #lang no-parens and halfway through realizes how good parentheses are and abandons the project :stuck_out_tongue:

ahah

but I’d really like something like (div #:class "rainbow)" instead of the tiresome (div ([class "rainbow"]))

Why not write a macro that rewrites it to the xexpr form?

Considering you can’t have kw elements in xexpr/html anyway (afaik)

maybe a reader ? x)

I’d like it most because if they’re functions I can use fancy-app to do things like this:

> (map (div #:class "rainbow" _) (list (p "foo") (p "bar") (p "baz")))
(list
 (div #:class "rainbow" (p "foo"))
 (div #:class "rainbow" (p "bar"))
 (div #:class "rainbow" (p "baz")))

right !

If they’re funcs you’ll have to define each kw. Is there a set amount of kws and can some elems not have certain tags?

you can make functions that accept arbitrary keywords

:open_mouth:

I didn’t know

It’s kind of painful to do so with make-keyword-procedure though

I wrote an arguments package that has a define/arguments form for making that easier

my mind blew up a bit

Well then, https://www.w3schools.com/TAGs/ into a macro to generate all these funcs

http://docs.racket-lang.org/arguments/index.html?q=define%2Farguments#%28form._%28%28lib._arguments%2Fmain..rkt%29._define%2Farguments%29%29

> (define/arguments (keywords-product args)
    (for/product ([(k v) (in-hash (arguments-keyword args))])
      v))
> (keywords-product #:foo 2 #:bar 3)
6
> (keywords-product 'ignored #:baz 6 #:blah 4)
24

the package defines an arguments? type that’s just a bag of positional and keyword arguments:

> (arguments 1 2 3 #:foo "bar")
(arguments 1 2 3 #:foo "bar")

(define/arguments (id args-id) body ...) is like define but for a procedure that accepts any arguments, collects them into an arguments? structure, and binds args-id to the collected arguments

that looks perfect

I’m happy you think so :D

gotta use that :smile:

I’ll make some kind of third-party library for HTML, then maybe we could add it to web-server

Thanks for all the insights! See you around!

Gettomg errortrace...: how do i make it print everythnig?

getting*

@macocio are you using drracket or is that output in a terminal?

terminal

Don’t know in general then. Is the error happening inside a rackunit test?

Yeah

I remember some truncation value for errors or something

but it’s so hard to find in the docs

I think rackunit has a bug related to this, let me check the github issues

found it: https://github.com/racket/rackunit/issues/97

#lang errortrace racket works better

but then you need to add it to the file when doing raco test

yeah, to be 100% clear - this is a bug and you shouldn’t have to add anything to get proper stacktraces from a rackunit test failure

but until that gets fixed, #lang errortrace racket is a workaround

When I try defining the ~alt as a syntax class, the compiler doesn’t like it. Any idea why this works inside define-syntax-parser but not in define-syntax-class?

You need a ... after the (~alt option ...) clause

(I think)

(define-syntax-class nodeclass-elem
    #:datum-literals [is-a archetype]
    (pattern (~alt (is-a ~! parent-expr:expr ...+)
                   (archetype ~! archetype-expr:expr ...+)
             ...)))

Indeed the original had ... there. So, it’s not possible to make a syntax class to match any nodeclass-elem?

You can still make a syntax class to match a single nodeclass-elem, but you can’t reuse that class with ~alt

as far as I know

You need to make it a splicing syntax class, then do {~seq {~alt } ...}.

oh I misread that you’re doing this in a syntax class and not a use of syntax parse

oops

So that’s what ~seq does!! I’d been wondering what was the difference from just matching the body of the ~seq.

The idea is that a syntax class always matches a single term, but a sequence of alternatives must match multiple terms.

A splicing syntax class allows you to match a sequence of terms.

Thanks! That clarifies a lot. This is where we got stuck yesterday with attributes that didn’t “bubble up” to the top-level matching expression.

~seq is essentially a pattern splicing form, and it can be used in a few different ways. One of the main uses is this one, when writing a splicing syntax class. Another is when you want to match pairs of arguments, a la arguments to the hash function, which you can do by writing {~seq a b} ....

Hmm, this ~seq around ~alt with ... still gets an error:

@lexi.lambda These explanations of the main uses of the various library functions/transformers are very helpful.

You need define-splicing-syntax-class to make the syntax class splicing.

Ah.

@bkovitz other miscellaneous tips: I’ve gotten into the habit of always using the #:attributes option with define-syntax-class / define-splicing-syntax-class to make it explicit what attributes I need each variant to bind (and what attributes users of the class can access), and I’ve found that very helpful for reading syntax class definitions. You might also want to use the #:description option to 1) have a snippet of in-source docs on the purpose of the class and 2) improve error messages

(define-syntax-class binding-pair
  #:description "a pair of an identifier to bind and a right-hand-side expression"
  #:attributes [id expr]
  (pattern [id:id expr:expr]))

@lexi.lambda That worked. The top-level code can even access the accumulated is-as and archetypes (with three ellipses!).

You probably want to set it up so that you only need two ellipses, not three. Make sure that you don’t put an extra ellipsis after the use of the nodeclass-elem syntax class, since the ellipsis is already inside the class.

@notjack I definitely like explicitness. We ran into troubles yesterday with #:attributes. I’ll try it right now with splicing.

@bkovitz note that when using #:attributes with ellipses you have to specify in the #:attributes clause the number of ellipses each attribute is used with (unless it’s zero, which is the default)

e.g. #:attributes [(parent-expr 1) (archetype-expr 2) some-other-attribute-which-needs-no-ellipses]

@lexi.lambda OK, now I think I see: the splicing syntax class doesn’t match one nodeclass-elem, it matches a sequence of them (hence ~seq). It sounds like matching a sequence of alternatives in a single pattern goes with the grain of the language, and the approach we tried yesterday, defining a syntax class for a single nodeclass-elem is just the wrong approach.

Ah, so the syntax-class should be named nodeclass-elems, not nodeclass-elem.

Yes, you can think of {~alt } ... as really being one unit, along with any uses of ~optional, ~once, or ~between nested immediately inside the ~alt form. That is sort of a mini-language inside syntax/parse, which is itself a mini-language!

OK, that clarifies yet more: {~alt } ... is really one unit, not to be broken apart across syntax patterns.

Now down to two ellipses—which seems like conceptually the right number.

@notjack Just added #:attributes. Yes, this helps readability: now it’s clear that we’re accumulating (in effect) lists of lists, since the number of ellipses is 2.

@jerome.martin.dev An alternative is to use at-expressions (@-expressions) to write html. Example: https://github.com/soegaard/urlang/blob/master/urlang-examples/parabola/parabola.rkt#L118

Does anyone know how to make an sxpath search function that searches for an arbitrary string? It appears to treat some characters specially, and I haven’t yet found an escape character.