Good day! How does Racket differ from traditional parsers with respect to parsing conventional syntax instead of s-expressions? Anyone here with experience with both kind of tools who could provide a more informed and practical comparison?

@mario.luis.guimaraes You can look at brag: https://docs.racket-lang.org/brag/

It allows you to write grammars for any language

The way it works is: 1. You write the lexer, which reads a character stream and generates tokens 2. You write the parser using a grammar DSL, which describes the rules for the position of tokens 3. You get an s-expression and can process it like any Racket syntax

@mario.luis.guimaraes for some examples of brag take a look at https://beautifulracket.com

That sounds promising. But, how does brag compares to other traditional parser generators, like Lex and Yacc, Flex and Bison, OCaml’s Menhir? Is it on par to these tools?

For example, could I write a Java parser using brag?

In my experience, brag is useful for relatively “simple” languages. For more complex ones, I’d go with nanopass https://docs.racket-lang.org/nanopass

I haven’t used them, but there’s also https://docs.racket-lang.org/parsack/ and https://docs.racket-lang.org/megaparsack/

Thanks for the links

I don’t know what it written with but ProfessorJ might be worth a look https://docs.racket-lang.org/profj/index.html

This means that for handling traditional languages (like Java or C) one has to resort to the same kind of tools in Racket as in other languages, is that so?

Well I don’t know what you mean by “the same kind of tools” but lexers, parsers, tokenizers… whatever you call them are fairly similar in any languages. They’re just a bit easier to write in Lisp-like languages because you have easy access to AST

Right, that is what I mean

@mario.luis.guimaraes Traditional parser tools with a Racket twist: https://docs.racket-lang.org/parser-tools/index.html

Here is a lexer for Pascal: https://github.com/soegaard/minipascal/blob/master/minipascal/mini-pascal-lexer.rkt

Here is a parser using ragg (predecessor of brag): https://github.com/soegaard/minipascal/blob/master/minipascal/mini-pascal-grammar.rkt

So the advanced stuff that makes Racket more suited to language-oriented programming is geared only to support s-expressions based languages, is that so? (For the other conventional languages one was to use the typical parser and lexer tools)

I don’t see like that. Lexing / parsing is a solved problem. S-expressions is just one kind of input.

Or one uses the traditional parser and lexer tools to convert conventional syntax to s-expressions, and then we can use Racket’s full language-oriented potential thereon?

The manipulation of syntax objects and easy rewriting between language constructs is the exciting part.

Yes.

Ok, understood, makes sense

That’s usually how it’s done, yup

Ok, so I see there are several variants of parser and lexers in Racket too. Which one would you consider more “industrial-strength” where I could start?

The interesting part about this is that using s-expressions for anything internally, you get to stitch together different languages which may have different syntaxes externally, but work on the same structures internally

Right! :thumbsup:

parser-tools is definitely “industrial-strength”

@jerome.martin.dev You use the traditional parser to convert to s-expressions and “get rid of conventional syntax” thereon.

Thanks @soegaard2

I don’t necessarily get rid of them, but it helps working on the same structures and have a single logic core, then multiple “language interfaces”

I definitely have to look into this Racket language-oriented stuff. There seems something here in the combination of s-expressions and language-generation/integration that is somewhat vague and different from the rest, I suppose.

usually I design my languages as s-expression first, close to the data I’m workin on, then I design a specific syntax if I feel the need to

As anyone here experience with OCaml’s and Racket’s toolchains for language processing and analysis? Perhaps the two best available today …

(I really find these two languages the best around for language processing, hence my question)

I know some people are working with Coq and Haskell too

Btw - I believe that ragg and brag both work by transforming the grammar into something that parser-tools can understand.

yep, but anyone with common experience dealing with this stuff in OCaml and Racket? (I suppose Haskell is on par with OCaml)

@soegaard2 yep, I love brag for that because it’s reeeeally simple to use

@mario.luis.guimaraes Maybe of interest to you if/when you want to shift from researching to trying some things hands-on: https://school.racket-lang.org/index.html

Looks like some course materials from last year, are here: https://summer-school.racket-lang.org/2018/plan/

Thanks @greg.

I don’t know if anyone on #beginners will have done exactly the sort of project you want to do and done it thoroughly in all of Racket, OCaml, Haskell, etc. :slightly_smiling_face: Even if someone did, and thought X was their favorite, you might not agree. Different things “click” for different people. It’s good to do lots of research. Eventually, at some point, you may just need to dig in and try. ¯_(ツ)_/¯

@greg yes, of course, but my question was to have some opinions from anyone that might have such practice, in order to know tradeoffs.

I asked here in beginners just because I am a beginner with Racket; perhaps I should have tried in #general.

Should I go with Racket or Ocaml for serious static analysis? I have to pick one because time is limited, hence I came here …

… hence my question(s)

I know OCaml is a strong choice for this in the “industry”. I wonder if Racket is at least on par to OCaml … but seems no one has such experience to inform me …

@mario.luis.guimaraes Is this a commercial project? What is the schedule like?

yep, the part with static analysis probably end this year

open source & commercial, the typical these days

the static analysis part must be performant and efficient

Will you be the sole developer, or will there be a team?

very small team to start

thing is to put in cloud, somewhere

If you/the team has much experience with any of those langs (Racket, OCaml, Haskell, whatever) I’d say that’s important to consider, along with the supposedly inherent advantage of any of those langs at some specific problem domain. 10 months isn’t a long time.

If you spend the first N months just getting comfortable and productive with lang X, and all on the same page with conventions, etc.

We have experience with Elixir and Erlang, but we know what’s in those langs (I have thaught Haskell once at university, but not used it in industrial practice)

I have lots of experience with Elixir macros (second nature, programming macros is to me as easy as normal programming these days)

In fact Elixir is a kind of Lisp for Beam, but with m-expressions

I don’t earn a commission if I convince you to use Racket :smile:. I’d still like you to use it. But if it were my company/$ and I had 10 months to ship something, I’d probably look at leveraging the thing the team knows how to use. So Elixir/Erlang. Maybe 1 or 2 people uses Racket (or Haskell or OCaml or X) for a piece of the problem.

But I’m biased, I think most software development problems have more to do with primates cooperating than with the tech per se. :slightly_smiling_face:

Yes, Racket (or Haskell or OCaml or X) would be for a piece of the problem (heavy duty static analysis)

For example, today I have been sanity testing a macro-based library I did in Elixir, and almost all of the bugs, if not all, have been due to type errors …

So Elixir is dynamically typed, “typespecs” sound sort of like contracts? Are you saying you wish you had a strong static type system, instead?

Yes, but on the Beam, which there isn’t

In this case the Beam wins for me, without any doubts

Sorry, I was on the phone…

Yep, Beam is fantastic, no doubts, but for heavy duty analysis might be slow.

My original idea of Racket would be to generate DSLs for the Beam (or anything else that comes along), since I prefer Lisp syntax

I tend to see Beam as an orchestrator at the core of a web server, with high-performance nodes in other languages.

it is these nodes that are best suited for heavy duty stuff

@greg I think I will explore Racket meta-programming tools for targeting other platforms with a Lisp-like language, but for parsing traditional languages, and since I like static types, I believe OCaml / Haskell is a better choice right now. Thanks.

@mario.luis.guimaraes Have you considered LFE (Lisp Flavoured Erlang)? http://lfe.io/