phew, that’s really cold. And yet, I keep complaining about +1 C outside.

How can I avoid section numbering in scribble files? This is related to using scribble files in frog. Section numbers in a post just look wrong. @greg?

@pocmatos use @section[#:style 'unnumbered]{Section title}

@ryanc thanks

Hey! I’m working on a syntax-highlighting plugin for VSCode, but it’s still early work in progress. I managed to fix number scoping with the help of Jens from our google group, so I figured I might ask some questions here as well.

(repo: https://github.com/Eugleo/magic-racket)

@wybitul.evzen Awesome! I put together a similar syntax file for SublimeText & Amp, but it’s really not accurate so I didn’t post it publicly yet. I’ll check if I can get some inspiration from your work :slightly_smiling_face:

Currently, the first symbol in a list is highlighted as a function call; it’s a little brittle, though (things like ( #;(sexp comments ftw) map add1 '(1 2 3 4) ) don’t work), and sometimes outright incorrect: (let ([x 1] [y 2])) highlights the x and y as functions. So, the question is: should I drop this “feature” altogether? DrRacket simply highlights known functions (exported from racket) and the rest is plain “symbols”.

I kinda like the "first symbol is different’ rule, but I guess it’s a personal preference

@jerome.martin.dev I’m not sure how syntax highlighting works for Sublime, but I guess it’s got something to do with textMate as well. My grammar file is a .yaml, so maybe I’d be able to export it to the SublimeText format. That means we could join forces on the project and make it the standart across more editors than just VSCode :slightly_smiling_face:

I’d be glad to join force on this, definitely! Writing good syntax files is hard and time consuming

The sublime syntax was textMate before, but changed to a new custom “sublimeSyntax” system. Amp decided to use the same system as sublime

most of the time, they’re just regexps with greedy/not greedy rules

Just checked out their webpage. It says > Sublime Syntax files are YAML files with a small header, followed by a list of contexts. […]

Seems good. Maybe no conversion will be needed, or maybe just some manual key-changing.

a bit

it looks like this :

https://github.com/euhmeuh/sublime-syntaxes/blob/master/Forth.sublime-syntax

(I did this one for Forth, but also did one for Racket I didn’t published)

Well, now that I’m looking into it, the semantics are little different. It seems to be more powerful than the textmate style, and has some additional constructs, too.

yep

most keys are custom, you can choose the names

then put them together

the only real keywords are “match”, “push”, “pull”, “scope”, “pop”

it works like a stack machine with regexps

Yep, that’s the notion I got from their webpage as well. I think the “ideas” will translate very well, though.

And it seems you’ll be able to handle quotes and unquotes correctly. Pushing and popping a stack, so you’ll be able to correctly asses whether the things are quoted, or unquoted, even in some deeply nested scenarios.

Pretty cool.

yep :slightly_smiling_face:

Some kind of stack (explicit or implicitly handled by the language) will be necessary to count the parantheses for anything like a #;(s-exp) comment

Oh hey. You’re the guy from SO!

Yep. One can emulate this in textmate grammars by matching begin: #;( and end: ) and between it some things like parentheses, strings etc.

The problems is, I’d have to basically have two grammars: one for those normal constructs which would scope them under numbers, strings etc, and other, which could recognize them just as well, but scope them under comments. 1kloc → 2kloc in a blink. Not worth it just for the comments, I think.

Grammars (things like context-free-grammars or CFGs) implicitly handle a stack for you, which is really convenient

I’ll have to read more on those grammars you keep mentioning. They work somehow like a Haskell Parser would, right?

(sorry, don’t know better example)

Are you referring to Parsec?

Yep. Or Trifecta. They all work similarly, I think.

At least… The way one works with them is really similar to the grammar spec in Racket’s docs (e.g. in the Reader–Numbers section). Not sure if this kind of “grammar” is the same as the CFG you talk about.

The grammars in the racket docs are a variant of CFG, and I think technically they have the same “power” as in they can compute the same class of things

And that “power” is greater than a regular expression, but less than a full (turing-complete) programming language

Seems really interesting. Shame it can’t be used to make the syntax highlighting better…

There are different levels of syntax highlighting.

The first level just tell whether some group of characters is a string or a symbol or a paren or something

That level can’t / shouldn’t count parens, so it wouldn’t be able to deal with s-expression comments

That first level could be expressed with regular expressions

The second level could deal with well-formed expressions according to a grammar, a CFG (or anything that can handle a stack)

that level would be able to handle s-expression comments, and if you want it could also handle coloring matching parens-pairs the same color while making them all different, like a rainbow or something

And that level would be able to distinguish the first element of a function call, even if there was an s-expression comment in the way, or if that first element was a complex expression like (compose (λ (x) (map add1 x)) range), within ((compose (λ (x) (map add1 x)) range) 5)

Then a third level could deal with scoping

The third level could tell that the x from (map add1 x) is bound by the x in (λ (x) above it

Or that map comes from Racket and not a local-variable

To distinguish that from (let ([map list]) (map add1 x))

That would be cool to have in so many languages. You can’t guess that from syntax alone, though, can you? In some lightweight way, I mean, without trying to “run” portions of the code.

Especially when somebody can ruin your day with macros or similar things.

If the code is powerful enough (if it has macros which can manipulate scope), then yes you would have run some code. But not all of it! And in racket at least, the parts you would have to run-in-the-editor for scoping are specially marked for compile-time

DrRacket does this with background-expansion, and its “check-syntax” arrows only work after that background-expansion has completed

And you’re right that a macro might ruin that, either by taking too long, or trying to do something that wouldn’t be safe for the editor to run

It’s kinda sluggish, though. Not sure if it’s caused by the background expansion of the forms, but still.

And that’s why that “third level” has to be separate from the other two

I wonder if the “stack machine” from Sublime could be implemented in VSCode as well. I heard Atom is pushing “tree-sitter” (not sure what it is, exactly, but supposedly better than plain regexps)… Maybe VSCode also has plans for some better way to make semantic highlighting possible? At least the first two levels

Anyway, what do you think about the colorization of the first symbol? I can drop it and only highlight the functions I know are exported from racket. But I’d have to highlight those wherever they appear, that means that (let ([map list]) (map add1 x)) and (compose (λ (x) (map add1 x)) range) won’t be differentiated, so some inconsistency will still be there.

Tree Sitter is very cool https://github.com/tree-sitter/tree-sitter

https://tree-sitter.github.io/tree-sitter/

https://www.youtube.com/watch?v=Jes3bD6P0To

Just watched a video about it. Seems exactly what we were talking about. A real parser to be used for syntax highlighting

@d_run Any idea about the lame VSCode highlighting? :smile:

I mean. About the first symbols etc.

VSCode does some pretty powerful highlighting of langs like TypeScript

I haven’t lisped in VSCode tho, so not sure what that experience is like

silly question - wasn’t lang server protocol supposed to offer syntax highlighting as well?

I thought thats what VSCode was doing with JS/TypeScript

Not sure how it works with TS, but I think they have some backend which they are querrying. Doesn’t seem to be universally usable (unlike tree sitter)

Pretty sure they’re passing code to the TypeScript compiler

@githree It was/is. But afaik it’s not complete, and it’s slow and… Yeah. WIP.

yes they are https://code.visualstudio.com/Docs/languages/typescript#_installing-the-typescript-compiler

Well, yeah. That’s along the lines of what I was thinking about. Not sure how I would do that with Racket; LSP would my safest bet I think.

Racket Mode in Emacs does some fun along these lines

Once it works, that is.

Well, I didn’t do any emacsing on the lisp, to paraphrase you :smile:

Right but it might illustrate an approach you could take

Sure. maybe it’d be best to help those guys with LSP, so it could then be reused in other editors as well.

the world where you specify language specific characteristics like syntax highlighting per language as opposed to current per editor would be a huge improvement, but well, seems like a long way ahead…

Yeah. Tree sitter could help that, as well as LSPs. I think we’re slowly getting there.

Does anybody here work on the LSP? I don’t have much experience with Racket, but I’m sure I could get into it.

@wybitul.evzen I’m interested in improving racket support in Atom, so I’ll definitely take a look at your VSCode plugin as well

I know there are 2 implementations so reaching their authors might be a good idea:

Well, unless you plan to implement it tree-sitter, we could join our forces on this one. The yaml can be easily converted to cson (at least I believe so)

https://pkgs.racket-lang.org/package/racket-langserver

https://pkgs.racket-lang.org/package/racket-language-server

I opened an issue on one of them. Didn’t think of there being two of them, though :smile:

Why does everything have to be so fragmented?

that’s a very good question…

I guess it’s a scheme heritage - everyone has their own

it’s an open source thing

if you look at the packages for VSCode or atom you’ll see lots of packages with similar funcionality

especially if it’s a simple package like a theme (I counted at least 3 zenburn theme packages for atom)

have you looked at NPM lately?

That’s unfortunate.

:joy:

It’s true that I built my syntax highliter from scratch as well… But initially I submitted a PR to one of the most popular packages for Racket, but without luck. So I resorted to making my own.

yes, that happens. you try to contact the maintainer and get no response

Well, it was more complicated than that. I contacted him, he responded, told me to make a PR, then accepted it and merged it; however, he didn’t update the package on the marketplace. I opened an issue about it and well… That has been well over 6 months ago.

it’s worse then

I would like in scribble to be able to show a portion of a file in a column, possibly a function, and interactions using the function on the right side. Is there anything out there resembling helpers for this or something?

… ok, i might be able to achieve the two column with a single-row, 2 column table.

hello, is writing Racket with (neo)vim/DrRacket viable option?

the other option which I consider is Julia lang

neovim and drracket are two options to develop in racket.

drracket has all the bells and whistles for racket dev. if you want to use neovim, take a look at https://docs.racket-lang.org/guide/Vim.html

@pocmatos thanks. i’m asking since i’m not into Emacs..few tiems experienced slight wrist pain which never happened with vim

understood. when i got wrist pain with emacs, I bought a new keyboard. but vim is also an option, although I have never used.

@sjaniska Some people like spacemacs, which configures Emacs to use many popular packages, one of which is evil-mode (vim modal editing bindings for emacs)

ever since I started using “Microsoft Sculpt Ergonomic” keyboard+mouse set, pain disappeared. Emacs lives on in my machine. :slightly_smiling_face:

Other people like to remap the control and alt keys (some vim users like to do that for escape, too :))

I tend to use the standard bindings, but as a (bad) piano player I tend to use my thumb for ctrl and alt sometimes

Or you could do the simple thing and just use a vim :smile:

heh, i’ve tried spacemacs, but it brings another layer on top of emacs and at that time it didn’t play well with e.g. gnus…now, i’m more int ovim-world - that’s why i’m asking…yeah, i keep caps & ctrl switched even with vim setup :slightly_smiling_face:

@greg well, i’m just curious if there are racket/vim users :wink:

I think there are. But the people I know directly tend to use evil-mode with emacs (not necessarily the whole spacemacs thing, just evil-mode) as opposed to vim. That’s just the people I happen to know. ¯_(ツ)_/¯

ok, got it

I’m a new Racketeer but have used VIM for a couple of decades. So far it’s been fine using VIM. I’m not an IDE guy, so as convenient as Dr. Racket is, I still end up in VIM.

@daniel thanks a lot. do you use vim or neovim?

which plugins?

so far, i’ve seen that Julia is nice with VIm, but i like Racket’s GUI support for writing desktop apps

Plain old VIM with no plugins, so far. I usually SSH into a VPS, so an IDE is out of the question anyway. I just have a couple of panes open in tmux, one with VIM and in the other, the Racket REPL.

ok

Any language is nice with VIM. I wouldn’t go and use Julia just because of that. It ain’t no Lisp! :stuck_out_tongue:

:joy:

@pocmatos is right: This is a good resource to get you going with Racket in VIM: https://docs.racket-lang.org/guide/Vim.html

@wybitul.evzen Great find (the pull request to fix brackets on Github (for Scheme)).

I am attempting to find someone that can help in #textmate (on Freenode).

@daniel Wrt IDEs: The most effort has gone into DrRacket and racket-mode in Emacs. It’s worth trying them out to see what they have to offer. If/when you return to Vim - you will know when to try something out in, say, DrRacket.

@wybitul.evzen I have now also sent a mail to the Textmate mailing list.

Great. Maybe they will respond to that. I see to value in not adding the support for brackets.

(sent you a mail btw.)

Still thinking!

I think your option 2 is best. Highlight known identifiers bound to functions. It’s rare to rebind functions exported from the main language - and if it is done, it’s likely to be bound to another function.

Oh, no pressure. Just so you know! I’ve got a bit lost in our conversations myself, you know :smiley:

But if you want to improve - you can copy the approach that racket-mode in Emacs uses.

Hmm - well maybe not. Racket-mode also handles running / evaluating - so it has access the names being defined in the current module.

But may be the lsp-server can be used instead?

Yep, we talked about this here. I contacted the maintainers of both of the LSP implementations. For the time being, textmate grammar is the best solution I’m afraid

Regarding IDEs do any of you have experience with https://www.theia-ide.org/

Impressive.

I like their approach and just found out they are building support for VS Code extensions which will make it a great choice especially if you want to build your own custom IDE

Ohh!!! There is https://github.com/theia-ide/theia-racket-extension

Looks nice. The dev of this particular lsp said he doesn’t work on the project (the LSP, that is) anymore.

I wonder is it possible with LSP to expose the information used by custom Racket DSLs (and not only #lang racket)

what I mean by that is if my DSL has DrRacket integration (e.g. coloring) could this information be automatically exposed via LSP?

@githree yes that should be possible