Basically, wrapper files and include

Seriously, if anyone wants to write that SRFI with me let’s do it. The reason I ended up here in the first place is we’re trying to improve the parsing/tooling around SRFI documents on the srfi-discuss mailing list. I wrote an experimental CI server for the SRFI process in Racket with a portable core in R7RS. It’s my first Racket (and R7RS) program ever and Racket made the job easy so I’m a happy camper :slightly_smiling_face:

On an unrelated note, I couldn’t find a database-url library for Racket (i.e. parse the DATABASE_URL environment variable, figure out what database driver and settings to use, and return a ready-to-use database connection). If there isn’t one yet, let’s make one. It

shouldn’t be much effort at all.

Another thing that would be nice to have a uniform interface to, is archive files. Currently the unzip/untar modules differ in functionality and interface. I can also help with this if someone can mentor (I have no idea how to publish packages and what the process is for getting things into the base distribution).

On yet another note, if anyone is interested in automatic code formatters, talk to me :slightly_smiling_face: Most Lisps have fairly similar syntax and indentation/line breaking concerns, so we could try to write a code formatter using only R7RS and immutable data structures. If the program used such a restricted language it ought to be easy to auto-translate it to Racket, Common Lisp, Clojure, etc. to fit in the respective workflows in each community.

We have a new time!

(require (for-indentation …))

I can’t tell whether you’re joking or being serious. Just consider me a complete noob and you’re not far off the mark :stuck_out_tongue:

Bit of both.

It would be great, when defining a macro with a new syntax, to be able to tell the indenter how the various subforms of an macro application should be indented.

But at the same time it also feels a bit over-engineered.

Not over-engineered if you ask me. I’ve been wanting that in Common Lisp for years. As they say, “the complexity has to go somewhere”. If the source code doesn’t specify macro indentation then everyone has to specify it in their own text editor config. The editor could hunt down the original macro definitions and parse them but that’s a steep requirement for an editor.

The editor could ask a “language server” about the indentation but relying on that is brittle and complex on many levels.

Plus it would be nice to be able to run an indenter from the command line without involving text editors or language servers.

The problem is that a Racket program doesn’t necessarily need to use the S-expression syntax.

So each #lang language ought to provide (at indentation time) a function/structure that describes how to indent … something… .

Hmm.

I don’t know which Racket lang has the most complex syntax at the moment, but for languages not as simple as Lisp, code formatter settings can get very complex over the years. Obligatory link to clang-format options: https://clang.llvm.org/docs/ClangFormatStyleOptions.html

Many existing formatters for non-Racket languages just use a configuration file in the root directory of your project, including clang-format.

Plus if Racket implements a language that also has non-Racket implementations, it’d be really nice if all those implementations could read formatting options from the same place. That could rule out storing the options in info.rkt.

Also some established languages have several competing formatters and people don’t agree on which one to use.

The Unibeautify project is trying to develop a common vocabulary for different formatters’ settings: https://github.com/unibeautify I’ll gladly help to get any Lisp-flavored languages represented in it.

Is there a known issue/interaction between dynamic-rerequire and the IO system? I’ve been trying to figure out why my DB queries sporadically take significantly longer (>1s rather than less than 1ms on average) than they should after my code reloads (and only after the first time it reloads). I’ve ruled out the database and I’ve also ruled out the db package (I linked it locally and started profiling every function on the call stack for a query). Profiling the db package, I found that read-char (i.e. reading the next character from the db connection) is taking up all of the time when those slowdowns occur. Upon every reload, my code restarts the web-server and the db connection pool so I don’t think this is to do with any lingering ports or custodians (I’ve tried creating a new custodian to hold everything and shutting it down for every reload, no luck w/ that either). Any ideas what might be causing my issue?

Best idea: Make a minimal example and post it to the mailing list.

Which isn’t that simple to make in your situation.

Yeah, there are quite a few moving parts, but I’ll try to do that if I can’t figure it out soon.

What would really help debug this is if I could somehow instrument all running threads and tell them to print something to standard out when they wake up. Is there such a hook available?

My suspicion is that, upon reload, some background threads end up leaking and when read-char is called, the scheduler wakes up one of those threads and it does… something that takes up all that time.

I’m almost done writing that database-url library. But… is there a “URL path for humans” function somewhere? (url-path u) gives me what I think is a list of structs of some kind, but I’m not sure. I can’t find anything in the standard libraries to effectively traverse or convert that thing to the usual “foo/bar/baz” form as it’s written in the URL.

It looks like it’s complex in an attempt to aid URL<->file-system-path conversions across operating systems, but I’m just totally confused.

This looks like a bug to me… https://stackoverflow.com/questions/55321755/applying-cast-to-dynamically-required-function-in-typed-racket

How, in the world of syntax-parse, can I call a syntax class as a subroutine, passing it syntax variables with all their attributes (and sub-attributes, etc.)?

What I’d like to be able to do, in this example, is move the code for that big lambda to its own syntax class, where it would be called with information not available in this syntax class (parsed elsewhere; needed for the TODO line). IOW, I’d like to be able to do all the parsing first, and then, when all the parsed information is available, do all the code generation. This is for maintainability and readability more than just to fix the TODO line.

@sorawee I think the problem is that TR treats cast as a typed/untyped boundary, so it (confusingly) treats even this program as an error: #lang typed/racket (define x : Any (lambda ([n : Integer]) (add1 n))) ((cast x (-> Integer Integer)) 42) This happens because it thinks that the use of x inside the cast is flowing into an untyped region and attaches any-wrap/c to it. I don’t think that’s actually necessary here, though, since none of the things any-wrap/c guards against can actually happen. So maybe contracts generated by cast should never include any-wrap/c contracts?

I have an idea for how this could be solved.

I’ve thought about this and the issue is TR currently doesn’t distinguish between: - typed values that it knows nothing about the type of, which it needs to guard with any-wrap/c when passing into untyped code or casting - untyped values that can be passed back into untyped code or casted safely without any-wrap/c

Currently TR treats everything with type Any as the first one, which is conservative and safe. However I think the second one is valuable too, especially if things like dynamic-require are involved. It could be a separate type, a subtype of Any (it would not be a supertype of any non-flat types), that could be called DynamicAny.

The DynamicAny type would be different from Any in two ways: (1) It is not a super-type of any other type that might involve typed higher-order values. (2) It is safe to pass values of this type into untyped code without any additional checks or contracts

When values go through an untyped->typed boundary they need a first-order check. When values go through a typed->untyped boundary they don’t need any check or contract.

dynamic-require would return DynamicAny DynamicAny <: Any FlatFirstOrderImmutableData <: DynamicAny (Boxof _) </: DynamicAny (not a supertype of any type might contain mutable values) (Vectorof _) </: DynamicAny (-> _ ... _) </: DynamicAny (not a supertype of any type that might contain function values or take inputs)

Have you considered just writing a syntax-time function that takes the syntax and all of the properties and returns the new syntax?

@popa.bogdanp What kind of server are you asking db to talk to — an in-process FFI one like sqlite, —or—, one it must connect to via TCP or Unix domain socket like postgres?

If latter, maybe you’re seeing something connection-related? (Maybe the connection always takes 1 sec but you don’t notice on the initial load because it’s just another second among many? And/or, maybe there is some delay due to recycling a TCP port? Just a couple wild guesses, that might not involve Racket per se.)

If latter, and you’re using TCP now, maybe try Unix domain socket instead. I believe that’s a relatively recent addition to the db package. It’s probably better generally, even if n/a for your immediate question. Probably unix domain sockets are more secure and faster, both.

If you’re in the REPL, how do you run a module that’s already been loaded or entered?

IOW, what’s the REPL equivalent of Cmd-R in DrRacket?

I’m using Postgres so the driver is written in pure Racket (not FFI). I thought it might be connection overhead, but it doesn’t look like it based on PG’s logs. Turning on debug logging for the db topic shows that the issue occurs even an idle connection is being grabbed from the pool. When I profiled the db library I narrowed it down to the moment a query response is retrieved from PG (i.e. that read-char call I mentioned) so I’m still thinking it’s some scheduling thing.

At any rate, I sprinkled1 a couple custodians in places I thought might be problematic and that seems to have helped!

The issue still happens and it still only happens after a reload, but it happens less often now.

Rerequire ?

I’ll try that as soon as my code compiles again…

@soegaard2 Here’s what I got in XREPL. This is Typed Racket. I haven’t tried it in Untyped Racket, but the module that I want to re-run at will is in Typed Racket.

So, do people not normally put code in at the top level of a module that they want to run at will?

I don’t know enough about TR.

@bkovitz One approach is to make your syntax-class take parameters (but the parameters are treated as ordinary variables, not syntax pattern variables). Another is to change apply-tag from a syntax object bound using #:with to a procedure-valued attribute bound with #:attr that takes the contextual information and produces the syntax. The caller of the syntax-class would have to get the procedure attribute and call it with the necessary information.

@samdphillips I’m actually calling a syntax-time function elsewhere, for a different purpose. The rough spot here is that I need to pass all the attributes from the taggee-info syntax class. I suppose I could package them all up in some big list, or maybe attribute will do that, but then I have to parse them all out again—and I’ll have to change this code every time I modify taggee-info. Not an insurmountable challenge, but I wonder if there’s some nice way to pass all the contextual information that #:with already has access to right inside the applies-to syntax class.

@ryanc Ah, I had not thought of making an #:attr whose value is a procedure—a closure! Thanks. If I understand this right, the closure would still have access to everything that #:with has access to inside the applies-to syntax class (the thing I was hoping for in the previous message).

@ryanc I tried passing an argument to the applies-to syntax class (via the nodeclass-body syntax class), like this: (define-syntax-class nodeclass #:attributes [cnodeclass] (pattern (head:nodeclass-head ~! decl:name+args body:(nodeclass-body #'decl.name)) but that didn’t work—and probably with good reason.

I think if you really want the same as Ctrl-R in DrRacket you should exit and restart.

@samdphillips I’m hoping not to do that, since that’s pretty slow, and speed is the main reason I’m now experimenting with XREPL. Is there no way in Racket to say “Run that module”?

@bkovitz how about ,enter https://docs.racket-lang.org/xrepl/index.html#(xrepl._enter)

,en doesn’t seem to re-run the module if it’s already been loaded.

Or maybe the two commands above it?

maybe ,require-reloadable?

Hmm, I haven’t tried either of those. I’ll try them next time my code compiles. :wink:

So, there is no way in Racket to say “Run the top-level code of such-and-such module”?

(Many years ago I tried to use XREPL in a shell buffer in Emacs, with some hotkeys to do enter. I was confused by reloading, or not. That was kind of what nudged me to making successively less-crude versions of racket-mode.)

Normally require or enter! the functions, or the similar XREPL commands, should eval module-level expressions in the file.

And the default #%module-begin wraps each top expression in a print, so you see each one (unless it’s void).

Regardless of the question about re-running a module, I’m pretty sure there is something weird about the way enter! decides which modules to reload. It’s often missed reloading a file that I just modified. I haven’t checked it out carefully yet, but it looks like enter! will not reload the module that you’re currently in. So, if you’re in "my-module.rkt" and you ,en "module-that-requires-my-module.rkt", "my-module.rkt" won’t get reloaded even if you just modified it. Not 100% sure of that, though.

There could be a bug like I thought there was years ago, or it could be I’m misunderstanding.

One thing to keep in mind, is that some modules with lots of macros can be quite time-consuming to expand. And most of the time to require them is spent on expansion. This tends to be true of Typed Racket, for example. So in bigger projects, it can help to use raco make directly, or indirectly through raco setup of your collection or package, to compile to bytecode zo files.

Those have already been expanded and tend to load quickly.

And then, in an env like DrRacket or racket-mode, that always reloads everything on each run, it’s not very painful.

I don’t use DrRacket often, but it has an option to make these bytecode files automatically, I think.

Ah, thanks for that. I’ve gotten the impression just from the compile times that modules are often getting recompiled unnecessarily. I figured there’d be some way to prevent that. Happily, there seems to be a lot less recompilation under XREPL. (Hopefully not because it’s failing to recompile changed modules! Most of the time it does, though. Happily, it tells you what it’s recompiling.)

BTW, I’m now in the midst of making all sorts of modifications to a big macro that implements a DSL, along with lots of modifications to the code that interfaces with the generated code. This could have been a undebuggable catastrophe, except that the last thing I did was convert everything to Typed Racket (including the generated code). It’s catching subtle errors very nicely, with very little extra effort to set up the types. I’m actually staying confident that the code is correct. Whew! Now I wish I’d converted everything to Typed Racket months ago.

This is something that I’ve found a bit mysterious about the Lisp languages ever since I started programming in them: how to “grow” a program incrementally, where you get a little bit to work, then you add something and maybe modify existing code to work with it, add a bit more and get all tests to pass again, etc., occasionally “refactoring” when the code starts to get messy. I know that the ability to do this is often said to be a great strength of Lisp, but I kept finding myself in long debugging sessions and generally losing confidence in my code.

In OO languages, I was pretty good at “growing” programs incrementally, because of course that’s pretty much what OO is designed for, and a whole lot of common wisdom has emerged about how to do that in ways that don’t invite bugs—hiding everything behind very narrow interfaces, Once-and-Only-Once, Law of Demeter, Dependency Inversion, Open-Closed, etc.

Has anything like that common wisdom emerged in the Racket world? Switching to Typed Racket has helped enormously, but I’ll bet there’s more.

@bkovitz You can’t use the colon notation with arguments. Use #:declare body (nodeclass-body #'decl.name) or (~var body (nodeclass-body #'decl.name)).

Oh, I see. Thanks. I thought that whole approach was hopeless. Now I’ll have a closer look.

@mbutterick I was sorta thinking roughly the same thing: that it’s not so easy in Racket to start with a clumsy/ignorant design and gradually streamline it into simplicity and elegance through many small changes, all of which compile and run without bugs. Hence get it pretty much right the first time and keep it small, because bigger things are harder to change, and if you get a better idea, you’ll probably need to restart from scratch. I hope someone will tell me this isn’t true!

How can I generate code conditionally (with an if) for the current taggee? Since the arity of taggee is 1, the #'s inside the #, are failing.

The only way I can think of is to generate something like taggee.let-binding in the syntax class that returns the taggee. I’d prefer not to, since that harms readability: that syntax class would have to know about the local variable ht/nodes, which is generated here in a syntax class a couple levels up. And the code here wouldn’t be as clear.

I’m usually not too concerned about starting from scratch—until the program gets large and rewriting it becomes a long and risky endeavor. The incremental approach is pretty risk-averse: you might delete a lot of code over the life of the program, but at any given moment, you’re never more than 10 minutes or so from a working version. But perhaps the flexibility of Lisp languages just doesn’t work that way. OO makes it possible by working within a lot of restrictions: basically by severely limiting each part of the program’s access to data.

Thanks. That is a good one. I’ve been looking for ways to separate this monster thing I’m working on into little libraries, maybe little languages. The biggest headache has probably been the graph representation. That’s hard to redo independently because pretty much everything calls upon it. Of course, even though some things are hard to split off, other things might be very splittable.

@mbutterick BTW, I noticed in sugar that you provided a very nice small technique for keeping programs easy to modify, which took me quite a while to hit on: Instead of making sure that I pass a function a list, or set, or whatever it wants, call something like ->list or ->set within the function, so the function accepts whatever you pass it. This has become particularly handy now that I’ve gone to Typed Racket. (Unfortunately, I didn’t browse sugar until hitting on this the hard way!)