could the efforts of http://akkuscm.org/\|akkuscm.org/ be ported to racket easily? It sounds like it does a lot of what was proposed.

Checking with the community: Quite often I want to write (>= (length l) 2) but I don’t because length is not cached so this takes time proportional to length(l) which is annoying for large l, so instead I write monstrosities like (and (not (empty? l)) (not (empty (rest l)))).

How would people (core devs and others) feel about a having built-in (length>= lst n) that takes time O(min{length(l), n})?

Been there… But match has reduced the need. Hmm. There’s got to be a length>= somewhere.

@notjack Has a length>? . Having one in racket/list would be better.

Yes, match works for some cases, but it’s also visually heavy for just a simple test

and I don’t want to use it for n=10, say (not even for n=5)

Yep. I have length<?, length>?, etc. in my utility library too.

Exactly, that sounds like quite some code duplication here. I believe this belongs to racket/list as @soegaard2 says.

Another way is to cache the length of the list, but this means that every cons node will incur one more memory cell.

Every cons node has a flag already whether it’s a list or not, so ideally we might be able to reuse it? But that’s just a boolean flag, so one bit suffices. For length, it could be arbitrarily large, though supporting only fixnum doesn’t sound too bad.

(edit: was I just repeating what you just said?… :smile: )

Given that lists are omnipresent in Racket, I think caching the length may well almost double the memory load :slightly_smiling_face:

@lexi.lambda I don’t really agree. Presumably the goal of specifying an = constraint for a version of a dependency is so that when you rebuild your package in some other context, it continues to work (as in moving between dev and prod). But version constraints can’t ensure that if we don’t store the packages. I can imagine two ways you might still want version pinning. One is to help deal with packages that break compatibility. The package system explicitly aims to deal with that in other ways. The other way is to make it convenient to get partial guarantees about what versions you’ll get somewhere else, that seems less than compelling.

@samth The SHA provides integrity checking. Many package ecosystems allow removing packages. The reason I want versioning is actually more from the perspective of a package author than as a package user (though as a user I would benefit indirectly). That is, I want the freedom to refine and improve my package’s API even if the improvements are backwards-incompatible.

The “reproducible build” aspect of package management is an admirable goal, and indeed, if I were in a situation where I was frequently finding my packages no longer built despite not changing anything about the version constraints, I would want a superior solution. But it is not in any way related to why I want package versioning.

Actually… the list? check is much more sophisticate than that. It is an amortized operation. See https://github.com/racket/racket/blob/master/racket/src/cs/rumble/list.ss

I stopped publishing Racket packages some years ago because I didn’t want to commit to indefinite backwards compatibility. Now I push my Racket libraries and utilities to my personal dotfiles repo and break their APIs whenever I like.

That is, it doesn’t actually always allocate memory for every cons cell.

That’s pretty smart!

So we could do something similar for length, no?

I wonder if doing so would lead to a decent speed-up of existing code :thinking_face:

in particular, functions like map test that lengths are equal when given several lists. If this itself happens within a loop, that could save a substantial amount of time!

I’m sad that you’re no longer publishing packages, but I have not found the need for backwards compatibility to be nearly so onerous as you apparently have found it. And I value the Racket community’s approach to keeping things working over the long term, in comparison to other systems where code from long ago is much less likely to work.

I know.

That’s true but I don’t mind that much in this case.

I also found the indefinite backwards compatibility requirement extremely onerous. Rather than not publish packages, I decided that I’m simply not going to commit to backwards compatibility “in the abstract”. I will try to avoid breaking other packages that I know depend on my stuff, but I’m not going to try not to break hypothetical code.

So far there are two conflicting desires here; backward compatibility and breaking compatibility. There are good reasons for both desires and people have varying degree of preference for one or the other. It seems to me that a package author’s preference boils down to using versioning to mean one or the other.

What if instead of trying to use versioning to support two opposing preferences; we introduce another parameter, call it *edition.* Reserve *version* to mean backward compatibility while *edition* means breaking compatability.

So a package would have two parameters describing its state of compatibility. Within each edition, version would mean backward compatibility. So if an author introduces new changes which preserve backward compatibility, they would simply increment the version number. If introducing breaking changes, the author would increment the edition number and reset version to zero - or 1 - or whatever base number is.

Obviously the drawback is we’re using an extra parameter which has to be maintained and introduces additional layer of complexity but it may be manageable and seems like a potential straightforward solution to allowing both backward and breaking compatibility changes to be handled at the package level.

When the package system was introduced, I thought we’d see package names like foo–1, foo–2, foo–3, etc. That didn’t happen.

myself, I personally don’t like the idea of naming packages that way, I’d much rather have one name and various versions or “editions” handled as package metadata

a edition/version attribute pair would map to a SHA hash while the packages keep their unique names, I’m concerned about a package catalog being cluttered by foo-* different names

The idea of having version and edition is fairly similar to what Go does.

In Go, packages have a x.y.z version number. When x is 0, authors have not yet committed to a stable API. When x is 1 or above, authors have committed to trying their best to be backward-compatible. x is incremented only when there is a backward-incompatible change. y is incremented when there is a new backward-compatible “version”. And z is for small patches, mainly bug fixes I think.

Other languages follow the same scheme. It’s generally known as semver: https://semver.org/

exactly

Indeed, that would be ideal, but it seems that packages in the past haven’t really followed that scheme, the version parameter can be assigned any string one pleases so it’s difficult to interpret what a version string actually means

IIRC, Elm’s package manager at one point enforced semantic versioning. If you made a breaking API change at the type level, it would force you to increment your major version. Of course, you couldn’t do that in Racket.

so either we mandate that moving forward, version strings must conform to semantic versioning but I think that would be seen as a breaking compatability in the current package system while introducing a separate parameter edition allows one to avoid breaking existing version information

Racket doesn’t even follow semantic versioning, the major number means significant addition of features or internal implementation redesign while still maintainingg backward compatibility

I think Racket’s scheme makes sense and I don’t think semver applies everywhere. The canonical example being a package that distributes tzinfo databases. CalVer would be more appropriate for such a package.

I think if the tooling were changed to allow a couple special characters in package names, you could create a package catalog that is able to support something like editions as well as namespaces, which I would definitely like to see

Like raco pkg install bogdanp/koyo#3 to get “edition” 3 of bogdanp’s koyo package.

something like package@mark-I:3..4.5?

oh i like the usage of #3

edition would be a string so you could use whatever scheme makes sense for your package

As a package user, honestly I only care about a boolean flag: Is this “out of beta” and stable and can I depend on it going forward, or not? “Not” or “not yet” is OK, I just want to know.

raco pkg install bogdanp/koyo#3@4.5.6 seems very readable

semver and all sorts of lawyerly intricate systems are great for some folks, but it’s just way over-complicating the basic issue, for me :smile:

here lies the danger of elaborating schemes for package management, it becomes too complicated, difficult to evolve or reverse

As Jay pointed out years and years ago, you can already effectively do this by just creating a new package name; that is, replace "#" with "-" and it’s already legal. The problem is that if there are two major versions that want to install the same module paths, then they conflict, and fixing that problem is much more difficult.

I think, as an application operator, I agree with you, Greg. With the caveat that I’d also want some guarantee that the package will work/build with other packages I may be interested in.

Yep. I was thinking of playing around w/ the notion of versioned catalogs, kind of like Stack in the Haskell world, but for Racket.

@ryanc couldn’t that be addressed by requiring only one package to be installed at a time?

Also your single package could provide a new module. package/blah and later package/blah2.

Sure, I think that’s what happens now. But that would mean there would be other packages that just could not be installed simultaneously.

The LinkedList.size() implementation in Java is O(1). https://stackoverflow.com/questions/863469/what-is-the-time-complexity-of-a-size-call-on-a-linkedlist-in-java/863496

@abmclin If you require both package A and B and they in turn use different versions of C… what then?

As an operator, I could say “ok, this application is going to use packages from the 2020–04–09 version of the catalog for the time being” and I’d know that all of the packages in that catalog all build together/work effectively. I wouldn’t be in danger of changes/updates to one package breaking my build. Upgrading would then become a PITA though

package A and B would have their own edition/versioning, if the author changes either package to require later version of package C, then both A and B’s versioning would necessarily change

@greg, yes, I’ve tried something like that with ppict/2, but it just feels awkward. Especially when the rest of the world doesn’t wear version badges.

If one needs to install more than one version of the package for various reason, it seems this could be effectively addressed by using virtual environments like it’s done in python to handle potentially conflicting versions on the same machine

In fact, I think the size () of all Java collections is O (1). Because performance matters.

@ryanc I understand, I mention it just as something that might be (for some people) less inconvenient/awkward than adding/maintaing an entire other package.

But maybe not many people care, given the meta/lib/test/doc 4-way split a lot of folks do. :simple_smile:

Sure, and you’re right, it’s probably the best that one can do given the current package system.

If you’re going to make 4, why not make 8 or 16? :slightly_smiling_face:

@abmclin The plot twist is that package A and B aren’t yours, so you can’t change them.

I think people should make two: mypkg-lib and mypkg. I think there are (often) good reasons for splitting out mypkg-lib, but I don’t find additional splitting useful.

Anyway I am probably old and cranky and insufficiently optimistic. I have just seen folks create many elaborate systems to deal with what are mainly social/etiquette/communication issues. And if the system is too complicated, people ignore it or use it wrong and then you’re back to square one. And on that note I end my periodic package version rant. ¯_(ツ)_/¯

wanders off to yell at some kids on his lawn

@greg that’s a very valid concern, it’s not something to be done lightly, it seems though there could be something we can do to help support the desire for both backward and breaking changes

For sure, don’t let me discourage you, and the Racket community is small enough that things might work better.

I do feel like a virtual env solution would be kind of sad, tho.

@soegaard2 hmm I expect I won’t have direct control over most or any packages, when I update bunch of packages, they’re usually updated all together. If I want to use packages that depend on a version of C and I would like to use an updated version of C which A and B haven’t yet been updated to use, that’s something I’ll have to live with. Prod the A and B authors to upgrade hopefully soon or not;.

I guess I’m not really seeing this as a huge problem

Yes, but Java doesn’t use lists nearly as much as Racket. For Racket, there’s a tradeoff with memory cost too.

Seems like we have the A, B, C problem, anyway

Since C could make a breaking change and then A and B won’t work

and it exists in all package systems so likely it’ll always be with us no matter what scheme we come up with

And then you’’ll have to resort to pinning an old version of C from version control

And if package C doesn’t use version control, you’re SOL

It’s almost like making a breaking change turns out to cause problems when people depend on you. I wonder what we could do about that? :smile:

we’re really talking about two separate concerns, end-users’ need to cope with various package dependencies and an catalog can effectively deal with archiving packages on a local server and pinning them to specific versions versus package authors’ need to express their intention to break or maintain compatibility

I think also people want to share code without necessarily signing up to maintain compatibility forever. And the package catalog seems like a nicer way to share than “just some repo on git{hub lab}”.

So letting people signal their intent, with the package, might help.

Again I feel like this is a boolean flag, or at least some package users (e.g. me) will want that, even if fancier things are also avail.

To be clear I think all sharing is awesome. I definitely don’t want to make packages into some sort of snooty club where you have to take an oath to be included at all.

node.js’s npm package system also comes with version (x.y.z). https://www.npmjs.com/package/pg

I think package catalog snapshotting might be a relatively simple iteration on what we currently have that would solve most of these problems. It would solve some of the pain and anxiety for end-users because they could depend on a specific and unchanging set of packages. It would solve the versioning pain for authors in that they could just publish new stuff and dependent packages would (presumably/hopefully) follow.

We use node.js in our production environment. A production system is very concerned about whether things will break down. :slightly_smiling_face:

Yes, and modern node package managers have locking, which is a form of snapshotting, to try and mitigate that problem.

Is it possible to store the value only in the head of the list?

May be a little bit overkill, but this guy (language creator) keeps a snapshot and SHA for each function body. Fun to watch though. https://youtu.be/gCWtkvDQ2ZI

It’s kind of like contract/type of the entire package.

That’s the problem, the head changes constantly in many algorithms. Think of recurring through a list, and sending the sublist to a different function to check something.

But as sorawee mentioned above, list? is cached in a pretty smart way and something similar may be doable for length without using too much space nor time.

The concept of head doesn’t make sense in Racket, because Racket list is immutable. cons in particular doesn’t mutate anything. Moreover, structures can be shared.

(define xs '(1 2)) (define ys (cons -1 xs)) (define zs (cons -2 xs))

@wlbberman: @ryanc’s code above is far better than the one you linked.

I just posted a link to this Udemy course on https://racket-stories.com Anyone know anything about it? https://www.udemy.com/course/racket-training/

weird

weird?

Hi! :wave:

hi

I just read that Racket ist about to switch to the Chez compiler. Does that mean that the ability to produce “binaries” is going to change? Is it better, worse? Does somebody know?

@brejoc Check https://blog.racket-lang.org/2020/02/racket-on-chez-status.html

@soegaard2 Thx for the link. Very interesting but I couldn’t extract anything regarding compilation of binaries out of it. But I found <https://cisco.github.io/ChezScheme/|this page> and it implies that Chez has the ability to generate binaries. So I guess Chez does that without additional steps!?

I believe raco exe works unchanged.

I was actually hoping for something more like a statically compiled (fat) binary like in Golang. But if raco distribute is still working, then I’m also happy enough.

The result (something selfcontained) is the same, but you won’t get a single binary. There were a thread on the mailing list recently on that topic. I think it is possible to make one, but it’s not automatic like raco exe or raco distribute .

Ah, interesting. I’ll try to find that. Thanks!

I keep getting this question about compiling Racket into portable statically-linked binaries from other people too. Maybe more aggressive static linking is something to prioritize in future infrastructure work.

(but sharing only as a tree, not as a graph, if without mutation)

How does Go handle FFI type things in that case?

(I mean linking to a C library)

@notjack to people really want statically-linked? or just single binaries?

@samdphillips good question, I’d also like to know that

@samth people want to be able to distribute command line tools, server programs, etc. as single binaries that are run in many different kinds of environments that they don’t have much control over. They want to be able to tell users “you only need to download and run one file, and you don’t need to install anything else”. So maybe not statically linking libc, but definitely statically linking any other foreign libraries your program happens to use.

I write dockerized backend services usually so that’s not something I tend to need. But for people who don’t write those, or who don’t want to be told “you have to install and learn Docker”, static linking is useful.

I think, in terms of effort, “binaries that still call dlopen when using the FFI” << “statically linking to FFI libs” << “statically linking to FFI libs and libc”, but I might be wrong about how much work the first step is

is the first of those three things the current state of racket executables?

no, the current state is that raco distribute produces something that is standalone but isn’t a single binary

ah, right, it spits out a folder of stuff

if you follow the directions here, you can manually produce a step 1 I listed: https://www.cs.utah.edu/plt/snapshots/current/doc/inside/cs-embedding.html

Had to delete that message. Slack isn’t nice to my formatting.

I’m trying to learn how Scribble is able to juggle top-level forms and markup text. Meaning that if I write this:

#lang something 1 2 @require[blah] 3 Some prose I just get the digits and prose, even though require still works. I started sloppy: eval each form read via Scribble’s read-syntax-inside in a module namespace. Then I tried hacking current-print to collect content for me within #%module-begin.

Both of those attempts came because I don’t know how to keep everything in a module context. I ended up looking at the Scribble base reader and grabbed the #%module-begin from syntax/module-reader, but I’m still not quite sure what I’m doing:

#lang racket/base (provide (except-out (all-from-out racket/base) #%module-begin) (rename-out [#%module-begin+ #%module-begin])) (require (rename-in syntax/module-reader [#%module-begin #%reader-module-begin]) (only-in scribble/reader read-inside read-syntax-inside)) (define-syntax-rule (#%module-begin+ form ...) (#%reader-module-begin racket/base #:read read-inside #:read-syntax read-syntax-inside #:whole-body-readers? #t form ...)) Is the idea that I should leverage #:wrapper1 to inspect each form and collect it into a list to later provide? Where do I go from here?

For a language like scribble/base, the #%module-begin form partially expands body forms to sort out declarations and definitions from expressions. See scribble/doclang for the implementation. It’s similar to what make-wrapping-module-begin would produce, but this #%module-begin implementation collects expressions into a (define m-id (post-process (decode (list ….)))) instead of wrapping individual expressions.

For others curious, here’s that implementation: https://github.com/racket/scribble/blob/9051e6d882b341a29dc37eb95fcf237ab0b9161e/scribble-lib/scribble/doclang.rkt

I had assumed that the declarations would all share some implicit form I haven’t heard of, but it looks like this manually checks for some identifiers. Thanks for clearing that part up for me.

Unrelated question: What does “trampoline” mean in the context of this comment?

;; `body' probably starts with lots of string constants; it's ;; slow to trampoline on every string, so do them in a batch ;; here:

The doc-begin macro works by expanding to a use of doc-begin; that’s the trampoline in this case.

If you’re asking about the term in general, this is what’s meant: https://en.m.wikipedia.org/wiki/Trampoline_(computing)\|https://en.m.wikipedia.org/wiki/Trampoline_(computing)

In particular the first entry under high level

In Clojure, count of Lists is O(1). https://clojure.org/reference/data_structures#Lists