Why is it that if I do a raco setup --clean --pkgs s10, it will issue a clean on all packages instead of just s10? I started noticing this after making the racket directory read-only.

@pocmatos I’m not sure. By “all packages” do you mean the entire Racket installation? I tend to be set-and-forget with raco commands because I usually have a Makefile for each project. The clean target is something like: # Note: Each collection's info.rkt can say what to clean, for example # (define clean '("compiled" "doc" "doc/<collect>")) to clean # generated docs, too. clean: raco setup --fast-clean --pkgs $(PACKAGE-NAME)

(And in Emacs I have a keybinding for projectile-compile-project. So I type that, and type e.g. make setup or make clean (it remembers the last-entered).)

I do very similar… my output looks like: raco setup --fast-clean --pkgs s10 raco setup: version: 6.12 raco setup: platform: x86_64-linux [3m] raco setup: installation name: 6.12 raco setup: variants: 3m raco setup: main collects: /home/pmatos/installs/racket-6.12/collects raco setup: collects paths: raco setup: /home/pmatos/.racket/6.12/collects raco setup: /home/pmatos/installs/racket-6.12/collects raco setup: main pkgs: /home/pmatos/installs/racket-6.12/share/pkgs raco setup: pkgs paths: raco setup: /home/pmatos/installs/racket-6.12/share/pkgs raco setup: /home/pmatos//.racket/6.12/pkgs raco setup: links files: raco setup: /home/pmatos/installs/racket-6.12/share/links.rktd raco setup: /home/pmatos//.racket/6.12/links.rktd raco setup: main docs: /home/pmatos/installs/racket-6.12/doc raco setup: --- cleaning collections --- raco setup: deleting: in <pkgs>/eli-tester/tests delete-file: cannot delete file path: /home/pmatos/installs/racket-6.12/share/pkgs/eli-tester/tests/compiled/eli-tester_rkt.dep system error: Permission denied; errno=13 ...

followed by long backtraces.

I am curious, is your racket folder writable? I have purposefully done a chmod -w -R /home/pmatos/installs/racket-6.12. All started falling apart, which as quite surprising.

I assume it’s writable — whatever default would be on macOS.

I’m surprised only because I do “clean” fairly frequently, and the subsequent “setup” is not taking the many many many minutes it would if it were rebuilding all of Racket.

(And/or, the execution of my program isn’t taking the many many minutes it would, if there were no zos and it were expanding the universe from scratch :smile:)

So honestly I never looked to carefully into what --fast-clean cleans exactly; it’s just seemed to do what I expected.

@pocmatos Is s10 actually installed as a package — i.e. do you see it in raco pkg show?

yes, it shows up. it’s a dev install - link.

also, if I do raco setup --pkgs s10, it succeeds but I get:

raco setup: WARNING: with-output-to-file: error deleting file
  path: /home/pmatos/installs/racket-6.12/share/info-cache.rktd
  system error: Permission denied; errno=13

I always install my rackets in my home dir, but I wonder how it works if it’s installed system-wide. This kind of thing would never work, that’s why I am surprised this is happening.

Unless racket just does this when it detects it’s installation to be in a user home.

Is the double slash in /home/pmatos//.racket/6.12/pkgs correct and if so what does that mean?

(Also /home/pmatos//.racket/6.12/links.rktd)

I am not sure why the double slash shows but that’s correct. it’s basically the racket home dir in $HOME/.racket

@pocmatos Sorry, I don’t know. Just FYI, make clean on frog for me does: make clean raco setup --fast-clean --pkgs frog raco setup: version: 6.10 raco setup: platform: x86_64-macosx [3m] raco setup: installation name: 6.10 raco setup: variants: 3m raco setup: main collects: /Applications/Racket_v6.10/collects raco setup: collects paths: raco setup: /Users/greg/Library/Racket/6.10/collects raco setup: /Applications/Racket_v6.10/collects raco setup: main pkgs: /Applications/Racket_v6.10/share/pkgs raco setup: pkgs paths: raco setup: /Applications/Racket_v6.10/share/pkgs raco setup: /Users/greg/Library/Racket/6.10/pkgs raco setup: links files: raco setup: /Applications/Racket_v6.10/share/links.rktd raco setup: /Users/greg/Library/Racket/6.10/links.rktd raco setup: main docs: /Applications/Racket_v6.10/doc raco setup: --- cleaning collections --- raco setup: deleting: in <pkgs>/frog/frog raco setup: deleting: in <pkgs>/frog/frog/config raco setup: deleting: in <pkgs>/frog/frog/config/lang raco setup: deleting: in <pkgs>/frog/frog/config/private raco setup: deleting: in <pkgs>/frog/frog/private raco setup: deleting: in <pkgs>/frog/frog/private/enhance-body/add-doc-links raco setup: deleting: in <pkgs>/frog/frog/private/enhance-body/syntax-highlight raco setup: deleting: in <pkgs>/frog/frog/private/upgrade After which my other pkgs, both personal and Racket-supplied, still have zos/deps — they were not deleted.

By any chance does your info.rkt for s10 have a (define clean ___) that includes eli-tester ??

@pocmatos Oh. Hmm. Does s10 happen to have a tests subdirectory?

Because eli-tester does: https://github.com/racket/eli-tester/tree/master/tests

I guess what I’m getting at is the docs say, bold mine: > —pkgs ‹pkg› … — constrain setup actions to collections that are within (or partially within) the named ‹pkg›s.

hummm, i have a tests subfolder but nowhere I have a (define clean __).

I wonder if s10 and eli-tester packages could somehow both be contributing to a tests collection

but interestingly I don’t even know what eli-tester is, I imagine it’s a dep of rackunit but I don’t use eli-tester directly at all.

And that “intersection” causes --fast-clean to try to clean both contributing packages.

hummm, interesting.

I don’t know if that even makes sense ¯_(ツ)_/¯

I do have a tests directory that doesn’t define a collection name.

if it defaults to ‘tests’ then maybe it becomes part of eli-tester?

what are you doing in frog? will check…

oh, you have no tests dir…

rosette has a test dir instead of tests. I will probably clash with rosette if I rename my folder to test.

was not aware that collections could clash like this. I expected all to be somehow encapsulated by the project name, although I know little about package/collection internals.

maybe I should name by tests as s10-tests and retry.

you can have multiple packages that all have tests directories

But if they’re packages whose info.rkt has (define collection 'multi), don’t each of the top-level directories end up being collections of those names?

yes, but collections can have files from multiple packages

officially confused

This is a feature I’ve never wanted to use for the pkgs I’ve happened to write, so, I’ve always found it a bit confusing.

But IIUC one use case is a data collection

And you want different pkgs to be able to contribute different data structures.

so a collection name can clash with any package known by racket?

I’m not sure what you mean by clash

if i create a package with a collection called frog, add it to pkgs, will I cause problems when frog tries to build?

no

how come my tests folder, seems to require eli-tester to be recompiled?

do your tests depend on eli-tester?

no.

then that seems weird

they might depend on something, that depends on eli-tester though. Is there a way to print the dep tree?

there’s a gui visualizer by mflatt that’s a package

This is what I have on my root info.pkg: (define deps '("benchmark" "profile-flame-graph" "profile-lib" "errortrace-lib" "aws" "rosette" "htdp-lib" "parser-tools-lib" "anaphoric" "rackunit-lib" "base")) (define build-deps '("scribble-lib" "quickcheck" "rackunit" "racket-doc"))

you mean this: https://pkgs.racket-lang.org/package/gui ?

no

https://github.com/mflatt/pkg-dep-draw

@samth It looks to me that the eli-tester package contributes an eli-tester.rkt module to the tests collection. So e.g. to use it I must (require tests/eli-tester) If @pocmatos has a (define collection 'multi) in the info.rkt for his s10 package, and he has a tests top-level subdir, then that means he is also contributing stuff within to the tests collection. Therefore, that is why raco setup --fast-clean --pkgs s10 is trying to delete zos/deps from eli-tester. The docs say (bold mine): > —pkgs ‹pkg› … — constrain setup actions to collections that are within (or partially within) the named ‹pkg›s. And in his case, that’s failing because read-only Racket install dir. ^^^ This is my “theory of the bug a.k.a. surprising behavior” :smile: Yay? Nay?

yes, that sounds plausible

that --pkgs p means do things to all of any collection that p has some files in

@pocmatos So depending on what s10 is, one simple thing might be to move almost everything except README and info.rkt down into an s10 subdirectory of your existing s10 “project” directory. That way, your tests folder is a s10/tests collection instead of a tests collection. In fact that foo/foo/* layout is probably a good pattern for any (define collection 'multi) that’s really just supplying a single collection. OR, use the single-collection incantation I can’t remember because it didn’t exist when I first learned about packages. :slightly_smiling_face:

@pocmatos is there actually a problem that needs to be solved here?

https://racket.slack.com/archives/C06V96CKX/p1528206888000206

TL;DR raco setup --clean or --fast-clean fails because his Racket dir is read-only and eli-tester is in there

We were both confused how/why that could be.

@greg @samth sorry, was interrupted by my kids. still need to investigate what’s going on on this side.

what if you pass --avoid-main?

@greg i think you are on the money with your explanation regarding the problem.

Might need a s10/s10 directory structure then.

I really think you don’t need to change anything

--avoid-main seems to do the trick.

@samth it seems a should do something, I don’t think it’s the correct behaviour to have racket confuse my tests with eli-tester tests collection.

using --avoid-main looks to me as being just a workaround.

it’s not confusing things, it’s just that --pkgs s10 doesn’t have as narrow a scope as you might want

But at the same time, racket thinks that there’s a connection between my tests and eli-tester tests so it sounds like I should clarify that. Not just for now, but in the future, I certainly wouldn’t want another sudden surprise because my tests and eli-tester tests files are in the same collection.

no, Racket doesn’t think there’s a connection

what’s happening is that raco setup --pkgs s10 looks at the s10 package, and sees what collections are in it, such as s10 and tests, and then does the equivalent of raco setup s10 tests

ICYI https://github.com/LeifAndersen/racket-_/tree/master seems to be an example of a single-collection package <=> https://pkgs.racket-lang.org/package/_

i understand that. but when it does raco setup ... tests it seems confusing to me, that racket goes pick up eli-tester tests collection to clean/setup.

Is it because a collection can have multiple root directories contributed from different packages, so when one root directory is cleaned or setup, all other root directories receive the same treatment in case there are cross-references in documentation files?

@pocmatos raco setup tests means “run setup in all the places that have a tests collection”, which happens to include eli-tester (but there’s nothing special about eli-tester in this)

Understood but the downside is of course, that if I have other packages installed which happen to have a tests collection, then I am suddenly cleaning/rebuilding those collection when it was not my initial intention to do so.

right, the implementation isn’t as precise as it could be, and so it sometimes does extra work (and this is more annoying for --clean)

I think it works this way because raco setup is collection-oriented

but I’m not certain

ok.

i certainly need to understand this better but I think this discussion was already a great help.

@samth quick update on immutable vectors: I have a small program that takes a long time to typecheck, and the time seems to be spent in infer.rkt when the args to infer are: X = (g49724 g49725) Y = () S = ((U (Immutable-Vectorof Exact-Rational) (Mutable-Vectorof Exact-Rational))) T = ((U (Immutable-Vectorof (U g49724 g49725)) (Mutable-Vectorof (U g49724 g49725)))) R = g49724 (edit: simplified S,T — its still slow with this smaller list of arguments)

its much faster if the program uses only Mutable-Vectorof, and also if I solve for each var in X one-by-one

do you know if (1) this is just a bad question to ask infer in the first place or (2) there might be a way to get the answer faster ?

that should definitely be fast

if you change Exact-Rational to String is it much faster?

can you write a program that just makes that call to infer?

I’ll try those things — how would I make a program that just calls infer? (serialize the arguments?)

you just require various parts of typed racket

there are unit tests for infer that should be helpful as a guide

ok, finally, this test case is slow (15 seconds): (infer-t (Un (-ivec -Rat) (-mvec -Rat)) (Un (-ivec (Un (-v a) (-v b))) (-mvec (Un (-v a) (-v b)))) #:vars '(a b)) and changing -Rat to -String makes it fast

maybe I did something bad in my changes to infer.rkt here?: https://github.com/racket/typed-racket/pull/575/files#diff-ca205189ac10d7d91266b647820cbe6a

hmm, those changes seem fine

what if you change (-ivec E) to (-pair E (-val null))?

still slow

and this test takes 15 seconds for me on TR master: (infer-t (Un (-pair -Rat (-val null)) (-vec -Rat)) (Un (-pair (Un (-v a) (-v b)) (-val null)) (-vec (Un (-v a) (-v b)))) #:vars '(a b))

ok then it seems not to be your fault

I’m trying now to open a separate issue .. maybe we can get this fixed in a few days

because I would like to merge the vector PR before the 7th, but not if its going to slow down the math library like it does

you might look at the call trace of infer