Hi Mason - I’m never sure which threads will provoke a response in different channels, but it’s possible that you’l get more of an answer on the mailing list.

For what it’s worth, I haven’t heard performance complaints for plain Racket in the last 5 or so years in the way that I used to, and I think it’s because plain Racket’s performance mostly caught up.

I keep writing “plain Racket”, because performance remains challenge (and I still sometimes hear complaints) for our more ambitious Racket variants, like Typed Racket and its sound interaction with untyped code.

As for the Racket-on-Chez buzz, maybe my message has gotten through: the Racket-on-Chez project will succeed or fail in the near term based on internal merits, like the structure and maintainability of the system. Those are good things, but they’re less exciting than an automatic performance boost.

@leif to clarify, it’s definitely a bug, I was just commenting on what kind of bug it was

@greg I’m confused about what sense identifier-binding handles plain contract-out ok

@samth In the sense that, in that case, the nominal-source-id returned by identifier-binding is the original non-contract-wrapped id symbol.

ah it’s just the right symbol

not the binding

Yes, the discussion above was all in the context of “making a go-to-definition tool”.

For which, if you know the correct file, and know the correct id symbol used in that file, you can find it.

that seems not true in general to me

but probably it works for your purposes

Well yes with this kind of tooling it seems often a thing isn’t true in general, but is true often enough to be useful. :slightly_smiling_face:

I think the issue is that while rename-out works by design, what you’re doing for contract-out only really works by accident, and so minor variations don’t work

Empirically the following is useful (but maybe just accidentally): (define (real-name id) ;symbol? -> symbol? (or (object-name (with-handlers ([exn:fail? (λ _ #f)]) (eval (namespace-symbol->identifier id)))) id)) It seems to reliably answer the question, what was the original id symbol <foo> behind provide/contract-id-<foo>.0.

Reliable enough for me to use that, instead of walking unexpanded file syntax looking for (contract-out [rename _ _ _]) forms — which is what I’d been doing and is obviously a bit slower.

Anyway, my main sad at the moment is that I have to walk fully-expanded syntax on the target file to discover definition sites, and that full expansion isn’t cached anywhere AFAIK. https://github.com/greghendershott/racket-mode/issues/288#issuecomment-403327777

@greg couldn’t you cache it yourself?

either from the previous time you did this, or by changing the eval-handler to do that

Maybe. What I wrote at that link: > - If expand is (for some files) so slow, can we avoid it? I don’t see how. It’s quite common for definitions to be created by “definer” macros, and these definitions can only be discovered in fully-expanded code. > > - If we can’t avoid it, could we at least shift the cost earlier than <kbd>M-.</kbd>? Could racket-mode generate expansions ahead of time — storing them under compiled/ in (say) .expanded files, or in a new expanded/ subdir? In theory, sure. But: > > - This seems like something that should be done by a raco tool that people explicitly opt in to, much like raco make. (Note there exists a raco expand tool — however its output isn’t syntax with the source location information that we need, it’s plain text syntax->datum of the expansion.) > > - The file that must be expanded is the “target” of the find-definition — the file identifier-binding says the definition is in. Even if racket-run were to cache the full expansion of the file being run, that’s often not where the definition is. Even if you were to raco some-new-tool all the files in your project, that won’t speed up finding definitions in other projects.

right

but racket-mode could still maintain some caches

Maybe my comment above is “well it wouldn’t be perfect”, which contradicts what I was just saying about “well it’s tooling and the perfect shouldn’t be the enemy of the good”.

for example, by changing current-eval to call expand, cache that result, and then call the underlying eval

Yes I could do it for the file currently being run. People are paying for that to be expanded anyway, may as well save that result in case they try to visit something used/defined in that same file. That’s a common case.

also i think there’s more you can do to get information from zo files

Oh! I just assumed “bytecode, there’s won’t be any srcloc info”. There might be?

there’s lots of interesting stuff in bytecode

That’s a corner of Racket I’ve barely looked at.

for example, try raco decompile on some file with some definitions, and you’ll see source locations for most functions

Huh. Wow. OK.

it’s not obvious to me what part of the structure has those, but it might be the name field of lambdas

Thanks I’ll dig in and take a look!

Until now, my only thoughts about zos have been, if they get stale, can I use them as croutons.

@greg re contracts, I think the exported names are bound to applicable substructs of provide/contract-info from racket/contract/private/provide, and that struct stores the original name (identifier). Maybe ask @robby if it’s safe to rely on that?

Sorry: what’s the behavior you’d like to rely on, @greg ?

I’m probably okay to guarantee things about names working right, but I should probably add some tests and docs to make sure that that it stays that wya.

@robby the discussion earlier was about how to get from a use of a provide/contract’d name to the original function in racket-mode’s version of Jump to Definition. I was suggesting to @greg that the provide/contract-info struct might be useful. (But now thinking about it again, I’m not sure if it’s useful when looking at expanded code.)

I don’t think that check syntax relies on that?

oh, but maybe that’s because it doesn’t work?

I once wrote some code that used it in a module-begin macro to do contract-aware analysis, so I think it works in that context. Just maybe not for a Check Syntax-like tool.

@robby When the definition is from another file, in DrRacket I usually get the choice, “Open Defining File”. However in racket-mode it tries to find and go to the actual location within the file.

This means (say) trying to find the source-id or nominal-source-id reported by identifier-binding, in a define-values in fully expanded stx for the file.

The issue is with (contract-out [rename orig new] ..), identifier-binding doesn’t report orig. It reports new and something like provide/contract-id-orig.0. Neither of which is in the file.

How to get from provide/contract-id-orig.0 to orig is the question. I used to walk non-expanded file syntax looking for the contract-out / rename form. More recently I noticed that object-name seems to work. Ryan wondered if the struct subtype accessor might provide this.

(one note: it says “open defining file” when it has not yet expanded the tab that contains the definition (or the file isn’t open), so that’s kind of a red herring here)

Check Syntax looks at the fully expanded program and tries to find the identifier, but I see that it also doesn’t work.

Specifically, for this program:

#lang racket/base
(require "tmp2.rkt")
abc
pqr

with tmp2.rkt:

#lang racket

(define abc 1)
(provide
 abc
 (contract-out (rename abc pqr integer?)))

It goes to the right place for abc but not pqr

I’m not sure this is check-syntax/emacs-mode’s fault. It may be that contract-out’s expansion needs to change.

Thanks for the reply and clarification! I’d like to give racket a go someday, I find the philosophy of focusing on DSLs very interesting but as a physicist who needs high performance I’ve never really found a use case for me to delve into racket.

@robby I may be missing something, but isn’t that true even if contract-out doesn’t use rename? I guess maybe the “Jump to Definition (in Other File)” option might work without the rename, but that seems largely accidental, since IIUC Jump to Definition looks at the result of read, not the fully-expanded program? I am obviously less familiar with this than you are, though, so I could be wrong.

@mflatt Is s-exp->fasl supposed to fail when when current-write-relative-directory is a cons pair? Such as in:

(current-write-relative-directory (cons (build-path "/" "Users" "leif")
                                        (build-path "/" "Users")))

(fasl->s-exp (s-exp->fasl (build-path "/" "Users" "leif" "foo.rkt")))

Or is that a bug?

@lexi.lambda it looks at the fully expanddd program

The define pop down doesn’t

ah! I didn’t realize they were different, but in retrospect, I don’t know why they’d have to be the same.

It would be better if they were the same, I think.

and yeah, it seems like the rename-out is not necc. to demonstrate the error.

just to be clear, I think that what’s happening is that check syntax figures out what file the definition is in, but can’t find the definition, so it just goes to the file (and leaves the insertion point alone).

but something is fishy, as I think I used this feature (and it worked) the other day.

maybe there is more to the bug

The documentation of match says that this pattern should work: (struct struct-id (pat …))

However I can’t get this to work: http://pasterack.org/pastes/21677

@soegaard2 You just want bird, not struct:bird.

But that won’t work here because you’ve shadowed bird.

Well, normally. I’d like to know what I can use in the match pattern, when I have shadowed the name.

I don’t think there’s a way to do it if you’ve shadowed the name; match needs the static information bound to that identifier.

Do I really need to use #:extra-name ?

Can’t you just rename the local variable so it doesn’t shadow the outer name? I’m unsure what you’re asking for.

That’s the sensible thing to do.

I was just surprised that I couldn’t use struct:bird which is bound to the struct type descriptor.

struct:bird is bound to the runtime structure type descriptor (that is, a value that satisfies struct-info?).

bird is bound to the structure type transformer binding, which is what match uses.

The difference between them is confusing.

:extra-name worked

@samth So reading the zo is easier than I expected, after grokking parse/zo-structs.rkt. I can walk all the mods (including submodules) and find lams and get srcloc. It works, except the srcloc for definitions created by macros is wrong. It’s the location of the macro definition, not the location of the usage of the macro.

Then I noticed mod-binding-names, which is a hashtable of phases to hashtables of symbols to syntaxes. And the srcloc there is correct, even for definer macro usages! And it’s just a simple hashtable lookup! Woot.

BUT. That’s only for the value of mod-binding-names in the file module. For submodules, the value of mod-binding-names is something like #96#. Um…. I’m not even sure what that means — is it something to do with cycle detection??? I don’t see the symbol 'sub in any hash table in the parsed zo when the rkt file has (module m racket (define (sub x) x) (provide sub)). So it doesn’t seem like #96# references something elsewhere?

Oh the #96# is a graph structure reference: https://docs.racket-lang.org/reference/reader.html#(part._parse-graph)

But in the zo-parse output, I have #96=#hash().

It’s empty. Which fits with me not seeing 'sub anywhere in the zo.

http://pasterack.org/pastes/91585

I think only @mflatt can answer this question

@leif It does look like s-exp->fasl is broken for that case

@greg Warning before other answers: parse/zo-structs is unusual in that it promises to change with the version. It’s very different for v7, because it’s at the linklet level instead of the module level.

I may be starting to forget, but I think the mod-binding-names field won’t have all defined names. It has a mapping only for names that have to be made up to hide them, such as when the definition is macro-introduced. If (define (sub x) x) appears in the program, then I think there’s no empty for 'sub, but if (define (sub x) x) is macro-introduced so that an unreadable name sub.1 is made up, then 'sub.1 will make to 'sub in the table. Is that consistent with what you’re seeing?

Well I’m on 6.10 at the moment (matches something I have deployed, waiting to upgrade). With the zo for this file: https://github.com/greghendershott/racket-mode/blob/various-changes/test/defn-examples.rkt

I’m seeing mod-binding-names entries for all the file module bindings, including things like plain.

I’m seeing #96# => #96=#hash() as the mod-binding-names for all the submodules in the file, like sub or red-herring.

I don’t have an example of a macro-introduced definition in a submodule. I’ll try adding one to see if that “forces” a non-empty hashtable for the submodule.

It doesn’t.

Welp, maybe this behavior means I should shelve this idea.

I did notice the warning about version changes, but was willing to maybe try to track along with those if the result was worthwhile.

I can at least do some more work on caching fully-expanded code.

Maybe for RacketCon day 2 we could have a little session about tooling stuff.

Apparently I misremember, so I’ll look a little more

Thanks but don’t spend too much time. The immediate motivation is to make go-to-definition on some sources files be “instant” instead of take a few seconds because fully-expanding them is slow.

I love “instant” as much as the next person but there’s a cost:benefit to everything.

Adding a macro-introduced definitition does seem to trigger the table When I change the sub submodule to (module sub racket/base (define (sub x) x) (define-syntax-rule (q) (define other 8)) (q) (provide sub (rename-out [sub sub/renamed]))) then the table for sub has sub and q. So, I think it’s that when there’s any identifier like a macro-introduced one, then the table is generated. Otherwise, it can be synthesized from the module content easily enough.

@mflatt Is that on 6.12, or 6.90-ish, or other?

6.12

OK I’ll try with that, too.

@leif I have a repair to push for s-exp->fasl

(after running more tests)

@mflatt Cool, thanks.

BTW, is there any way to get the location of your current file (if its on your filesystem) at syntax time?

Like: (begin-for-syntax (writeln (quote-module-path)))

Will just return ’<modname>

Anyway, without that it seems like its not possible to do something like, say, dynamic-require at phase > 0

I think you want (variable-reference->module-path-index (#%variable-reference)), but I don’t know if there’s a prettier wrapper along the lines of quote-module-path.

I actually tried that first.

But if, say the file is in: /Users/leif/bizzle.rkt, then:

(begin-for-syntax
  (writeln (variable-reference->module-path-index (#%variable-reference))))

gives #<module-path-index='bizzle[7427]>

Maybe I’m missing something, but how about just: (begin-for-syntax (writeln (syntax-source #'here)))