sschwarzer
2022-7-23 12:50:11

Why shouldn’t it work for booleans? Can you give an example?

Regardless, there’s still a difference between the two forms. The if form would evaluate value twice if it’s true-ish.


sschwarzer
2022-7-23 12:52:33

> (for* ([b1 '(#f #t)] [b2 '(#f #t)]) (displayln (~a "if b1 b1 b2 = " (if b1 b1 b2))) (displayln (~a "(or b1 b2) = " (or b1 b2))) (newline)) if b1 b1 b2 = #f (or b1 b2) = #f if b1 b1 b2 = #t (or b1 b2) = #t if b1 b1 b2 = #t (or b1 b2) = #t if b1 b1 b2 = #t (or b1 b2) = #t > (for* ([b1 '(#f "foo")] [b2 '(#f "bar")]) (displayln (~a "if b1 b1 b2 = " (if b1 b1 b2))) (displayln (~a "(or b1 b2) = " (or b1 b2))) (newline)) if b1 b1 b2 = #f (or b1 b2) = #f if b1 b1 b2 = bar (or b1 b2) = bar if b1 b1 b2 = foo (or b1 b2) = foo if b1 b1 b2 = foo (or b1 b2) = foo


soegaard2
2022-7-23 13:10:50

Does a #f mean “false” or “argument not give, use default”?

In that sense, this doesn’t work [How do you set blue? to false]: (define (foo [blue? #f]) ; use #t if argument not given (set! blue? (or blue? #t) ...) But the example is not the best, since this works fine: (define (foo [blue? #t]) ...)


capfredf
2022-7-23 13:12:41

> But keep in mind that plain racket-test (without the coverage stuff) is effectively the same as doing racket-run-module-at-point with point inside some test submodule. And you could use C-u with that (to get the better errortrace context). Thanks for the pointer. if I am editing test cases in the test submodule, then this is my go-to approach. However, when I am editing main parts of my code, racet-test is handier.



soegaard2
2022-7-23 13:15:22

The None value in Python bypasses this problem. But it confuses me that bool(None) is False.


greg
2022-7-23 13:27:10

What do you think about these two choices:

  1. Plain, racket-test (without C-u) should just always use high for racket-error-context, that is more, errortrace context.

OR

  1. Plain racket-test remains the same, follows the racket-error-context variable. C-u racket-test means more, errortrace context. C-u C-u racket-test means full coverage.

greg
2022-7-23 13:27:26

Neither is quite backward compatible, but maybe that’s OK?


greg
2022-7-23 13:28:17

The 2nd option seems better to me, “clean sheet of paper”, it’s closer to how racket-run-module-at-point works. (However it’s even less backward-compatible).


greg
2022-7-23 13:28:43

If you prefer either, or have some 3rd or more ideas, please let me know @capfredf.


capfredf
2022-7-23 13:30:00

I prefer the 2nd as well


soegaard2
2022-7-23 13:38:22

This doesn’t work: (require (relative-in ".." "python-c-api.rkt" "python-initialization.rkt" "python-environment.rkt" "python-evaluation.rkt" "python-types.rkt")) The problem is that relative-in needs a module path and "…" is not.

Is there a variant somewhere?


soegaard2
2022-7-23 13:38:56

For now I’ll just write "../python-c-api.rkt" etc.


greg
2022-7-23 14:06:03

Moved to https://github.com/greghendershott/racket-mode/issues/634 and pushed initial commit. I only smoke-tested it quickly, have to offline soon, back later today.


sschwarzer
2022-7-23 15:18:46

> Does a #f mean “false” or “argument not give, use default”? Ah, you mean that! :slightly_smiling_face: I was only thinking about the if/or equivalence.

But you’re completely right. I ran into this before, and have been thinking about it a lot. I’d probably use (void) as the default argument now if it was for an unknown boolean. But first we should think about if there is a sensible #f or #t default value, after all.

Another possibility would be (define (foo . list-with-arg-or-not) ..., but IMHO that’s quite awkward and it doesn’t really show well what the intention is.

Instead of (void) , an alternative would be a (gensym)-generated symbol, but if (void) works, I’d use that because there’s already void?, so the value would be easy to test against.

In any case, I think it’s a difference if the “unknown value” value is only used internally or if it becomes part of the API. In the case of un-provided arguments, it would probably be internal only (if the function is part of “our own” API), but a struct field without a clear default of #f or #t, might become public. In this case it would be possible to encapsulate the special value with a symbolic name exported by the module, but maybe that’s a bit heavy-handed. Finally, (if that’s possible), users shouldn’t be supposed to access the field value directly to begin with and only use APIs that internally use the value but never expose it; that might make the API a bit simpler.


sschwarzer
2022-7-23 15:20:22

> The None value in Python bypasses this problem. Yes, I like None for that, i.e. that you don’t need a special case/design for booleans. I had very few cases (maybe a handful in 20 years?) where I used a special value made with object() because None was a legitimate value. :smile:


sschwarzer
2022-7-23 15:23:00

> But it confuses me that bool(None) is False. Especially compared to Scheme, Python’s boolean-context evaluation is super-complicated. All kinds of numeric zero are false, as well as empty strings and empty containers like [] and {}. And classes can override __bool__ to decide when their instances should be interpreted as false or true.


capfredf
2022-7-23 15:40:06

In this case, I think a proper value to indicate an argument is not provided is any value from a set disjointed from the set of arguments. And racket developers tend to pick #f and python developers pick None. We just need to be careful if the provided inputs overlap with those values.


sorawee
2022-7-23 15:44:53

To me, there are three scenarios:

  1. When the set of possible values doesn’t include #f, you can use #f to indicate “missing” 1.5 When the set of possible values includes #f but doesn’t include like 'missing, you can use 'missing to indicate “missing”
  2. When the set of possible values includes essentially everything, and you only want to detect if an argument is missing, use gensym-based approach.
  3. When the set of possible values includes essentially everything, and you want to want users to be able to specify that an argument is missing, create (struct none ()) and let users provide (none).

sschwarzer
2022-7-23 15:44:55

Yes, I think #f is very idiomatic and therefore should be used. It’s also done that way in lots of APIs in the standard library. The other arguments only come up because of the overlap (when #f is also a “legitimate” value).


sschwarzer
2022-7-23 15:47:08

> When the set of possible values includes essentially everything, and you > want to want users to be able to specify that an argument is missing, > create (struct none ()) and let users provide (none). I think that’s essentially the same as using (define none (gensym)) (with the exception that you don’t need the application (i.e. none instead of (none) ).


sorawee
2022-7-23 15:47:58

yeah sure. The point is whether you provide none to users so that they can use it as an argument, or you keep it private in the module.


sschwarzer
2022-7-23 15:49:12

I like 1.5 in the case that the “missing” value has a meaning on its own. But in that case you might ask if the value is still something boolean or kind of “ternary.” :slightly_smiling_face:


sorawee
2022-7-23 15:52:14

Write your own requier-transformer :slightly_smiling_face:


sschwarzer
2022-7-23 15:52:58

> I think that’s essentially the same as using (define none (gensym)) (with the exception that you don’t need the application (i.e. none instead of (none) ). Actually the struct approach is nicer for an external API because the default representation of the value looks nicer than some arbitrary symbol, which may confuse users whether everything’s still alright or if they got hold of a value they shouldn’t have gotten hold of. :smile:


sschwarzer
2022-7-23 15:55:02

capfredf
2022-7-23 15:57:54

@sorawee then you might end up getting rid of those doing-almost-nothing abstractions one day. https://github.com/racket/typed-racket/pull/1114/files\|https://github.com/racket/typed-racket/pull/1114/files


sorawee
2022-7-23 16:01:43

I think it provides values here a lot more than in Typed Racket. You don’t need to type common prefix several times, and it’s kinda general enough to be used in many scenarios

In Typed Racket code that you showed, it hard codes path name into the transformer. That’s probably overly specific. But I’d argue that it can also have a use in certain scenarios.


soegaard2
2022-7-23 16:38:45

(define-syntax prefix-in (make-require-transformer (lambda (stx) (syntax-parse stx [(_ rel-path:string in:string ...) (define rp (syntax->datum #'rel-path)) (define ins (syntax->datum #'(in ...))) (define rp/ins (for/list ([i ins]) (string-append rp "/" i))) (with-syntax ([(rp/in ...) (datum->syntax stx rp/ins)]) (expand-import #`(combine-in rp/in ...)))]))))


soegaard2
2022-7-23 16:39:57

It surprised me that (datum->syntax stx rp/ins) was needed. A simple rp/ins wasn’t enough [the imports gets a wrong scope].


philip.mcgrath
2022-7-23 23:52:45

This is great to read! I haven’t looked at any code beyond the blog post, but it looked to me like there might be some intermediate allocation and generic iteration that might be avoidable. Taking this example: (define (atab->set-cover atab) (match-define (alt-table pnts->alts alts->pnts alt->done? alt->cost _ _) atab) (define tied (list->mutable-seteq (hash-keys alts->pnts))) (define coverage '()) (for* ([cost-hash (hash-values pnts->alts)] [rec (hash-values cost-hash)]) (match (cost-rec-altns rec) [(list) (error "This point has no alts which are best at it!" rec)] [(list altn) (set-remove! tied altn)] [altns (set! coverage (cons (list->mutable-seteq altns) coverage))])) (set-cover tied coverage)) Using hash-values in the for* clauses will allocate lists and then iterate over them using generic sequence operations. Trivially, you could wrap them with in-list, and in https://blog.racket-lang.org/2020/02/racket-on-chez-status.html just getting out of the slow-path sequence API gave a ×5 speedup. I think you don’t have any concurrent modification issues here, so you’d probably do even better going to in-hash-values, which avoids allocating the intermediate list, or, better yet, a more specialized versions like in-immutable-hash-values.


pavpanchekha
2022-7-23 23:55:10

That’s a good call. A lot of this code was written ages ago, when I knew less racket, so it’s no surprise. I kind of don’t think these micro optimizations will make a big difference, since this function is nearly invisible in the profile (and I do try to use the right in-x function in the tight loop) but there’s no reason not to do it either.


philip.mcgrath
2022-7-24 00:10:34

[pressed enter too soon …] Similarly, I wonder about (define tied (list->mutable-seteq (hash-keys alts->pnts))). It would at least be possible to avoid the intermediate list, but, if it’s an immutable set, I’d try rewriting the loop to use for*/fold with just hash-remove. Similarly in (set->list (set-subtract (list->seteq (hash-ref alt->points altn)) chnged-set)), I haven’t fully taken in the context, but it would irk me to convert from a list to a set and back: I’d either want to keep it as a set or do the removal without conversion. My guess is that a lot of the speedup vs. remove* is from using eq? rather than equal?, in which case remq* might get part of the way there, but you could probably write a more specialized version if you can make assumptions like that there won’t be duplicates. As you say, these may well be insignificant overall, there just the things that jumped out at me as relatively easy gains.


pavpanchekha
2022-7-24 00:12:33

I’ll try them out when I have a chance, thanks!


philip.mcgrath
2022-7-24 00:27:42

More generally, re: > Weirdly, there’s no entry for step 3! That’s because atab-add-altns calls minimize-alts in tail position, meaning that when minimize-alts is on the stack, atab-add-altns no longer is. This is one of the many reasons I think tail call optimization is a mistake, but let’s not focus on it. The point is that if you add minimize-alts back into the atab-add-altn runtime, you see that in fact atab-add-altn spends about 92% of its runtime in step 3, about 6% in step 2, and about 1% in step 1. So let’s focus on step 3, minimize-alts. I haven’t done much with profiling, but I wonder if using Errortrace rather than continuation-mark-set->context would be helpful here—but maybe the overhead would be too high, or maybe there would be other problems. Likewise, > Even after this, however, our specialzed atab-remove-alt is the biggest portion of minimize-alts. And, unhelpfully, its profile looks like this: > […] > That ??? entry is annoying, but in this case it refers to hash operations (which are probably provided from C code and therefore don’t appear in the stack). This isn’t a bit surprise, because atab-remove-alt, like most of pruning, uses immutable hash tables. That leads to a lot of copying, mostly totally unnecessary. made me wonder about how we might communicate information from the Chez Scheme profiler to Racket. (Immutable hash operations are implemented in Scheme in https://github.com/racket/racket/blob/master/racket/src/cs/rumble/hamt-stencil.sshttps://github.com/racket/racket/blob/master/racket/src/cs/rumble/hamt-stencil.ss.\|.) It also reminded me of a question I’ve had before: with lists and pairs, I think I have a fairly good understanding of how to take advantage of sharing, but I feel much less confident about the fine details of sharing with immutable hash tables.


pavpanchekha
2022-7-24 00:28:50

I had no idea immutable hash tables had sharing


pavpanchekha
2022-7-24 00:29:19

And yeah better profiling info, including tail call crap, would really help a lot in optimizing Racket code


pavpanchekha
2022-7-24 00:29:36

So would a proper flame graph profiler, instead of gprof…


philip.mcgrath
2022-7-24 00:45:20

If you haven’t seen it, https://cisco.github.io/ChezScheme/csug9.5/system.html#g118 has information on the profiling available at the Chez Scheme level.


samth
2022-7-24 03:26:42

In your case, since there is one particular tail call that’s annoying, it’s easy to defeat by just adding a let binding or a call to the identity function


samth
2022-7-24 03:27:26

For flame graphs, there’s a package that does it by @asumu I think


pavpanchekha
2022-7-24 04:33:03

Yeah, I know how to avoid the tail call. At that point it’s easier to just read the profile and mentally add them. The tricky thing is that it can mislead you while you’re still reading the profile, since you don’t know which calls are tail calls and what you’re missing


pavpanchekha
2022-7-24 04:33:43

It’s easy to say, oh, what’s slow, is it X, and it like like it’s not, and go look at something else, when it really is X, it’s just that X appears under many names