@shu—hung I’m not sure it fits with the Bee but it does sound like a valuable contribution - thank you for thinking of it. The Bee is a community effort to enhance <https://docs.racket-lang.org/syntax-parse-example/index.html|Syntax Parse Examples (link)> as a learning resource for syntax parse macros. Maybe submit one example for discussion with @ben ?

How can Python pandas be faster than Racket especially w/Chez? I would guess there should be nothing Python does that Racket couldn’t do better. Is Python taking shortcuts, playing fast and loose somewhere?

It’s using C and/or Fortran (AFAIK), as commented previously

Plus pandas has been under development and widespread use for some time: that naturally leads to lots of work on improvements, including performance.

Likely many factors all adding up together. Some are easy, some are not so easy to compete with. Obviously profiling is needed, but a rough list of likely candidates:

1. Pandas has chunks written in C. Along these same lines, ability to use SIMD, etc.
2. Multi-threading. There are many dataframe operations that benefit tremendously from multi-threading (e.g. folding multiple columns at once). This is something I plan to do still, assuming Racket is up for it.
3. Algorithm choices that I have no control over (e.g. sorting) without hand-rolling.
4. Message dispatching (think using __lt__ in a sort vs. a define-generic less-than? in Racket). Those built-in method dispatches in Python have been optimized to hell-and-back, likely being optimized to a specific bytecode instruction at this point, so there’s no need to even look it up.
5. Memory allocation and GC strategies. For example, Python + Pandas will rarely collect memory and will just grow in the name of performance.
6. Applying functions (read cons’ing + apply) instead of optimizing the <= N cases, which I can all but guarantee that Pandas does. And that’s just off the top of my head. I have plans for #2 and #6.

Thanks @massung, great list. I wonder, for #1 (and maybe #4), if Racket could write “safe C/Rust/whatever” so you’d get the performance, without having to have a C/Fortran programmer pore over C/Fortran or special-case things. Because Racket is a Lisp, it could do AI-like things, such as choose foreign libraries it “likes/trusts” based on capabilities, and possibly create new optimized but reliable native code when existing wheels have not already been invented.

Using FFI unfortunately blocks the VM. A program could move the FFI portions into a separate place but then there would be communication overhead to the place. IIRC in 8.1 CS there is now call-in-os-thread and friends but they have restrictions that need to be worked around.

I’m very happy to be wrong!

Re-posting this from Discord, in case someone knows the answer: I have a file private/for-jsond.rkt that starts #lang jsond, but raco setup --tidy --avoid-main --check-pkg-deps --unused-pkg-deps --pkgs <PACKAGE> reports jsond as an unused dependency. How do I fix that? The file exists because I thought it would make the dep explicit; the dep really shows up at runtime in generated files, if that matters. (Is this because I have the local development version of jsond installed?) (module-recorded-dependencies "private/for-jsond.rkt") from compiler/depend includes ~/path/to/jsond/main.rkt

Ah, sillily, changing to #lang racket/base (require jsond) works just fine.

It may be bc the expansion of a jsond module has no references to jsond?

Question about define-syntax-rule (or syntax templates, more specifically). I want to have “keywords”, so I can write something like

(dada-test (a b c) (true-when ()) (false-when (atomic))) My first try was something like this:

(define-syntax-rule (dada-test place-term (true-when read-holds-atomic-term ...) but I think that true-when there is becoming a binding, not a “keyword”. How do I do that?

You can’t do that with define-syntax-rule since it doesn’t let you specify “literals”, but you can with syntax-rules and syntax-case:

(define-syntax (true-when stx) (raise-syntax-error 'true-when "may only be used within a dada-test form" stx)) (define-syntax (false-when stx) (raise-syntax-error 'true-when "may only be used within a dada-test form" stx)) (define-syntax (dada-test stx) (syntax-case stx (true-when false-when) [(_ (pt ...) (true-when ()) (false-when ())) #'(void)])) (dada-test (a b c) (true-when ()) (false-when ())) Also with syntax-parse:

(define-syntax (dada-test2 stx) (syntax-parse stx #:literals (true-when false-when) [(_ (pt ...) (true-when ()) (false-when ())) #'(void)])) (dada-test2 (a b c) (true-whens ()) (false-when ())) I’d recommend going with the latter since it gives better error messages (try replacing true-when with something else in both examples to see what I mean).