I ran the shell script on the webpage. The original version was whatever I got from apt
oh great. Now I’m going to suspect everything I get from apt. I think I need to go back to getting software from the authors to be sure I have the current version. (to avoid bugs and known vulnerabilities)
~…and of course I haven’t been able to reproduce the problem in an minimal example—and still can pinpoint the issue in my original code~ :confused:
Post to GitHub, maybe? (if your code can be shared publicly, that is)
Aha, I do have a MWE (or MFE maybe): https://gist.github.com/Metaxal/def826defa710617a16ef5d6c3f5f7dc With (read in) it takes twice as long as with (with-input-from-string (read-line in) read) . Not sure exactly what’s happening. I think a single message is not enough for read alone.
This is on Ubuntu 20.04LTS, Racket v8.4.0.3 [cs]
This should work on Unix, as long as /usr/bin/env racket works, but not Windows.
FWIW, I got:
error writing to stream port
system error: Broken pipe; errno=32
context...:
/Users/sorawee/playground/server-worker.rkt:54:2
body of (submod "/Users/sorawee/playground/server-worker.rkt" worker)
body of top-level when running the variant with read-line. It’s sporadic though. Not always happening. And I don’t know if the variant without read-line suffers the same problem.
A broken pipe probably means that something broke the worker before that. Don’t you have another error message above?
nope
It appears to happen after the main script ends…
oh, that’s fine. It’s just the cleaning up that’s rushed
Probably the (sleep 2) is either skipped or not long enough for the workers to terminate, but we should actually kill them and close the ports instead
The cleanup should be fixed now (I just killed the workers)
Here’s a version that uses thread instead of subprocess, and still has the behavior that you describe. Just trying to make it self-contained further.
#lang racket
#\|
Save this file to "server-worker.rkt"
then run
$ time racket server-worker.rkt
Then toggle the commented expression in `read-msg` and again
$ time racket server-worker.rkt
Result:
First run takes ~20s, while second run takes ~10s (as it should).
\|#
(struct worker (thd in out)
#:property prop:evt
(λ (self) (wrap-evt (worker-in self) (λ _ self)))
#:transparent)
(define (make-worker)
(define-values (in-for-in out-for-in) (make-pipe))
(define-values (in-for-out out-for-out) (make-pipe))
(worker
(thread
(λ ()
(send-msg 'ready out-for-in)
(let loop ()
(define cmd (do/debug (read-msg in-for-out)))
(sleep 2)
(send-msg (list 'done cmd) out-for-in)
(loop))))
in-for-in
out-for-out))
(define-syntax-rule (do/debug cmd)
(let ()
(define-values (res cpu real gc)
(time-apply (λ () cmd) '()))
(eprintf "~a cpu: ~a real: ~a gc: ~a\n" (~a 'cmd #:min-width 50) cpu real gc)
(apply values res)))
(define (send-msg msg out)
(writeln msg out)
(flush-output out))
(define (read-msg in)
;; TOGGLE BETWEEN THESE TWO:
(read in)
#;(with-input-from-string (read-line in) read))
(module+ main
(define workers (build-list 4 (λ _ (make-worker))))
(for ([t 20])
(printf "t: ~a\n" t)
(define wk (begin #;do/debug (apply sync workers)))
(begin #;do/debug (read-msg (worker-in wk)))
(begin #;do/debug (send-msg (list 't t) (worker-out wk))))
(for-each (λ (w) (kill-thread (worker-thd w))) workers))
So you do observe the issue too, it’s not just me?
yep
compared between read-line and read, the one with read-line are mostly 0s.
The one with read have a lot of 2000ms, with occasional 8000ms
And you sleep for 2s
read-line has 2s on the server side, on apply sync ....
Yes indeed. i see 4000 and 6000 too
But read has significant multiple of 2s pretty much every command.
I think one more write unblocks a previous read or something
like buffer issues
Does (open-output-nowhere) need to be closed? The docs don’t say
More importantly, can something like with-limits with a timeout prevent with-output-to-file to close the output port when raising a resource exn? I’d expect not, but I’m getting the dreaded too many files opened error
As a starting point for any server that wants accept multiple “connections” (whether that’s TCP or some other protocol), I’d recommend following the examples in https://docs.racket-lang.org/more/index.html
And for instance there’s a good example of timing out something with a watcher thread, and freeing resources with a custodian, by the time you get to section 7 https://docs.racket-lang.org/more/index.html#%28part._.Terminating_.Connections%29