I fear this may be a very basic question, but I’ve hit one of those points where I evidently need to know about five percent more to make sense of the documentation. Lots of Googling hasn’t helped me either.

Can someone please give me at least the skeleton of how I would use subprocess? If a concrete use case would be useful, suppose I had a list with a bunch of strings. I would like to send each list item to wc in the shell and capture the response. I’d like to do so without having to start a new process each time (that is, system is not what I want)

I know that doing this via wc would be silly, but it’s a simple shell command that easily takes standard input and emits something simple to standard output.

Thank you!!

@ghoetker I think, even if you need the power of subprocess, process*/ports is probably an easier interface to work with. http://docs.racket-lang.org/reference/subprocess.html#%28def._%28%28lib._racket%2Fsystem..rkt%29._process%2A%2Fports%29%29

@ghoetker I have an example of using process*/ports here, which spawns a git subprocess and communicates with it over stdout and stdin to check if a file is ignored by .gitignore. https://github.com/lexi-lambda/dotfiles/blob/e016f350631236ab5c7b015a1f263502d0cf3549/bin/watch-exec#L57-L69

@lexi.lambda Thank you for the quick response. Happy to go with process*/ports. Unfortunately, I’m still missing something in your example (feeling rather dense here, to be honest). If you have time, could you share a really stripped down example? Say, send the contents of a variable to wc and capture the response. Regardless thank you for your assistance.

If you’re not going to keep the process open, just use system or system*?

This code does what you’re asking for: (with-input-from-string "one\ntwo\nthree\n" (λ () (with-output-to-string (λ () (system* (find-executable-path "wc"))))))

You claim you “don’t want to start a new process each time”, but as far as I can tell, wc isn’t useful as a long-running process, so I don’t think you can get away with not starting a new process each time.

Thanks for the quick answer. My choice of wc as the command may have been poor. I wish to keep the process open, thus the interest in process*/ports.

My actual problem relates to the code below, which I need to run a string through base64 -D \| plutil -convert xml1 -o - -- -" where -o - means to output to stdout and -- - says to take standard input. base64 decodes the string from base64 and plutil is Apple’s command line tool to convert binary plists to xml plists. I suspect the echo ~a part of my command isn’t best practice. The program it this is part of takes a super long time to run and I suspect starting a new process 7000 times might be why.

Thoughts on how to do the below more efficiently or a simpler example I could build on would be awesome. Thank you so much for your help!

;; Extracting filename from a base64 string
(define (extract-filename aString)
  ;; Convert binary string to xml-formatted plist
  (define xmlPlist
    (with-output-to-string
      (lambda ()
        (system (format "echo ~a \| base64 -D \| plutil -convert xml1 -o - -- -" aString)))))

(close-output-port (current-output-port))

other stuff...

Can plutil accept multiple files from stdin? How does it delimit them?

@lexi.lambda

Hopeing I’m answering your question properly, either

plutil -convert xml1 -o - -- file1 file2

or

echo file1 file2 \| xargs plutil -convert xml1 -o - -- -

work. The -- option flags that everything that follows is a file name.

From that, it looks like you’ll still have to spawn plutil once per file, so I don’t think you can get away with just spawning it once.

@lexi.lambda Bummer. Many thanks for the help, however. I will return to your original example and see if I can tease more out of it over time. Very appreciated.

It does seem possible that the fact that you’re sending all your data through the shell via echo is causing some unnecessary loss of performance. You might still be able to get things faster by avoiding the shell and doing the base64 decoding in Racket, then sending your data to plutil’s stdin directly.

@ghoetker Here’s a program that streams base64-decoded data to plutil: #lang racket (require net/base64) (define (base64-binary-plist->xml in) (define-values [plutil-stdin-in plutil-stdin-out] (make-pipe)) (thread (λ () (base64-decode-stream in plutil-stdin-out) (close-output-port plutil-stdin-out))) (with-output-to-string (λ () (parameterize ([current-input-port plutil-stdin-in]) (system* (find-executable-path "plutil") "-convert" "xml1" "-o" "-" "--" "-"))))) (call-with-input-file "/tmp/binary-base64.plist" base64-binary-plist->xml #:mode 'text)

I don’t know if that would actually be meaningfully faster than what you’re doing, but you could try it, and it might be illustrative.