Junk Drawer Logo Junk Drawer

For all those little papers scattered across your desk

Stop, sed i!

D. Ben Knoble on 06 Aug 2020 in Blog

Right tool for the job people, right tool for the job :roll_eyes:

What am I talking about?

I’m referring to all the use of sed -i spread wantonly on the internet, sans appropriate caveats.

Background: most implementations of the -i flag allow sed to edit files in place; by default, it edits standard in and writes to standard out, so you end up having to do <file sed ... >file.new && mv file.new file. Trying to redirect back into the file (<file sed ... >file) fails because the file is truncated before sed gets to read it!

The -i flag is not specified by POSIX, which makes it non-portable. This isn’t such a big deal, except that different seds require different arguments! GNU sed edits in place, no argument provided. BSD sed, such as the one on macOS, requires a suffix:

So now, every sed -i solution to a problem needs to at least

I don’t know about you, but I wonder if the solution to editing files in place is a lot simpler… :thinking:

sed for streams, ed for files

sed stands for “stream editor.” It was designed based on the text-editor ed’s commands! ed is built to edit files.

I cannot emphasize this enough. Trying to use sed -i in any kind of portable anything is likely to break; worse was the thought that sed should ever be used for editing files! Use a text-editor: if you thought “I’m automating, I don’t want to run a text-editor to automate,” well, remember:

Addendum: scripting ed

ed takes commands on standard in. It can be made silent with -s, and the argument is a file to edit. I ignore -p for now because it isn’t useful for scripting/automation. So, to script ed, generate a list of commands and pipe it in!

printf '%s\n' g/abc/d w q | ed -s file

That’s the most portable, but bash users can probably do

ed -s file <<EOF

There is quite a bit of flexibility here; the left-hand side could actually be another program that generates ed commands on standard out! It turns out, this is what diff does: in its default output mode, we have

These lines resemble ed subcommands to convert file1 into file2. The line numbers before the action letters shall pertain to file1; those after shall pertain to file2. Thus, by exchanging a for d and reading the line in reverse order, one can also determine how to convert file2 into file1. As in ed, identical pairs (where num1=num2) are abbreviated as a single number.

And when we ask for the -e output mode:

With the -e option, a script shall be produced that shall, when provided as input to ed, along with an appended w (write) command, convert file1 into file2.

So now, you can probably write a patch command :smile: you parse the output from diff, with maybe slight modification, and then pass it as input to ed!


Categories: Blog

Load Comments
Previous Next
Back to posts