Junk Drawer Logo Junk Drawer

For all those little papers scattered across your desk

The Fields Extracter You Always Wanted

D. Ben Knoble on 11 Sep 2019 in Blog

Ever awk‘d a bunch of fields out of a pipeline? Tired of typing single quotes, dollar signs, and prints, when you just want the fields? Me too.

See github for the most up-to-date version.

Update 17th September 2019: The cut(1) version only works if you use exactly tabs between your fields. awk(1) is less picky about delimiters, matching whitespace by default (and controllable with FS or -F). So, while awk(1) is “reinventing the wheel” in one sense, I think it’s really extending it. Most command output that I care about does not use tabs, so awk(1) is more robust than cut(1).

Update 15th September 2019: Most of this can be done with cut(1) via the -f flag. I might rewrite the script to take advantage of this, but preserve the same functionality.


# ls -l | fields 1 9
-rw-r--r-- 404.md
-rw-r--r-- Gemfile
-rw-r--r-- Gemfile.lock
-rw-r--r-- LICENSE
-rw-r--r-- README.md
-rw-r--r-- Session.vim
-rw-r--r-- _config.yml
drwxr-xr-x _data/
drwxr-xr-x _drafts/
drwxr-xr-x _includes/
drwxr-xr-x _layouts/
drwxr-xr-x _posts/
drwxr-xr-x _sass/
drwxr-xr-x _site/
drwxr-xr-x _writings/
drwxr-xr-x assets/
-rw-r--r-- index.md
drwxr-xr-x pages/
-rwxr-xr-x serve*
drwxr-xr-x sitemap-styles/
lrwxr-xr-x sitemap.xsl@
drwxr-xr-x subs/
-rwxr-xr-x tags-list*


The script (below) is a wrapper around awk(1): we just generated the code we would normally type by hand, and feed it to awk(1).

If you’re concerned about timing to generate code, replacing eval with echo and running fields {1..100} takes 0.012s on my machine. (If you’re pulling out 100 fields, I have questions for you…)

I’m considering replacing eval with exec, but we will have to see.

The script

#! /usr/bin/env bash

set -euo pipefail

log() {
  printf '%s\n' "$@"
} >&2

usage() {
  cat <<DOG
usage: $0 field...

die() {
  local ex="${1:-1}"
  exit "$ex"

usage_and_die() { usage && die; }

generate_awk_code() {
  local fields=("$@")
  case $# in
    0) return ;;
  printf "'{ print "
  # from the beginning to the second to last
  for field in "${fields[@]:0:${#fields[@]}-1}"; do
    printf '$%d, ' "$field"
  # last
  printf '$%d' "${fields[${#fields[@]}-1]}"
  printf " }'"

main() {
  eval awk "$(generate_awk_code "$@")"

main "$@"


Categories: Blog

Load Comments
Previous Next
Back to posts