Shell scripts that outlive you

Every codebase has a scripts/ directory. Most of the files in it were written in ten minutes, work correctly on exactly one machine, and fail in ways that produce no error and no output.

Shell is a fine language for gluing programs together. It is a terrible language for doing it safely unless you tell it to be, and telling it to be takes six lines.

the preamble#

#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'

What each one prevents:

set -e — exit on any command that fails. Without it, a script continues merrily after cd /nonexistent and then runs the rest of its commands in the wrong directory. This is how a cleanup script deletes the wrong thing.

set -u — error on an undefined variable. Without it, rm -rf "$BUILD_DIR/" with an unset BUILD_DIR expands to rm -rf /. This has happened to real people, at real companies, more than once.

set -o pipefail — a pipeline fails if any stage fails, not just the last one. Without it, curl bad-url | jq . succeeds, because jq was happy with the empty input.

IFS=$'\n\t' — stop splitting on spaces. This is what makes filenames with spaces stop being a source of bugs.

Four lines. They convert an entire category of silent wrong behaviour into loud failure.

the next four things#

Quote every expansion. "$var", not $var. Always, including inside [[ ]] where it usually does not matter, because "usually" is not a rule anyone remembers correctly.

Use "${var:?message}" for required inputs.

: "${DATABASE_URL:?DATABASE_URL is required}"

One line, fails immediately with a useful message rather than three steps later with a confusing one.

Make it idempotent. A script that is safe to run twice is a script that is safe to run at all. mkdir -p, rm -f, check-before-create. The second run is the one that happens during an incident when nobody is sure whether the first one worked.

Clean up with trap.

tmp="$(mktemp -d)"
trap 'rm -rf "$tmp"' EXIT

Now the temporary directory is removed whether the script succeeds, fails, or is interrupted.

the usability part#

A script that other people run needs the same courtesy as any other interface:

usage() {
  cat <<'EOF'
Usage: deploy.sh ENVIRONMENT [--dry-run]

  ENVIRONMENT   staging | production
  --dry-run     print what would happen, change nothing

Requires: awscli >= 2, jq. Reads DEPLOY_ROLE from the environment.
EOF
}
[[ $# -eq 0 || "${1:-}" == "-h" ]] && { usage; exit 0; }

Add a dry-run mode to anything destructive. It costs one conditional and it is the difference between a script people trust and a script people read three times before running.

Echo what you are about to do. set -x is the crude version and it is better than silence. A script that prints "deleting 4 objects from s3://bucket/prefix/" before doing it lets a human catch the mistake.

when to stop using shell#

Shell is right for: calling other programs in sequence, moving files, gluing a pipeline together. Under about a hundred lines.

Switch to a real language when you need:

• Data structures. Bash arrays are a trap and associative arrays are worse.
• Any arithmetic beyond counting.
• Error handling with recovery, rather than exit-on-failure.
• Parsing anything structured. If you are pulling JSON apart with sed, stop.
• Tests. You can test shell, and almost nobody does, which tells you something.

Python or Go for anything past that line. The rewrite is an hour and it pays back the first time somebody has to change it.

the check that costs nothing#

shellcheck scripts/*.sh

Run it in CI. It catches unquoted expansions, useless cat, subshell variable scoping, and roughly a dozen other things that produce silent wrong behaviour.

Every shell script I have ever run it against had at least one real finding. It takes five minutes to add and it is the highest-value linting available for the least-linted language in most repositories.