HowToCodePage

Process handling, signals and parallelism

Background jobs and wait, job control, signals and traps, timeouts, xargs -P, GNU parallel, and controlling concurrency without flooding a machine.

Background jobs and wait

slow_task &
pid=$!                        # the child's process id
echo "started $pid"

wait "$pid"                   # block until it finishes
echo "exit status was $?"

# several at once, then collect all of them
for host in web1 web2 web3; do
    ssh "$host" backup.sh > "backup-$host.log" 2>&1 &
done
wait
echo "all backups finished"

jobs -l                       # what is still running in this shell
kill -TERM %1                 # signal job number 1
  • $! is the PID of the most recent background command; $? after wait is its status.
  • wait with no arguments waits for every background job in the current shell — but only for its own children, not for grandchildren started elsewhere.
  • wait -n returns as soon as any job finishes, which is how you recycle a worker slot.
  • A background job that writes to the terminal interleaves with everything else; redirect each job to its own file.

Signals, traps and timeouts

trap 'echo "interrupted" >&2; exit 130' INT
trap 'kill 0' TERM            # forward the signal to the whole process group
trap 'cleanup' EXIT           # runs on success, failure and signals
trap - INT                    # reset a handler to the default

timeout 30s curl -fsS "$url"          # SIGTERM after 30s
timeout -s KILL 5m rsync -a src/ dst/ # hard kill after 5 minutes
timeout --preserve-status 10s cmd     # report the command's status, not 124

curl -fsS --max-time 10 --connect-timeout 3 "$url"   # timeouts at the client too
SignalNumberCan be caught?Means
SIGHUP1YesTerminal closed; often used as "reload"
SIGINT2YesCtrl-C
SIGTERM15YesPlease stop — the polite default
SIGKILL9NoUnconditional kill; no cleanup runs
SIGCHLD17/20YesA child process changed state
  • kill 0 signals every process in the group, including the shell itself — the simplest way to stop a tree of workers.
  • SIGKILL cannot be trapped, so anything relying on a trap for correctness will be left dirty. Make the next run repair that state.
  • timeout exits 124 when it fires, which is a distinct status from the command failing on its own.

Bounded parallelism

# xargs: fixed number of workers, safe with any filename
find . -name '*.jpg' -print0 |
    xargs -0 -n 1 -P 4 ./convert-to-webp

# read a list from a file, four at a time
xargs -a hosts.txt -P 4 -I {} ssh {} uptime

# GNU parallel: per-job log files and a progress bar
parallel -j 8 --bar --results logs/ ./process ::: *.csv

# a hand-rolled worker pool with wait -n
max=4
for f in *.csv; do
    ./process "$f" &
    while (( $(jobs -rp | wc -l) >= max )); do wait -n; done
done
wait
ApproachBest forWatch out for
& + waitA handful of known jobsNo output interleaving control
xargs -PStreams of arguments, one command per itemArgument quoting; use -0 with -print0
GNU parallelComplex fan-out, retries, logsNot installed everywhere; quoting rules differ
wait -n poolCustom per-job logicYou write the bookkeeping yourself
⚠️
More parallelism is not more throughput. Beyond a small multiple of the available CPU or of the remote service's rate limit, every extra worker adds contention, timeouts and retries. Size the pool to the slowest shared resource and measure — and give each worker a timeout so one hung job cannot hold a slot forever.

FAQ

Why does my script leave orphan processes behind?
Background children outlive a shell that exits without waiting for them, and a killed script does not signal its children automatically. Use trap 'kill 0' EXIT INT TERM so the process group is torn down together.
How do I stop the whole run when the first job fails?
With xargs, exit 255 from the command. Otherwise collect PIDs and wait -n in a loop, checking each status, then kill the remaining jobs yourself.

A practical automation project Testing and linting shell scripts

Last refreshed 2026-09-18.