Files, redirection and here-documents

File descriptors, redirecting stdout and stderr, appending and truncating, here-documents and here-strings, process substitution, and reading files line by line.

File descriptors and redirection

cmd > out.txt 2> err.txt      # stdout and stderr to separate files
cmd > both.txt 2>&1          # merge into one file (order matters)
cmd 2>&1 > both.txt          # WRONG: stderr still points at the terminal
cmd &> both.txt              # bash shorthand for the merge above
cmd >> out.txt 2>&1          # append instead of truncating

exec 3> "$logfile"           # open descriptor 3 for writing
printf 'started
' >&3
exec 3>&-                    # close it again

exec >> "$logfile" 2>&1      # from here on, the whole script logs to the file
grep -q ERROR app.log 2>/dev/null   # silence noise, keep the exit status
printf 'silent
' >/dev/null

• Order is evaluated left to right: 2>&1 copies wherever stdout points at that moment.
• > truncates before the command starts. If that command is also reading the file, it will read an empty file.
• Diagnostics belong on stderr so that result=$(tool) captures data and the user still sees warnings.

Here-documents, here-strings and process substitution

# quoted delimiter: nothing expands, the text is copied literally
cat <<'CONFIG' > /etc/app.ini
[server]
host = ${HOST}
CONFIG

# unquoted delimiter: variables and command substitution expand
cat <<EOF
generated $(date -Is) on $(hostname)
EOF

# indented document (the dash allows leading TABs only)
cat <<-EOF
	indented body
	EOF

# here-string: feed one value to stdin, no echo needed
grep -c 'ERROR' <<< "$log_text"
read -r first rest <<< "alpha beta gamma"

# process substitution: a command's output appears as a readable file
diff <(sort new.txt) <(sort old.txt)
while IFS= read -r line; do
    count=$((count + 1))
done < <(grep -h 'ERROR' ./*.log)

• A here-document is the readable way to embed multi-line text: config files, SQL, JSON bodies and usage messages.
• Only <<- strips indentation, and only tabs — if your editor inserts spaces the delimiter will not match.
• Use process substitution rather than a pipe when the loop must set variables that survive afterwards, because a pipe runs the loop in a subshell.

Reading files line by line

while IFS= read -r line; do
    printf '%s
' "$line"
done < "$file"

# tolerate a file that does not end with a newline
while IFS= read -r line || [[ -n $line ]]; do
    printf '%s
' "$line"
done < "$file"

# fields, not lines
while IFS=: read -r user _ uid _; do
    printf '%s %s
' "$user" "$uid"
done < /etc/passwd

# never do this with data that can contain spaces
for line in $(cat "$file"); do echo "$line"; done

• IFS= stops leading and trailing whitespace being eaten; -r stops backslashes being interpreted.
• Reading a file with < "$file" lets the loop run in the current shell, so counters and accumulators keep their values.
• mapfile -t arr < "$file" is the fastest way to get every line when you do not need a loop at all.

FAQ

Related

Text processing: grep, sed and awk in scripts Shells, startup files and running scripts

Last refreshed 2026-09-18.