HowToCodePage

Archiving, syncing and backups

tar and compression formats, rsync options and semantics, incremental snapshots, and a backup you have actually restored.

tar and compression

tar -czf app-2026-09-18.tar.gz -C /srv app       # archive a directory
tar -tzf app-2026-09-18.tar.gz | head            # list without extracting
tar -xzf app-2026-09-18.tar.gz -C /tmp/restore   # extract somewhere else

tar --exclude='node_modules' --exclude='*.log' -czf backup.tar.gz -C /srv app
tar -cJf archive.tar.xz big-directory            # xz: smallest, slowest
tar -c --zstd -f archive.tar.zst dir             # zstd: fast and small
tar -cf - dir | zstd -T0 > dir.tar.zst           # compress in parallel

id                                        # note the numeric uid/gid of the owner
tar --numeric-owner -xvzf backup.tar.gz   # restore ownership as recorded
FormatSpeedRatioUse it for
.tarFastestNonePiping into another tool
.tar.gzFastGoodThe portable default
.tar.zstFast, parallelBetter than gzipLarge archives you compress often
.tar.xzSlowBestLong-term storage of rarely read data
.zipModerateGoodInterchange with Windows or macOS users
  • -C sets the directory before the paths are read, which keeps extracted trees free of absolute paths and leading ./ noise.
  • Never extract a third-party archive as root into /; unpack into a scratch directory and inspect it first, since archives can contain .. paths and absolute links.
  • Archives are not backups on their own: a single corrupt byte can cost the whole file, so keep several generations.

rsync and what the trailing slash means

rsync -av --progress src/ dest/          # contents of src into dest
rsync -av  src  dest/                    # the directory src itself, inside dest

rsync -av --delete --dry-run ./public/ deploy@web:/var/www/public/
rsync -av --delete ./public/ deploy@web:/var/www/public/

rsync -avz --partial --append-verify big.iso deploy@web:/srv/
rsync -av -e "ssh -p 2222" ./data/ deploy@web:/srv/data/
rsync -av --exclude-from=.rsyncignore ./ ./backup/

# incremental snapshots: unchanged files become hard links, storage stays flat
rsync -av --link-dest=../2026-09-17 ./ /backups/2026-09-18/
⚠️
--delete removes files on the destination that are missing from the source, and one missing slash changes what the source is. Combine the two wrongly and you delete the wrong tree. Always run the exact command with --dry-run first, and add --max-delete=100 as a safety valve on scheduled jobs.

A backup that survives being needed

  • Three copies, two media, one off-site — and at least one copy the compromised host cannot write to (append-only or object-lock storage).
  • Encrypt off-site copies, because copied data leaves the boundary of your disks.
  • Record the restore procedure next to the backup, not in someone's memory.
  • Back up the configuration that makes the data usable: the compose file, the database dump command, the fstab entries.
  • Alert on the job failing. A silent backup schedule is worse than none, because you believe it ran.
# a database needs a logical dump, not a copy of live files
docker exec db pg_dump -U app app | gzip > /backups/app-$(date +%F).sql.gz
docker exec db pg_dumpall -U postgres | zstd > /backups/cluster-$(date +%F).sql.zst

# volume data as a tar stream through a helper container
docker run --rm -v pgdata:/data -v /backups:/backup alpine \
  tar -czf /backup/pgdata-$(date +%F).tar.gz -C /data .

# restic to object storage, with retention
restic -r s3:s3.example.com/backups backup /srv /etc
restic -r s3:s3.example.com/backups forget --keep-daily 7 --keep-weekly 4 --prune
restic -r s3:s3.example.com/backups restore latest --target /tmp/restore

# and then actually check it
ls -l /tmp/restore/srv/app/config.yml
gzip -t /backups/app-2026-09-18.sql.gz && echo archive intact
ApproachRPOCostNote
rsync --link-destHoursLow diskHard-linked snapshots on any filesystem
restic / borgMinutesDeduplicatedEncrypted, incremental, verifiable
Filesystem snapshotSecondsStorage-sideSame disk; pair with a copy elsewhere
Database logical dumpHoursSmallPortable and consistent across versions
Streaming replicationNear zeroA second serverHigh availability, not a substitute for a backup

FAQ

Does the root's crontab backup run include the right files?
Use absolute paths, set PATH explicitly, and have the job write a status line to a log you monitor. A cron job inheriting a different environment is the most common reason a backup silently produces an empty archive.
Why is my incremental backup as large as the first one?
Check the semantics rather than the flags: --link-dest needs an existing previous snapshot at the exact relative path, and a changed mtime or permission bit makes rsync treat an otherwise identical file as different. -a with --delete and no --checksum trusts size and time.

Disks, filesystems and mounts Scheduling work: cron and systemd timers

Last refreshed 2026-09-18.