systemd units and reading logs

Unit types and dependencies, a service unit you can trust, restart policies, and getting real answers out of journalctl.

Units and dependencies

Unit suffix	Represents	Typical command
`.service`	A process or a group of them	`systemctl restart app`
`.socket`	A listening socket activated on demand	`systemctl status app.socket`
`.timer`	A schedule that starts a unit	`systemctl list-timers`
`.target`	A group used for ordering	`systemctl isolate multi-user.target`
`.mount` / `.path`	A filesystem or a watched path	`findmnt`, `systemd.path`

systemctl list-units --type=service --state=running
systemctl list-unit-files 'app*'
systemctl cat app.service            # every file that contributes to the unit
systemctl show app -p ExecStart -p Restart -p User
systemctl status app --no-pager

systemctl daemon-reload              # after ANY unit file change
journalctl -u app -n 100 --no-pager

Requires= is a hard dependency; Wants= is best-effort. Neither controls order — that is After= and Before=.
Drop-in overrides live in /etc/systemd/system/app.service.d/override.conf; use systemctl edit app and the original unit file stays untouched.
Units in /etc/systemd/system win over /usr/lib/systemd/system, which package upgrades overwrite.

A service unit worth deploying

[Unit]
Description=Inventory API
Documentation=https://example.com/docs
After=network-online.target postgresql.service
Wants=network-online.target
Requires=postgresql.service

[Service]
Type=simple
User=app
Group=app
WorkingDirectory=/srv/app
EnvironmentFile=-/etc/app/app.env
ExecStartPre=/srv/app/bin/migrate
ExecStart=/srv/app/bin/server --port 3000
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=3
TimeoutStopSec=20
KillSignal=SIGTERM

# baseline hardening
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/app
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
LimitNOFILE=65535

[Install]
WantedBy=multi-user.target

Restart value	Behaviour
`no`	Never restart; the default
`on-failure`	Restart on a non-zero exit, a signal or a timeout — the usual choice
`always`	Restart even after a clean exit; use for supervisors
`on-abnormal`	Only on signals, timeouts and watchdog trips
`unless-stopped`	Like `always`, but not after an explicit stop

ExecStartPre runs the migration before the server starts; a failing pre-command aborts the whole start, which is what you want.
EnvironmentFile keeps secrets out of the unit file. The leading - makes a missing file non-fatal.
Type=simple assumes the process stays in the foreground. If your program daemonises, use Type=forking or, better, disable daemonising.
After editing, run systemctl daemon-reload and then systemctl restart app — a reload alone does not restart the process.

journalctl without guessing

journalctl -u app -f                       # follow one unit
journalctl -u app --since "1 hour ago" --no-pager
journalctl -u app --since "2026-09-18 08:00" --until "2026-09-18 09:00"
journalctl -b -p err                       # errors since the last boot
journalctl -b -1 -p warning                # the previous boot
journalctl _PID=1234                       # everything one process logged
journalctl -o json-pretty -n 1             # all structured fields
journalctl --disk-usage
sudo journalctl --vacuum-time=14d
sudo journalctl --vacuum-size=500M

# /etc/systemd/journald.conf
Storage=persistent      # keep logs across reboots
SystemMaxUse=1G
MaxRetentionSec=1month
ForwardToSyslog=yes     # also feed rsyslog if you collect logs centrally

💡

By default the journal is kept in memory and in /run, so it disappears on reboot. Create /var/log/journal (or set Storage=persistent) before you need last week's logs. There is no restart of journald required for that directory to start being used, but existing boots are not recovered retroactively.

FAQ

The service is 'active (running)' but nothing works?

Check systemctl status for the PID and uptime, then the logs for the first failure after start. A common cause is a process that forks while the unit says Type=simple, so systemd tracks the wrong PID and reports success.

Unit fails with 'Failed to start ... Unit not found'?

You edited or created the file but did not run sudo systemctl daemon-reload, or the file has the wrong name or is in a directory systemd does not scan. Confirm with systemctl cat app.service.

Scheduling work: cron and systemd timers Networking: addresses, DNS and firewalls

Last refreshed 2026-09-18.

systemd units and reading logs

Units and dependencies

A service unit worth deploying

journalctl without guessing

FAQ

Related