How to Harden systemd Services

To harden a systemd service is to decide, line by line in its unit file, what the service process is allowed to touch — and a stock service is allowed to touch almost everything. It usually runs as root, can write to the whole filesystem, can load kernel modules, and can call any syscall it likes, because those are the defaults: a unit file that sets nothing has opted into nothing. The sandbox directives systemd already ships — ProtectSystem, PrivateTmp, NoNewPrivileges and the rest — let you take those privileges away from the outside, without changing a line of the application itself. This article explains what each change actually closes off, how to read the exposure score that measures the result, and how to roll the changes out without breaking the service. You can audit your own unit file in the systemd Unit Auditor, which scores it, simulates its timers, and produces a hardened version — entirely in your browser.

A service unit is a sandbox boundary, not just a start script

Most people read a .service file as a start script: ExecStart says what to run, Restart says what to do when it dies, and the rest is boilerplate. That reading misses the point. The [Service] section is a list of permissions, and every directive you do not set is a permission you have implicitly granted. A unit with no sandboxing directives runs its process with the full reach of the user it runs as — usually root — over the filesystem, the network, devices, the kernel, and every other process on the machine. When such a service is compromised, the attacker does not need to escalate privileges: the service already has them.

The fix is to treat the unit file as the security boundary it is. systemd can confine a process to a read-only filesystem, a private /tmp no other process can see, a minimal /dev, no new privileges, no kernel modules, and a short list of syscalls — all declared in the unit, all enforced by the kernel, and none of it requiring a change to the application. That is what makes hardening a systemd service different from hardening most software: the sandbox lives outside the code, so you can add it to a daemon you did not write and cannot modify, measure the result, and roll it back by deleting a file. The auditor exists to make that boundary visible: it reads the unit you actually have and lists what it still allows.

Reading the exposure score: what 9.6 vs 2.0 means — and what it does not

systemd-analyze security scores a unit on a weighted exposure scale from 0.0 to 10.0, where a high number means loose and a low number means tightly locked down, and labels it with a band — OK, MEDIUM, EXPOSED, or UNSAFE in the short list, with PERFECT, SAFE and DANGEROUS at the ends in the full list. A stock service with no sandboxing directives typically lands around 9.6 UNSAFE; a carefully hardened one lands around 2.0 OK. The per-directive rows in the output are the useful part: each row names a setting, whether it is set, and how much it contributes to the exposure, so the highest-weighted red rows are your to-do list, in priority order.

What the number does not mean matters more than what it does. The man page is explicit: the analysis covers only the per-service security features systemd itself implements, and a high exposure level neither means there is no effective sandboxing in the service’s own code, nor that the service is actually vulnerable. A score of 9.6 says the unit file applies almost no sandboxing — it says nothing about the application’s authentication, input validation, or the secrets it holds. Some services legitimately need broad access: a container runtime genuinely needs to reach everything in order to run containers, and no directive can change that. So read the score as a measure of how much of the host a compromised process could reach, not as a verdict on the service, and do not chase 0.0 — chase the directives that close off reach your service does not need.

The ten directives that do most of the work

Ten directives do most of the real work, and each closes off one specific piece of the host rather than all of it at once. ProtectSystem makes the filesystem read-only — yes covers /usr and /boot, full adds /etc, strict covers everything except /dev, /proc and /sys — and ProtectHome hides /home, /root and /run/user entirely. PrivateTmp gives the service its own /tmp and /var/tmp that no other process can see, killing predictable-temp-file attacks and symlink races. NoNewPrivileges ensures the process and all its children can never gain new privileges through execve, which neutralises most setuid-based escalation in a single line. CapabilityBoundingSet limits the Linux capabilities the process can ever hold; set it empty and the service keeps none at all, which is the single biggest score drop for many units.

The other five close off the kernel and the process itself. PrivateDevices replaces /dev with a minimal stub, so a compromised service cannot poke at raw disks or other hardware nodes. ProtectKernelModules forbids loading kernel modules, and RestrictNamespaces stops the service creating new namespaces of the kinds you list — a container escape route, closed at the unit. LockPersonality prevents the process changing its execution domain, which blocks a small but real class of personality-based attacks. MemoryDenyWriteExecute forbids memory that is both writable and executable, which stops the classic pattern of writing shellcode into memory and then jumping to it. Together with ProtectSystem=strict, punch the exact holes the service genuinely needs with ReadWritePaths — the whitelist, not a loosening of the whole policy — and apply the syscall filter SystemCallFilter=@system-service last, because it is the most likely to break something subtle.

Timers vs cron: OnCalendar, Persistent, and why the next firings are worth simulating

A cron line and a systemd timer both run a job on a schedule, but they are not the same mechanism, and the differences are where the mistakes live. A timer is a .timer unit that activates a .service unit of the same name when it elapses, and its schedule is an OnCalendar expression — daily, Mon *-*-* 02:00, *-*-01 03:30:00, or lists, ranges and steps such as Mon..Fri *-*-* 09:00 — with weekday names, month-end rules and repetitions that a five-field cron line cannot express. The systemd.timer(5) man page defines the semantics; the practical consequences are that a timer’s firings are logged in the journal like any other unit, that a missed firing can be caught up, and that the schedule itself can be validated before it ever runs.

Two features make timers worth simulating before you deploy them. Persistent=yes means that if the machine was off — or the timer was inactive — when a firing was due, the service runs once at the next opportunity instead of the run silently never happening, which is the difference between a backup that is late and a backup that never ran. And OnCalendar expressions have edge cases a glance will not catch: a firing at 02:30 on a day when daylight saving time skips 02:00–03:00, a monthly job on the 31st in a 30-day month, a weekday rule that lands on a holiday. That is why a next-firing table is worth generating: run systemd-analyze calendar against the expression, or simulate the next 100 firings in the auditor, and read the table before the schedule goes live — a timer that fires twice, or never, is a bug you want to find on a screen, not in a missing backup three months later.

The safe workflow: drop-in overrides, a few directives at a time

Never hand-edit a packaged unit file in /usr/lib/systemd/system: the next package update overwrites it, and your hardening silently disappears. The supported route is a drop-in override — run systemctl edit myservice.service and your changes land in /etc/systemd/system/myservice.service.d/override.conf, where they take precedence over the shipped file, survive upgrades, and can be removed by deleting one file. The auditor can generate exactly that file for you: a [Service] section containing only the directives your unit is missing, so the original unit stays untouched.

Then apply a few directives at a time and re-measure after each step. Run systemctl daemon-reload, restart the service, confirm it is actually running with systemctl status, check the journal for denials, and re-score it — the score should fall at each step, and a step that breaks the service tells you exactly which directive did it. The failure mode is predictable and even helpful: the service starts, tries to write to a path you did not whitelist, and dies with a read-only-filesystem error, and journalctl -u myservice names the exact path, so you add it to ReadWritePaths and move on. That is why you harden incrementally rather than pasting a giant block and hoping — and why the sharpest tools, the syscall filter and the namespace restrictions, go last, after the service has proven it works under the filesystem and privilege restrictions.

The audit-first habit: score before and after, keep the unit in version control

Hardening without a baseline is guesswork with extra steps. Before you change anything, score the unit as it is — that number is your starting line, and every change after it is measured against it. Then, after each batch of directives, score it again: the score falling from 9.6 to 4.9, and then to 2.0, is the evidence the work did something, and a score that does not move after a change is a signal that the directive did not take effect the way you expected. Keep the before-and-after numbers with the change, the way you would keep a benchmark with a performance fix.

The second half of the habit is keeping the hardened unit where it cannot be lost: the drop-in override, and the timer if there is one, belong in version control next to the service’s deployment configuration, not only on the machine. A unit that lives only in /etc is one disk failure, one rebuild, or one well-meaning colleague away from reverting to stock — and a stock unit on a rebuilt machine scores 9.6 again, silently. Re-audit on a schedule, too: systemd gains new checks over time, so a unit that scored 2.0 two years ago can score worse today without a single line changing. The same habit transfers directly to the sibling config file — audit sshd_config the same way with the SSH Server Config Auditor and the companion article How to Harden Your SSH Server Config — because the discipline is the same whatever the file: measure first, change a little, measure again, and keep the result somewhere it survives the machine.

Sources

Frequently asked questions

Is a low exposure score proof my service is secure?

No. The score is a weighted heuristic over the sandbox directives systemd itself applies — it measures how much of the host a compromised process could reach, not whether the application is safe. It knows nothing about the application’s authentication, input validation, logic bugs, the secrets it holds, or its exposed network endpoints. Treat a good score as one layer of defence in depth: combine it with a dedicated non-root user, least-privilege network exposure, and normal application security practice. A service can score 1.5 and still be vulnerable, and a service can score 9.6 and still be fine — the number is a measure of attention, not a verdict.

Will hardening break my service?

It can, which is why you apply directives a few at a time and test after each step. The filesystem restrictions are the most common cause: ProtectSystem=strict makes the whole filesystem read-only, so a service that writes outside its whitelisted paths fails with a read-only error — check the journal, add the path to ReadWritePaths, and retry. The syscall filter is the sharpest tool and goes last: a filter that is too tight can let a service start fine and then fail hours later on a rare code path, so test under real load and keep SystemCallErrorNumber=EPERM during rollout so denials surface as ordinary errors instead of an instant kill. Nothing here is irreversible — a drop-in override is deleted, not unpicked.

Should I replace cron with systemd timers?

For jobs that belong to a service systemd already manages, usually yes: timers give you journal logging, dependencies on other units, Persistent catch-up for missed runs, and a schedule you can validate with systemd-analyze calendar before it goes live. Cron is still fine for simple per-user jobs and for systems without systemd. Whichever you choose, simulate the schedule first — OnCalendar expressions have DST, month-end and weekday edge cases that a five-field cron line does not, and a next-firing table is the cheapest way to catch a timer that fires twice, or never, before it matters.