Reading Linux memory pressure: free, swap, the OOM killer and OOMScoreAdjust

What it is

free reports total, used, free, shared, buff/cache and available memory, plus a swap line. The free column alone undercounts headroom because Linux uses otherwise-idle memory for disk cache; the available column estimates how much memory can actually be given to a new process without swapping, accounting for reclaimable cache. When available memory and swap both run out, the kernel's out-of-memory (OOM) killer selects a process to kill to keep the system running, and it records its action in the kernel log, visible with dmesg or journalctl -k.

Two systemd unit settings influence this per service: OOMScoreAdjust=, a value from -1000 to 1000 written to the process's /proc/[pid]/oom_score_adj, biasing the kernel's OOM heuristic toward or away from killing it (more negative makes it less likely to be chosen); and MemoryMax=, a hard cap enforced by the cgroup controller that gets a process killed for exceeding its own budget before the systemwide OOM killer ever has to choose between unrelated services.

Why it matters

A service that dies with no application-level error and no crash log is a common symptom of an OOM kill: the kernel terminates it with SIGKILL, which the application cannot catch or log. Confusing this with an application bug wastes debugging time; confusing free memory with available memory leads to false alarms or missed ones.

How to apply

• Read free -h and act on available, not free; a small free value with a large available value is normal and not a problem.
• Check swap usage and whether swap is even configured (swapon --show); a system with no swap has less warning time before the OOM killer engages.
• After an unexplained service death, check journalctl -k | grep -i "out of memory" or dmesg | grep -i oom for a kill record naming the process and its score.
• Set OOMScoreAdjust=-500 (or similar) in a systemd unit's [Service] section for a process that must be the last one killed, and a positive value for a disposable batch job that should go first.
• Set MemoryMax= on services with known worst-case memory use so a leak in one service cannot starve unrelated services on the same host; verify with systemctl show <unit> -p MemoryMax -p OOMScoreAdjust.

Pitfalls

• Treating every process death as a code bug before checking the kernel log for an OOM record.
• Setting OOMScoreAdjust=-1000 (effectively OOM-immune) on a service that then hides a genuine leak by pushing the kill onto something else instead of surfacing the problem.
• Forgetting that MemoryMax= triggers a cgroup-level OOM kill of the service's own processes, which looks identical to a systemwide OOM kill in dmesg but has a different, narrower cause.