Kubernetes resource requests and limits: scheduling, throttling and OOM kills

What it is

Each container in a Pod may declare resources.requests and resources.limits for CPU and memory. The Kubernetes documentation (cited) separates the two roles: the scheduler uses requests to choose a node with enough unreserved capacity, and the kubelet reserves at least the requested amount for the container; limits are enforced by the kubelet and container runtime and ultimately by the kernel through cgroups. The enforcement differs by resource: CPU limits are enforced by throttling, so a container never gets more CPU than its limit; memory limits are enforced reactively by out-of-memory kills, which the documentation says happen only when the kernel detects memory pressure, so a container may use more than its memory limit for a while and then be killed. If a limit is set without a request, Kubernetes copies the limit into the request.

The QoS page (cited) derives a class from these fields: Guaranteed when every container has equal requests and limits for both CPU and memory, Burstable when at least one container has a request or limit but the criteria for Guaranteed are not met, and BestEffort when nothing is set. Under node pressure, BestEffort Pods are evicted first, then Burstable, then Guaranteed, and only Pods that exceed their requests are eviction candidates.

Why it matters

Requests decide bin-packing and therefore cost; limits decide failure mode. A CPU limit set close to typical usage turns latency spikes into throttling that is invisible in application logs; a memory limit set below the real working set turns a slow leak into periodic restarts with exit code 137. Pods without requests land wherever there is room and are the first to be evicted.

How to apply

• Always set memory requests and limits; set them equal for services whose eviction would be an incident, which yields the Guaranteed class.
• Set CPU requests from observed usage; be deliberate about CPU limits, because throttling rather than eviction is the cost of exceeding them.
• Watch the container's restart count and last termination reason (OOMKilled) and CPU throttling metrics from the runtime, not only application latency.
• Apply namespace defaults with LimitRange so that forgotten fields do not silently produce BestEffort Pods, and cap totals with ResourceQuota.
• Size requests for the JVM, Node.js or Python process itself, including its runtime heap settings, not for the idle container.

Pitfalls

A memory limit is not a memory reservation for the node: the sum of limits across Pods may exceed the node's capacity. Changing requests or limits has historically meant recreating the Pod; the cited page describes in-place resizing, whose availability depends on the cluster version.

CPU limits: usually leave them off

A CPU request is turned into the cgroup's CPU weight, which already guarantees the container its share when the node is contended; a CPU limit adds only throttling. The CFS quota is granted per 100 ms period, so a container limited to one CPU whose four threads wake at once spends the period's quota in 25 ms and waits for the remaining 75 ms, every period, which shows up as latency while CPU graphs look calm. Check container_cpu_cfs_throttled_periods_total against container_cpu_cfs_periods_total before blaming the code. Keep CPU limits where needed: the static CPU manager policy (Guaranteed QoS with integer CPUs), a multi-tenant cluster where idle capacity must not be consumable, or a Pod that needs the Guaranteed QoS class, which requires CPU limits equal to requests. Set memory limits always.