Linux cgroups v2 Internals for SREs

The operator view of cgroups v2

In modern distributions and Kubernetes nodes, cgroups v2 provides unified control of resource accounting and limits. The practical advantage for SRE work is consistent semantics across controllers, which makes node-level symptoms easier to map back to workload behavior.

Memory pressure and throttling

Key files in a memory cgroup:

• memory.current
• memory.max
• memory.high
• memory.events

memory.high enables proportional reclaim pressure before OOM, which can be safer for multi-tenant nodes. It also creates a useful early signal: a workload can be degraded by reclaim pressure before it is killed.

cat /sys/fs/cgroup/my-service/memory.current
cat /sys/fs/cgroup/my-service/memory.events

CPU control with realistic fairness

Use cpu.max for strict caps and cpu.weight for relative fairness. In shared nodes, cpu.weight tuning often improves tail latency more than hard caps because it preserves burst capacity while still expressing priority.

IO behavior under contention

With io.max and io.weight, you can prevent noisy neighbors from saturating block devices. This is critical for stateful workloads where latency spikes create cascading retries.

What to export to observability

For each service cgroup, export:

• Throttled periods and throttled time
• Memory reclaim events and OOM kills
• IO bytes and latency by operation type

SRE teams should correlate these with Kubernetes events, Prometheus node metrics, service dashboards, traces, and logs. Hidden saturation usually becomes obvious only when kernel-level counters and application symptoms are read together.

Incident questions to answer

• Is the workload throttled, reclaiming memory, or waiting on IO?
• Did the signal begin before or after a deployment, node drain, or traffic change?
• Is the fix a workload limit change, node pool change, storage change, or application retry change?
• What dashboard or alert would have made the failure mode visible sooner?