Kubernetes Pod Evictions: Interpreting Node Pressure Before SLO Impact

Why eviction signals matter

Availability incidents in shared Kubernetes clusters often start as resource contention, not hard platform failure. Evictions are a protective action by kubelet, but if the signal is late or noisy, user-facing workloads still suffer.

Eviction signal model

Kubelet tracks pressure across three domains:

• memory.available
• nodefs.available and imagefs.available
• pid.available

When soft or hard thresholds are crossed, kubelet begins reclaiming resources and eventually evicts pods based on QoS and priority.

# kubelet config excerpt
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
evictionHard:
  memory.available: "500Mi"
  nodefs.available: "10%"
  imagefs.available: "15%"
evictionSoft:
  memory.available: "1Gi"
  nodefs.available: "12%"
evictionSoftGracePeriod:
  memory.available: "90s"

QoS class behavior under pressure

• BestEffort pods are evicted first.
• Burstable workloads are evaluated by overcommit and observed usage.
• Guaranteed pods are last, but still not immune during severe pressure.

The practical SRE strategy is to enforce requests/limits discipline, use priority intentionally, and eliminate accidental BestEffort workloads from critical namespaces.

Production controls that reduce incident frequency

1. Guardrails in admission policy

Use policy checks to reject deployments without memory requests and without a defined priority class in platform-critical namespaces.

2. Headroom per node pool

Reserve enough allocatable margin to absorb expected burst and one-node failure in each pool. Node pool headroom should be reviewed alongside autoscaler behavior, not separately.

3. Alert on signal trend, not just threshold crossing

Static threshold alerts fire too late. Alert on slope and sustained pressure windows.

avg_over_time(node_memory_MemAvailable_bytes[15m])
  / avg_over_time(node_memory_MemTotal_bytes[15m])
  < 0.08

Post-incident checklist

• Confirm if evictions were hard or soft threshold driven.
• Correlate with deployment events and autoscaler actions.
• Check if eviction victims map to missing requests/limits.
• Review node image garbage collection and ephemeral storage usage.
• Capture the runbook gap: missing alert, missing dashboard, unsafe workload default, or capacity assumption.

Closing

Pod evictions are an expected safety mechanism, but frequent evictions indicate an operating model problem. Reliable platforms treat eviction metrics as early-warning telemetry, not Kubernetes noise.