Node Scaling Bottlenecks

Use this workflow for unexplained node growth, repeated scale churn, pod evictions, pending pods, or stranded CPU and memory. Collect representative history and current state for:

• node allocatable capacity, requests, utilization, pod count, conditions, and extended resources;
• pending and evicted pods, scheduler evidence, node-pressure events, and autoscaler decisions when available;
• pool VM SKU, node count and bounds, maxPods, zones, OS disk size/type, kubelet disk type, upgrade surge, and reservations;
• ephemeral-storage requests and limits, node filesystem and image filesystem usage when exposed, and workloads using emptyDir or node-local storage;
• placement-control evidence from the bin-packing workflow, including eligible nodes, excluded capacity, scheduling failures, and autoscaler/NAP actions;
• per-node disk, network, storage-attachment, and other capability limits.

Missing history means the cause is unverified. Do not infer storage pressure from image size, scale-out from a pending pod alone, or savings from unused CPU.

Correlate Cause and Cost

Identify the first proven limiting dimension:

Confirm with time- and workload-aligned evidence that the bottleneck prevented pod placement or node removal and caused capacity to be added or retained. Otherwise report it as a likely contributor, not a proven cost cause. Do not call scale-out a false positive or added nodes unnecessary until a supported counterfactual shows the workloads could run with fewer nodes after the proposed correction. Low CPU and memory rule out only CPU or memory saturation; do not rule out VM shape because the SKU can determine local disk capacity and performance.

Compare Remedies

Evaluate the lowest-risk correction first:

1. Correct workload requests, image footprint, logs, caches, writable layers, emptyDir, or an obsolete placement restriction.
2. Adjust a supported node limit or capability, such as pod capacity, OS disk, kubelet storage placement, or VM shape. Verify OS, kubelet, image, and temporary filesystem roles independently; an Ephemeral OS disk is not automatically a separate kubelet or container-data disk.
3. Reconfigure or replace a pool while preserving required isolation, availability, zones, reservations, and autoscaling behavior.
4. Change workload architecture only when simpler corrections cannot satisfy the requirement.

Compare the avoidable VM, managed OS disk, reservation, and related costs with the correction’s storage or VM cost and migration overlap. A cheaper component is not an optimization when total cluster cost or reliability worsens.

Return to Bin-Packing

Return the limiting dimension, causal evidence, affected pools and workloads, scaling consequence, attributable cost, remedy candidates, and confidence. Keep unverified causation or cost unknown rather than zero. Do not execute changes without approval. Require explicit approval before pool creation or reconfiguration, cordon, drain, or scaling.