Autoscaling and Scheduling
Autoscaling adds pods; the scheduler decides where they go; the cluster autoscaler adds machines when there is nowhere. Three separate systems, and most "autoscaling is broken" is really one of the other two.
What is it?#
Two questions that people merge and should not:
| Question | Answered by |
|---|---|
| How many pods should exist? | HPA, VPA, KEDA |
| Where does each pod go? | The scheduler — via taints, affinity, spread |
| Are there enough machines? | Cluster Autoscaler, or Karpenter |
load rises
│
↓
① HPA reads metrics, raises replicas 3 → 8
│
↓
② Scheduler tries to place 5 new pods
│
├── room exists ──→ placed, done
│
└── no room ──→ pods Pending
│
↓
③ Cluster Autoscaler sees Pending pods, adds a node
│
↓
scheduler places them
Scaling pods#
HPA — horizontal, the one you will use#
hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata: { name: web }
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: web
minReplicas: 3
maxReplicas: 20
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70 # 70% OF THE REQUEST, not of the node
behavior:
scaleDown:
stabilizationWindowSeconds: 300 # wait 5 min before shrinking
policies:
- type: Percent
value: 50
periodSeconds: 60
scaleUp:
stabilizationWindowSeconds: 0 # scale up immediatelyThe others, briefly#
| Scaler | Changes | Use when |
|---|---|---|
| HPA | Replica count | Web services, anything horizontally scalable |
| VPA | Requests and limits of a pod | Right-sizing; batch jobs |
| KEDA | Replica count, from external events | Queue depth, Kafka lag, cron |
Controlling where pods land#
Taints and tolerations — the node repels#
# The node repels everything that does not tolerate this
kubectl taint nodes gpu-1 workload=gpu:NoScheduletolerations:
- key: workload
operator: Equal
value: gpu
effect: NoSchedule| Effect | Meaning |
|---|---|
NoSchedule | New pods without a toleration are not placed here |
PreferNoSchedule | Soft — avoid if possible |
NoExecute | Also evicts running pods that do not tolerate it |
Affinity, anti-affinity, and spread#
Spread replicas across zones, and keep them off each other
spec:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels: { app: web }
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
topologyKey: kubernetes.io/hostname
labelSelector:
matchLabels: { app: web }| Mechanism | Use for |
|---|---|
nodeSelector | Simple: "must be on a node with this label" |
nodeAffinity | Same, with required / preferred and expressions |
podAntiAffinity | "Do not put my replicas together" |
topologySpreadConstraints | Even distribution — the modern, better option |
PodDisruptionBudgets#
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata: { name: web }
spec:
minAvailable: 2 # or maxUnavailable: 1
selector:
matchLabels: { app: web }A PDB constrains voluntary disruption — node drains, cluster autoscaler scale-down, rolling node upgrades. It does not protect against a node crashing, which is involuntary.
Adding machines#
| Tool | How it works |
|---|---|
| Cluster Autoscaler | Watches Pending pods; grows a node group; removes underused nodes |
| Karpenter | Provisions individual right-sized instances directly, no node groups |
Pending pods exist
│ and they would fit on a new node of this group
↓
Cluster Autoscaler raises the ASG desired count
↓
node boots, joins, scheduler places the pods ← 1-4 minutes, typically
What goes wrong#
| Symptom | Cause | Fix |
|---|---|---|
HPA shows <unknown> | No resources.requests, or no metrics-server | Set requests; install metrics-server |
| HPA scales constantly | Requests too low, or no stabilisation window | Right-size; add behavior |
Pods Pending after scale-up | No node capacity | Cluster Autoscaler / Karpenter |
| Autoscaler never adds nodes | Node group at maxSize, or the pod cannot fit any node type | Check its logs and group limits |
| Autoscaler never removes nodes | A PDB, a DaemonSet, or local storage blocks eviction | kubectl drain --dry-run to see |
| GPU pod on a normal node | Toleration without affinity | Add nodeSelector |
drain hangs forever | Unsatisfiable PDB | maxUnavailable: 1 |
| Replicas all on one node | No anti-affinity or spread | topologySpreadConstraints |
Interview corner · الأسئلة اللي بتتسأل#
Your HPA says <unknown> for CPU. Why?
Two causes, and they are easy to tell apart.
1. No resources.requests.cpu on the container. HPA utilisation is a percentage of the request, so with no request there is no denominator and the calculation is undefined.
2. No metrics pipeline. metrics-server is not installed, or it cannot scrape the kubelets.
kubectl describe hpa web # the Events name which one
kubectl top pods # fails entirely → metrics-server
kubectl get deploy metrics-server -n kube-system
kubectl get pod <pod> -o jsonpath='{.spec.containers[*].resources}{"\n"}'Explain taints, tolerations and affinity, and how they combine for a dedicated node pool.
They solve opposite halves of the same problem:
| Mechanism | Direction | Effect |
|---|---|---|
| Taint (on the node) | Repels | Nothing schedules here unless it tolerates |
| Toleration (on the pod) | Permits | This pod may go there — it is not attracted |
| nodeAffinity / nodeSelector (on the pod) | Attracts | This pod must or prefers to go there |
A dedicated GPU pool needs both:
kubectl taint nodes gpu-1 workload=gpu:NoSchedule # keep everyone else off
kubectl label nodes gpu-1 workload=gpu # give it an identitytolerations: # I am ALLOWED on the tainted node
- { key: workload, operator: Equal, value: gpu, effect: NoSchedule }
nodeSelector: # and I REQUIRE that node
workload: gpuWith only the taint, other workloads stay off — but your GPU pod may land on a cheap CPU node instead.
With only the affinity, your GPU pod goes to the right place — but nothing stops a batch job filling the expensive nodes.
Traffic spikes 5× in 30 seconds. Will autoscaling save you?
No, and understanding why is the point of the question.
metrics scrape up to 60s ← the spike is not even visible yet
HPA evaluation 15s
pods created, Pending —
autoscaler notices 10s
instance boots and joins 1-4 min
image pull + startup 10-60s
───────────────────────────────
≈ 2-5 minutes before new capacity serves
Your spike has come and gone — or taken the service down — long before that.
What actually works:
| Approach | Trade-off |
|---|---|
| Headroom — run at 40–50% utilisation | Costs money, works instantly |
| Over-provisioning pods — low-priority placeholders that get preempted | Nodes pre-warmed; small constant cost |
| A queue in front | Absorbs the spike; adds latency, needs the work to be async |
| Rate limiting / load shedding | Protects the service by rejecting some load deliberately |
| Karpenter over Cluster Autoscaler | Faster provisioning, still not seconds |
Autoscaling is for the daily curve, not for step changes. Naming that distinction — gradual versus instantaneous — is the answer.
Key takeaways#
- HPA adds pods; the Cluster Autoscaler adds machines.
Pendingafter a scale-up means the second one, not a broken HPA. - HPA percentages are of the
request. No request means no HPA, and a wrong request amplifies into wrong replica counts. - Do not run HPA and VPA on the same metric — they oscillate.
- Tolerations permit, affinity attracts. A dedicated pool needs both.
requiredanti-affinity caps replicas at your node count, permanently.- An unsatisfiable PDB blocks every drain, which silently stops node upgrades and scale-down. Use
maxUnavailable: 1. - Autoscaling takes minutes. For instant spikes, buy headroom.
Last updated 2026-09-13