Gendi Notes

Kubernetes Deployments

A Deployment is how you get zero-downtime updates: it never edits a running pod, it creates new ones and removes old ones in a controlled order.

Level 2 guide kubernetesdeploymentsrollout

What is it?#

A Deployment keeps a set number of identical pods running, and manages the transition when you change them. It is the object you use for any stateless application — web servers, APIs, workers.

You do not create pods yourself. You create a Deployment; it does the rest.

Why it exists#

A bare pod has no safety net. Delete it and it is gone. A Deployment adds:

  • Self-healing — a pod dies, a replacement appears
  • Scaling — one command changes the replica count
  • Rolling updates — new version phased in without dropping traffic
  • Rollback — one command returns to the previous version

What it is made of#

Three layers, and knowing them makes rollbacks make sense:

Deployment "run 3 pods of nginx:1.27, update them safely" │ │ creates and owns one ReplicaSet per version ↓ ReplicaSet (nginx:1.27) "keep exactly 3 pods alive" │ ↓ Pod Pod Pod

When you change the image, the Deployment creates a new ReplicaSet and shrinks the old one:

during a rolling update: ReplicaSet v1 (old) ███░░░░ 3 → 2 → 1 → 0 ReplicaSet v2 (new) ░░░████ 0 → 1 → 2 → 3 └─ both exist at once, briefly

How to use it#

deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1      # how many may be down during the update
      maxSurge: 1            # how many extra may exist temporarily
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
        - name: web
          image: nginx:1.27-alpine
          ports:
            - containerPort: 80
          readinessProbe:              # "may I receive traffic?"
            httpGet: { path: /healthz, port: 80 }
            initialDelaySeconds: 5
            periodSeconds: 5
          livenessProbe:               # "should I be restarted?"
            httpGet: { path: /healthz, port: 80 }
            initialDelaySeconds: 15
            periodSeconds: 20
          resources:
            requests: { memory: "64Mi", cpu: "100m" }
            limits:   { memory: "128Mi", cpu: "500m" }

Commands#

kubectl apply -f deployment.yaml

kubectl rollout status deploy/web        # watch the update; exits when done
kubectl get rs -l app=web                # the ReplicaSets, old and new
kubectl rollout history deploy/web       # revision list

kubectl set image deploy/web web=nginx:1.28-alpine   # trigger an update
kubectl scale deploy/web --replicas=5

kubectl rollout undo deploy/web              # back one revision
kubectl rollout undo deploy/web --to-revision=3

kubectl rollout restart deploy/web       # recreate all pods without changing the spec

kubectl rollout restart is the way to make pods pick up a changed ConfigMap or Secret — it recreates them with no spec change.

Probes are the part that actually matters#

The two probes look similar and do very different things. Getting them wrong is the most common cause of self-inflicted outages.

readinessProbelivenessProbe
Question"Can I serve traffic?""Am I broken beyond recovery?"
On failureRemoved from the Service — no restartContainer is restarted
Use forWarm-up, temporary overloadDeadlock, unrecoverable state

Rolling update knobs#

replicas: 10, maxUnavailable: 1, maxSurge: 1 ┌─ at most 11 pods exist at any moment (10 + maxSurge) └─ at least 9 are serving (10 − maxUnavailable) maxUnavailable: 0 → never lose capacity, but needs room for extra pods maxSurge: 0 → never exceed the count, but capacity dips during update

Setting both to 0 is invalid — nothing could ever change.

What goes wrong#

SymptomCauseFix
rollout status hangs foreverNew pods never become Readykubectl describe pod on a new pod
Old pods never terminateA PodDisruptionBudget blocks evictionkubectl get pdb
Update did nothingThe spec did not actually changeUse rollout restart, or change the tag
Pods restart in a loop after deployLiveness probe failingCheck the probe path, port, and timing
A few 502s on every deployEndpoints not updated before the pod diesSee below

Scaling automatically#

# Add and remove pods based on CPU. Requires resource requests to be set
# and the metrics-server add-on to be installed.
kubectl autoscale deploy/web --min=3 --max=10 --cpu-percent=70
kubectl get hpa

The HPA needs resources.requests — the target percentage is measured against the request, so with no request there is nothing to compute against.

Interview corner · الأسئلة اللي بتتسأل#

What is the difference between a readiness and a liveness probe, and how would you break production with them?

Readiness answers "can I serve traffic right now?" — failing removes the pod from the Service, and nothing is restarted. Liveness answers "am I wedged?" — failing restarts the container.

How to break production, in one config:

livenessProbe:
  httpGet: { path: /health, port: 8080 }   # and /health checks the database

Now the database has a two-second blip. Every pod's /health fails. Every pod fails liveness. Kubernetes restarts your entire fleet at once — and the restarted pods all reconnect to the already-struggling database simultaneously.

You converted a recoverable dependency blip into a full outage plus a thundering herd.

RuleWhy
Liveness checks only the processA dependency check makes their outage your outage
Liveness gets generous timeoutsIt kills; strictness belongs in readiness
Slow start → startupProbeSuspends liveness during boot instead of loosening it forever
kubectl rollout status hangs forever. The new pods look fine to you. Debug it.

"Look fine" is the trap — rollout status waits for Ready, not for Running, and it will wait forever by design rather than declare a bad version healthy.

# 1. Which pod belongs to the NEW ReplicaSet?
kubectl get rs -l app=web           # find the one scaling up
kubectl get pods -l app=web

# 2. Why is that pod not Ready? The Events say so.
kubectl describe pod <new-pod>      # probe failures, image pull, scheduling

# 3. Is it the readiness probe specifically?
kubectl get pod <new-pod> -o jsonpath='{.status.containerStatuses[*].ready}{"\n"}'

The four usual causes:

CauseTell
Readiness probe failing (wrong path/port)describe shows Readiness probe failed
No room on any nodeNew pod is Pending, Events show FailedScheduling
Image cannot be pulledImagePullBackOff
PodDisruptionBudget blocks draining the old podskubectl get pdb — old pods stay Running
You changed a ConfigMap. The pods still use the old value. Why, and what do you do?

Because nothing restarts a pod when a ConfigMap changes. The Deployment's pod spec did not change, so there is no new ReplicaSet and no rollout.

What happens depends on how the ConfigMap is consumed:

Mounted asOn ConfigMap change
Environment variables (envFrom)Never updates — env is set once at process start
A volumeThe file does update, after a sync delay (~60s) — but only if your app re-reads it
subPath volume mountNever updates, even though it is a volume
kubectl rollout restart deploy/web    # recreate the pods, no spec change needed

Key takeaways#

  • Deployment → ReplicaSet → Pods. One ReplicaSet per version, which is what makes rollback instant.
  • Readiness removes from traffic; liveness restarts. Confusing them causes restart cascades.
  • Liveness must never check dependencies, and slow starters need a startupProbe.
  • rollout undo is one command. Old ReplicaSets exist for exactly this.
  • rollout restart to pick up new config without changing the spec.
  • Deploy-time 502s are the endpoint-propagation race — fix with a preStop sleep.

Last updated 2026-09-08