Kubernetes Architecture

What is it?#

A cluster is two halves: a control plane that decides, and worker nodes that run containers. They never talk to each other directly. Everything goes through one door — the API server — and everything else is a loop watching that door.

Why it is built this way#

The alternative — components calling each other — was tried by earlier orchestrators and it does not survive partial failure.

What it is made of#

Control plane#

Worker node#

The three plug-in interfaces#

Kubernetes deliberately implements none of these itself:

How it works: kubectl create deploy end to end#

This is the question. Follow one command all the way down.

kubectl create deployment web --image=nginx:1.27 --replicas=3

Step 1 — kubectl builds an HTTP request#

kubectl is a plain HTTP client. It reads ~/.kube/config for the API server address and your credentials, turns the flags into a Deployment object, and POSTs it.

# Prove it — this prints the request and response without the abstraction
kubectl create deployment web --image=nginx:1.27 --replicas=3 -v=8

POST https://10.0.0.1:6443/apis/apps/v1/namespaces/default/deployments
Authorization: Bearer <token>
Content-Type: application/json
{"kind":"Deployment","spec":{"replicas":3,...}}

Step 2 — the API server runs four gates, in order#

This order matters and is asked about constantly.

Step 3 — controllers notice, in a chain#

Nobody told the controllers anything. They hold an open watch on the API server and receive the new object on that stream.

Watch the chain yourself:

kubectl get deploy,rs,pods -l app=web
kubectl get events --sort-by=.lastTimestamp | tail -20

Step 4 — the scheduler binds each pod to a node#

The scheduler watches for pods where spec.nodeName is empty. For each one it runs two phases:

The scheduler's only output is a write. It never contacts the node.

# Why did this pod land where it did — or not land at all?
kubectl describe pod <pod> | tail -20        # Events name the failing predicate
kubectl get pod <pod> -o jsonpath='{.spec.nodeName}{"\n"}'

Step 5 — the kubelet makes it real#

Each kubelet watches for pods where spec.nodeName equals its own node name. When one appears, it does the actual work:

That status write is what makes the pod appear as Running in kubectl get pods. The status you read is always the kubelet's report, not a live query.

Step 6 — the pod becomes reachable#

Two more controllers react to the pod turning Ready:

Only now does traffic reach your container — and the gap between "pod Ready" and "every node has reprogrammed" is exactly the race that causes deploy-time 502s.

The whole path, in one picture#

How components actually communicate#

Watch, not poll#

A naive controller would poll: "list all pods, every second". With 5,000 pods that is unusable. Instead:

This pattern is called an informer, and it is why client-go controllers scale. Two details worth knowing:

• The queue holds keys, not events. Five rapid changes to one Deployment collapse into one key, reconciled once against final state. This is level-triggered behaviour falling out of the design.
• If the watch breaks or falls too far behind, the client gets 410 Gone — resourceVersion too old and does a full re-LIST. Correctness is preserved through any disconnection.

Leader election#

kube-controller-manager and kube-scheduler run multiple replicas for availability, but only one may act — two schedulers binding the same pod to different nodes would be a disaster.

# The lease shows who currently holds it
kubectl get lease -n kube-system kube-scheduler -o yaml

They contend for a Lease object in kube-system. The holder renews it every few seconds; if it dies, the lease expires and another replica takes over. It is active–passive, not load-balanced.

What goes wrong#

Interview corner · الأسئلة اللي بتتسأل#

# 1. See which webhooks exist and what they match
kubectl get validatingwebhookconfigurations
kubectl get mutatingwebhookconfigurations

# 2. Is its backing service actually up?
kubectl get pods -n <webhook-namespace>

# 3. If you are locked out, delete the configuration. The webhook stops
#    being consulted immediately; reinstall it once the service is healthy.
kubectl delete validatingwebhookconfiguration <name>

Key takeaways#

• Nothing calls anything. Every component watches the API server and reconciles a difference. That one sentence is the architecture.
• The API server is the only door, and it runs authn → authz → admission → validation → write, in that order.
• created means "recorded in etcd", not "running". Always wait for rollout status.
• The scheduler only writes nodeName. It never contacts a node; the kubelet notices the binding itself.
• Level-triggered, not edge-triggered — which is why any controller can be killed and restarted with nothing lost.
• The control plane is the change path, not the request path. Running pods survive its loss; your ability to fix anything does not.
• ContainerCreating usually means CNI or CSI, not Kubernetes itself.
• A failurePolicy: Fail webhook can deadlock the cluster. Exclude kube-system.

Last updated 2026-09-13