Almost nobody is rejected for not knowing a command. They are rejected for not being able to say *why*, and for having no method when the question has no clean answer.
A worked set of the questions senior DevOps, SRE and Platform interviews actually ask — with the answer, and with what the interviewer is really listening for underneath it.
Every answer here is collapsed on purpose. Read the question, answer it out loud, then open it. A question you have read the answer to feels known; a question you have answered out loud is known. Those are very different things in a room with a stranger.
Most preparation fails in the same way: you memorise definitions, the interviewer asks "and why would you do that?", and there is nothing there.
Senior interviews are not knowledge tests. They are judgement tests wearing a knowledge costume. The question is a pretext; the interviewer wants to hear how you decide.
What juniors prepare
What seniors are asked
"What is a Service?"
"Why is this Service returning 503 for 1% of requests?"
"What does terraform apply do?"
"Someone changed it in the console. Now what?"
"What is a container?"
"Why is this image 1.2 GB and how do you know?"
Definitions
Trade-offs, failure modes, and what you would measure
The method that works in every troubleshooting round#
When you are handed a broken system, do not start naming tools. Narrow the blast radius out loud, in this order:
1. WHAT CHANGED? deploys · config · certs · DNS · traffic
│ (70% of incidents end here)
↓
2. HOW WIDE IS IT? one user / one pod / one AZ / everything
│ narrow = client or instance
│ wide = dependency or config
↓
3. WHICH LAYER? DNS → TCP → TLS → HTTP → app → database
│ walk it in order, do not jump
↓
4. PROVE IT one command whose result splits the
│ possibilities in half
↓
5. STOP THE BLEEDING roll back first, understand second
(say this part out loud — seniors do)
Four moves, in order. Most candidates skip the first and never recover.
Ask about scale and constraints. Requests per second? How many services? What is the team size? A design for 5 services and one for 500 are different designs, and picking without asking is the actual mistake.
State the trade-off you are optimising for. "I will optimise for rollback speed over deploy speed, because…"
Draw the boxes, then the arrows, then label the arrows.
Say what your design is bad at. Volunteering this is the strongest single signal of seniority in the whole loop.
A server is at 100% CPU. Walk me through it. · السيرفر معلّق على ١٠٠٪
Do not open top and stop there. top tells you who; it does not tell you why.
The order that actually narrows it
# 1. Is it CPU-bound at all? Check load vs core count first.
uptime; nproc
# load 8.0 on 4 cores = genuinely saturated
# load 0.5 with "100% CPU" = you are misreading a single-core spike
# 2. WHERE is the time going? This is the question top answers badly.
vmstat 1 5
# us high → application code
# sy high → kernel: syscalls, network, often a container escaping cgroups
# wa high → NOT CPU. You are blocked on disk. Different problem entirely.
# st high → steal. The hypervisor is throttling you. Not your fault.
# 3. Now find the process.
top -o %CPU # or: ps aux --sort=-%cpu | head
# 4. And what is it doing?
strace -c -p <pid> # syscall counts — cheap and often decisive
df says the disk is full. du says there is plenty of space. Who is lying?
Neither. A deleted file is still held open by a process.
du walks the directory tree and adds up files it can see. df asks the filesystem how many blocks are allocated. When a process has a file open and someone rms it, the directory entry is gone — so du cannot see it — but the inode and its blocks are not freed until the last file descriptor closes.
Classic cause: log rotation that deletes the log instead of truncating it, while the application keeps writing to the same open descriptor.
# Find the deleted-but-open files and who is holding them
lsof +L1
lsof -nP | grep '(deleted)'
# Fix: signal the app to reopen its logs, or restart it.
# Truncating via the descriptor also works and needs no restart:
: > /proc/<pid>/fd/<fd>
Your app can reach a service by IP but not by name. Where do you look?
Walk the resolution path in order; do not guess.
# 1. What does the app think the resolver is?
cat /etc/resolv.conf # in a pod: is it the cluster DNS IP?
# 2. Does the resolver answer at all?
dig @<that-ip> myservice.default.svc.cluster.local
# 3. Does it answer for a public name? (separates "DNS is down"
# from "this record is missing")
dig @<that-ip> example.com
# 4. Is something short-circuiting it before DNS is consulted?
cat /etc/hosts
cat /etc/nsswitch.conf # order of hosts: files dns
Common causes, in the order they actually occur: search domain and ndots making a short name expand wrongly; the DNS pods themselves being unhealthy or rate-limited; a NetworkPolicy that forgot to allow egress to port 53; and /etc/hosts overriding the name on one node only.
A Pod is Running but the Service returns 503. Where is the fault?
Running is not Ready, and a Service only sends traffic to Ready.
# The single most useful command here — is the endpoint list empty?
kubectl get endpointslices -l kubernetes.io/service-name=my-svc
# Empty endpoints = the Service is matching nothing that is Ready.
Then work through the four things that produce an empty endpoint list:
Cause
How you confirm it
Selector does not match the Pod labels
kubectl get pods --show-labels vs the Service selector
Readiness probe failing
kubectl describe pod → probe events
targetPort names a port the container does not expose
Compare Service targetPort to the container's containerPort
Pod is Terminating and already de-registered
kubectl get pods -w during a deploy
What actually happens between kubectl apply and a running container?
The answer they want is the control loop, not a list of nouns.
kubectl apply
│ HTTP POST, authenticated
↓
┌──────────────────┐
│ API server │ authn → authz (RBAC) → admission → validate
│ │ writes the desired state to etcd. THAT IS ALL.
└──────────────────┘ nothing has been scheduled yet
│
│ controllers are WATCHING etcd, not being called
↓
┌──────────────────┐
│ Deployment ctrl │ sees a Deployment with no ReplicaSet → creates one
└──────────────────┘
↓
┌──────────────────┐
│ ReplicaSet ctrl │ sees 0/3 pods → creates 3 Pod objects
└──────────────────┘ (still just rows in etcd — nodeName is empty)
↓
┌──────────────────┐
│ Scheduler │ sees pods with no nodeName
│ │ filters feasible nodes → scores them → binds
└──────────────────┘ writes nodeName back. Still nothing running.
↓
┌──────────────────┐
│ kubelet on node │ sees a pod bound to ITSELF
│ │ calls the CRI runtime → pulls image → starts it
└──────────────────┘ reports status back up
Why did the OOM killer take my container when the node had free memory?
Because the limit that applied was your limit, not the node's.
A resources.limits.memory becomes a cgroup memory ceiling for that container. Exceed it and the kernel OOM-kills the process inside the cgroup regardless of how much RAM the node still has. kubectl describe pod shows OOMKilled with exit code 137.
The follow-up is the real question — the difference between requests and limits:
requests
limits
Used by
the scheduler, to place the pod
the kernel, at runtime
Too low
pod is placed on a node that cannot really hold it
Someone changed infrastructure in the cloud console. Terraform state is now wrong. What do you do?
Answer in two halves — the fix, then the reason it happened. Only giving the first half reads as junior.
The fix:
terraform plan # see the drift; do NOT apply blindly
terraform refresh # or: terraform apply -refresh-only
# reconciles state with reality, changes nothing real
terraform import ... # if they created something Terraform doesn't know about
Decide deliberately: does the console change stay (import it and write the code to match) or go (let apply revert it)? Reverting someone's emergency fix without asking causes a second incident.
The reason: console write access existed at all. The durable fix is process, not commands — remove human write permissions in production, make the pipeline the only identity that can apply, and add drift detection on a schedule so you find this in an hour rather than in six weeks.
Deploys are safe but take 40 minutes. The team wants them under 5. Where do you start?
Measure the stages before you touch any of them. Then attack in this order, because the order reflects where the time usually is:
Suspect
Typical fix
Tests run serially
Shard across parallel runners; split unit from integration
Dependencies re-downloaded each run
Cache the dependency directory, keyed on the lockfile hash
Docker layers rebuilt each run
Registry-backed layer cache; order the Dockerfile by change frequency
The whole monorepo rebuilds
Build only what changed, from the dependency graph
Sequential deploy to N environments
Promote the same artefact; never rebuild per environment
How do you achieve 100% uptime? · إزاي توصل ٩٩.٩٩٩٩٩٪؟
You do not, and agreeing to try is the failed answer.
100% is not a target, it is a category error. Your cloud provider will not sell it to you, your upstream DNS does not have it, and the marginal cost of each nine rises roughly tenfold while the marginal user benefit falls.
The correct move is to reframe:
Ask what the users actually notice. Availability of what, measured where — at the load balancer, or from the user's browser?
Set an SLO below 100% and derive an error budget from it. 99.9% monthly is ~43 minutes of allowed downtime.
Point out that the budget is a feature: with zero allowed downtime you can never deploy, patch, or migrate anything.
Should we move everything to Kubernetes? · نروّح كل حاجة على كوبرنيتيس؟
The expected answer is not yes or no. It is "what problem are we solving?"
Kubernetes buys you bin-packing across many services, a uniform deploy interface, and self-healing. It costs you a control plane to operate, a networking model to learn, and a permanent platform-shaped headcount.
Kubernetes earns its cost when
Reach for something simpler when
Many services, many teams, one platform team
Three services and no platform team
Bin-packing genuinely saves money
A managed runtime already fits
You need uniform deploys across teams
One team ships one monolith
Saying "it depends" and stopping is also a fail. Name the specific condition that would decide it — team size, service count, whether anyone is on call for the cluster.
Tell me about an outage you caused. · حكيلي عن مشكلة إنت سببتها
There is one wrong answer: "I can't think of one." It reads as either inexperience or dishonesty, and it is the single most common way strong candidates lose the behavioural round.
Structure it in four beats and keep it under two minutes:
What broke, and the blast radius — in user terms, not component terms. "Checkout failed for about 8% of users for 20 minutes."
What you did to stop it — and how long detection took. Be specific.
The real root cause — a missing guardrail, not a person. "The migration had no dry-run step" beats "I ran the wrong command."
What you changed so the class of failure cannot recur — and say whether it has recurred. That last part is what proves the story is real.