Rather than one enormous protocol, networking is split into layers, each solving one problem and trusting the layer below.
your app (HTTP: "GET /users")
│
↓ needs a reliable ordered stream
TCP (ports, retransmission, ordering)
│
↓ needs to find the machine
IP (addresses, routing between networks)
│
↓ needs to reach the next hop
Ethernet / WiFi (MAC addresses, one physical link)
The value of the split: your app never thinks about cables, and IP never thinks about retransmitting lost data. When you debug, you work down this stack — and each layer you rule out eliminates a whole class of cause.
Worth knowing because it explains what "connection refused" means:
client server
│ ── SYN ────────────────────→ │ "can we talk?"
│ ←──────────── SYN-ACK ────── │ "yes, can you hear me?"
│ ── ACK ────────────────────→ │ "yes"
connected — data can now flow
Three messages, because two cannot prove both directions work.
Work through these in order — each one rules out a layer.
# 1. Does the name resolve? (rules out DNS)
getent hosts api.example.com
# 2. Does the machine respond at all? (rules out routing)
# Note: many cloud hosts block ICMP, so a failed ping is not proof of a problem.
ping -c 3 10.0.1.5
# 3. Is the PORT open? This is the important one.
# -z scan only, -v verbose, -w timeout in seconds
nc -zv 10.0.1.5 5432
# 4. Does the application actually answer?
curl -v https://api.example.com/health
Reading the result of step 3 is most of the skill:
Result
Meaning
Next step
succeeded / open
Something is listening and reachable
Problem is in the app or the protocol
Connection refused
Reached the machine, nothing listening on that port
ip addr # this machine's addresses (replaces ifconfig)
ip route # the routing table. "default via ..." is your gateway
traceroute 8.8.8.8 # each hop along the way
mtr 8.8.8.8 # traceroute + live loss stats. Better for intermittent issues
If it stops after "Trying", it is a firewall. If it stops at TLS, it is a certificate. If you get a 5xx, the network is fine and the application is not.
internet
│
↓
① CLOUD firewall ← AWS security group / Azure NSG / OCI security list
│ configured in the web console, NOT over SSH
↓
② HOST firewall ← ufw or iptables, on the machine itself
│
↓
③ the service ← and it must be bound to 0.0.0.0, not 127.0.0.1
All three must allow the traffic. Opening only the host firewall and wondering why the port is still closed is the single most common cloud networking mistake.
sudo ufw status # if ufw is installed
sudo iptables -L INPUT -n --line-numbers # if it is not
"The service is unreachable." Walk me through it. · «السيرفيس مش بيرد»
This is the most common troubleshooting question in DevOps interviews, and the answer is a method, not a tool list. Narrow it one layer at a time, out loud:
# 1. Does the name resolve, and to what? (rules out DNS)
getent hosts api.example.com
# 2. Is the PORT open? (rules out firewall vs service)
nc -zv 10.0.1.5 5432
# 3. On the server: is anything listening, and on which interface?
sudo ss -lntp | grep 5432
# 4. Does the application answer? (rules out app vs network)
curl -v https://api.example.com/health
Then read step 2's result, because it splits the problem in half:
Result
Eliminated
Remaining
refused
Firewall, routing, DNS — the machine replied
Service down, wrong port, or bound to 127.0.0.1
timeout
Nothing — you got silence
Firewall: cloud security group, or host ufw
succeeded
The whole network stack
The application or the protocol above it
A large file transfer hangs at exactly the same point every time, but small requests work fine. What is it?
Almost certainly MTU / path MTU discovery, and this question exists to find people who have debugged real networks.
Small packets fit anywhere. Once a packet exceeds the smallest MTU on the path, the router must either fragment it or reply ICMP fragmentation needed. If a firewall blocks that ICMP message, the sender never learns and just retransmits the too-large packet forever — a PMTUD black hole.
client (MTU 1500) ──→ router (MTU 1400) ──→ server
│ │
│ small packet │ fits, fine
│ 1500-byte packet │ too big
│ │
│ ←── ICMP "frag needed, use 1400"
│ ✘ BLOCKED by a firewall
│
└── never learns. retransmits 1500 forever. hangs.
# Find the real path MTU: -M do = do not fragment, -s = payload size
ping -M do -s 1472 8.8.8.8 # 1472 + 28 header = 1500
ping -M do -s 1372 8.8.8.8 # try lower until it succeeds
Where it bites in practice: VPNs and tunnels (WireGuard, IPsec) and overlay networks — including several Kubernetes CNIs, which encapsulate packets and so reduce the usable MTU. The signature is always the same: handshakes and small responses fine, large payloads hang.
Why does a TCP connection need three messages and not two?
Because two messages can only prove one direction works.
client server
│ ── SYN ────────────────────→ │ "can you hear me?"
│ ←──────────── SYN-ACK ────── │ "yes — can you hear me?"
│ ── ACK ────────────────────→ │ "yes"
both directions now proven
After SYN → SYN-ACK, the client knows both directions work — it sent and it received. But the server has only proved that its own send worked; it has no evidence the client received anything. The third message supplies that. Each side also uses the exchange to communicate its initial sequence number, which is what makes ordering and retransmission possible at all.