Why networks are layered

The wire fails in five ways, and everything else follows

sentwhat arrivesDelay123123all three arrive, but late0.1 ms inside a rack · 70 ms across a continent · 300 ms by satelliteDrop1231✗3a router’s queue was full, so it threw one away0.01% loss is healthy · 1 – 5% means congestionReorder123213two packets took different routeseach packet is routed on its ownDuplicate1231223a retry and its original both got throughcommon after timeoutsCorrupt1231?3noise flipped a bit on the way1 bit in a billion on fibre · 1 in a thousand on poor wifiThe wire promises only “best effort”.Every protocol in this lesson exists to hide one of these five from the layer above it.

A network link promises only best effort. Each protocol above it exists to hide one of these failures.

Why networks are layered · Going deeper · 1 of 2

Five failure modes of the unreliable wire A B the wire delay RTT 0.1 – 300 ms · transcontinental ≈70–150 ms propagation + queueing + processing drop loss 0.01% – 5% reorder 1 2 3 4 5 → 1 3 2 5 4 ECMP duplicate 1 2 3 → 1 2 2 3 3 retx echo corrupt 01101001 → 01111001 BER 10⁻⁹ – 10⁻³

The five failures, with real rates

Signals travel at about two-thirds the speed of light. Loss of 0.01% is healthy; above 1 to 5%, a reliable protocol turns into stop-and-go traffic. Routers crash and routes change too: a full buffer loses a packet in microseconds, a route change can stall traffic for tens of seconds.

Why networks are layered · Going deeper · 2 of 2

The networking stack — five layers and the failure modes they mask Application HTTP · DNS · SMTP · TLS — what the program speaks Transport TCP (reliable stream) · UDP (fast datagram) · QUIC Network IPv4 · IPv6 — packets routed across networks Link Ethernet · Wi-Fi — frames between neighbours Physical copper · fibre · radio — voltage, photons, EM waves The unreliable wire what every layer above masks delay drop reorder duplicate corrupt RTT 0.1 – 300 ms loss 0.01 – 5% queue rebalances retx echoes BER 10⁻⁹ – 10⁻³ + partition + congestion + adversaries each layer adds a header above and consumes the header below HTTP/3 swapped TCP for QUIC without touching IP, link, or physical

The stack and what each layer masks

Remove TCP and the network does not become reliable; the wire’s behaviour just stops being hidden from the application. That is the whole design.

Encapsulation

Each layer seals the one above inside its own envelope

Wifi or Ethernet framegets it to the next device on this linkIP packetgets it to the right machine, anywhere in the worldTCP segmentmakes the bytes complete and in orderHTTP requestGET /index.html: what you actually asked forEach layer wraps the one aboveand treats it as sealed contents.A router on the way opens onlythe outer two envelopes.One frame carries at most1,500 bytes, so a large pagetravels as many packets.Because the envelopes areseparate, each layer couldchange without the others:wifi replaced cables, and theweb kept working.Five layers, one job each: physical (signals) · link (next device) · network (any machine) · transport (reliable stream) · application (meaning).

Encapsulation: every layer adds a header for its own job and never looks inside. That is why each could evolve alone.

Encapsulation · Going deeper · 1 of 2

Encapsulation: an HTTP request wrapped in TCP, IP, and Ethernet Ethernet frame IP packet TCP segment HTTP request opaque payload to TCP, IP, Ethernet on the wire: ETH | IP | TCP | HTTP — each header read and stripped in reverse on receive

Headers, layer by layer

Each layer prepends a header and strips it on receive. Real systems cross the lines for speed: network cards do TCP segmentation and TLS in hardware, and firewalls read inside HTTP on purpose. Layering is the abstraction, not always the implementation.

Encapsulation · Going deeper · 2 of 2

ARP resolves a local IP to a MAC before the frame can leave A B 192.168.1.2 aa:aa:…:01 192.168.1.5 bb:bb:…:05 shared Ethernet segment (one broadcast domain) ARP req: who has 192.168.1.5? broadcast to ff:ff:ff:ff:ff:ff ARP reply: 192.168.1.5 is at bb:bb:…:05 Frame on the wire dst MAC: bb:bb:…:05 src MAC: aa:aa:…:01 payload: IP{src 1.2, dst 1.5} | TCP | data link layer addresses neighbours; network layer addresses endpoints. crossing a router rewrites the MAC pair; the IP pair survives end to end

Two kinds of address on one link

Ethernet uses 48-bit MAC addresses; IP uses 32 or 128 bits. ARP bridges them: the host broadcasts “who has 192.168.1.5?” and the owner replies with its MAC. Across a router the MAC pair changes at every hop while the IP pair survives end to end.

Routing

No router knows the route; each one only knows the next hop

Your laptopRouter 1Router 2Router 3Server93.184.216.34a packet addressed to 93.184.216.34Router 2 has never heard of the server. It only has this table:addresses that start withsend todoes it match?anythingRouter 9yes, 0 leading bits93.184.… (/16)Router 7yes, 16 leading bits93.184.216.… (/24)Router 3yes, 24 bits: the most specific winsthe rule:longestmatching prefixNo router knows the whole path. Each makes one local choice, and the route is whatever those choices add up to.IPv4 has 4.3 billion addresses and ran out in 2011, so a whole home or office shares one public address (NAT).IPv6 addresses are 128 bits. Between companies, routes are chosen by business policy first and distance second (BGP).

IP gives every machine a number. Routers pass a packet along by matching its address against a table of prefixes.

Routing · Going deeper · 1 of 4

Subnetting a /24 into four /26 blocks 192.168.1.0/24 256 addresses · 24 network bits · 8 host bits borrow 2 host bits to make 4 subnets 192.168.1.0/26 192.168.1.64/26 192.168.1.128/26 192.168.1.192/26 .0 – .63 .64 – .127 .128 – .191 .192 – .255 62 usable hosts 62 usable hosts 62 usable hosts 62 usable hosts prefix mask addresses usable hosts /24 255.255.255.0 256 254 /25 255.255.255.128 128 126 /26 255.255.255.192 64 62 /30 255.255.255.252 4 2

Prefixes and subnets

93.184.216.0/24 covers 256 addresses; the number after the slash counts the bits that name the network. Each bit borrowed from the host part doubles the subnets and halves their size. Two addresses per subnet are reserved.

Routing · Going deeper · 2 of 4

Routers forward by longest-prefix match R1 R2 R4 R5 R6 src dst 198.51.100.42 prefix next hop metric 10.0.0.0/8 R3 10 198.51.100.0/24 R4 5 0.0.0.0/0 R3 100 default route — last resort R2's routing table longest-prefix match 198.51.100.42 matches /24 (most specific) forward to R4

Longest-prefix match in action

Traceroute reveals the path with a trick: every router subtracts one from a packet’s time-to-live and reports back when it hits zero, so sending probes with TTL 1, 2, 3 names each hop in turn.

Routing · Going deeper · 3 of 4

BGP route propagation: AS-PATH grows by one with each hop AS 1 AS 2 AS 3 origin transit customer 203.0.113.0/24 UPDATE → AS 2 prefix: 203.0.113.0/24 AS-PATH: [1] UPDATE → AS 3 prefix: 203.0.113.0/24 AS-PATH: [2, 1] each AS prepends its number before re-announcing policy attributes (LOCAL_PREF) override length when business rules dictate

Between companies: BGP

Each operator is an autonomous system. BGP announcements carry the list of systems a route has crossed, which detects loops. Operators prefer customer routes over peers over paid upstreams, so routes follow money as much as physics. Anycast announces one address from many sites, so each user reaches the nearest.

Routing · Going deeper · 4 of 4

NAT rewrites source address and port on outbound flows Private LAN 10.0.0.0/24 A B C NAT rewrite src 203.0.113.7 Public internet sees one IP per LAN 203.0.113.7 internal (src:port) external (src:port) remote (dst:port) 10.0.0.5:50321 203.0.113.7:40001 93.184.216.34:443 10.0.0.6:50321 203.0.113.7:40002 93.184.216.34:443 10.0.0.7:33108 203.0.113.7:40003 1.1.1.1:53 NAT translation table (per flow)

Sharing one public address: NAT

A NAT rewrites the source address and port of outgoing packets and keeps a table to reverse it on replies. One public address has only 65,535 ports, so heavy users behind one NAT can run out.

TCP reliability

TCP numbers every byte, so anything missing can be asked for again

ClientServerSYN: “my bytes start at 1000”SYN-ACK: “got it, mine start at 5000”ACKhandshake:one round tripbefore any databytes 1001 – 1460ack 1461bytes 1461 – 1920✗ lostbytes 1921 – 2380ack 1461 againresend bytes 1461 – 1920ack 2381every byte is numbered;an ack names the nextbyte it expectsa gap: it keeps askingfor 1461the sender fillsonly the gapTCP turns a lossy wire into a complete, in-order stream of bytes.UDP skips all of this: an 8-byte header and no promises. Calls, games and DNS use it, because a late packet is worth nothing to them.

TCP builds a reliable stream from unreliable packets: number the bytes, acknowledge them, resend what was not acknowledged.

TCP reliability · Going deeper

TCP three-way handshake and small data exchange Client Server SYN seq=x SYN-ACK seq=y, ack=x+1 ACK ack=y+1 established · 1.5 RTT later DATA seq=x+1, len=460 ACK ack=x+461 cumulative — covers everything up to x+460

Handshake and acknowledgements in detail

Starting numbers are random so an outsider cannot guess them and inject data. Acks are cumulative; selective acks (SACK) let the receiver list exactly which ranges it holds. Two windows limit the sender: the receiver’s buffer, and the sender’s own estimate of what the network can take.

Check yourself

what the receiver has so far1001 – 14601921 – 2380missingthe byte stream, left to right

The receiver holds bytes 1001 – 1460 and 1921 – 2380. What does it acknowledge?

Congestion control

Senders find the network’s speed by probing: climb slowly, fall fast

packets the sender allows in flight at once020406080051015202530round trips (50 ms each: the whole chart is 1.5 seconds)start: double every round trip10 → 20 → 40 → 80one packet lost: cut in halfthen +1 per round trip: 26 trips later, still only 66Climbslowly,fallfast.That iswhat letssendersshare alink fairly.No one tells a sender how fast the network is. It probes: speed up until a packet drops, back off, try again.If senders did not back off, queues would fill, drops would cause resends, and the network would collapse.

Congestion control is an agreement among strangers: back off when a packet drops, so the shared link does not collapse.

Congestion control · Going deeper · 1 of 2

Congestion window: slow start, AIMD, loss cwnd time ssthresh loss loss slow start × 2 / RTT congestion avoidance + 1 / RTT cwnd ← cwnd / 2

Slow start, then additive increase

Most web requests finish before leaving slow start. CUBIC, the Linux default, grows as a cubic function of time since the last loss. BBR measures bandwidth and round-trip time directly instead of waiting for loss. Oversized router buffers hide loss and add delay (bufferbloat); CoDel drops early to keep latency low.

Congestion control · Going deeper · 2 of 2

TCP vs UDP on the same lossy exchange TCP UDP client server client server SYN SYN-ACK ACK data #1 data #2 (lost) ACK #1 retransmit #2 datagram #1 datagram #2 (lost) datagram #3 no ACK · no retransmit · #2 simply gone connection · ordered · reliable · congestion-controlled connectionless · datagram-preserving · best-effort

When reliability is the wrong goal: UDP

UDP has an 8-byte header and no connection, ordering or resending. A video frame 200 ms late is useless, so real-time traffic prefers it. TCP also has head-of-line blocking: one lost segment holds back everything behind it. QUIC, built on UDP, gives each stream its own recovery.

DNS

DNS turns a name into a number by asking three servers in turn

Your laptopasks one questionResolverrun by your providerdoes the walking,remembers answersRoot servers13 names, about 1,800 machines worldwide.com serversknow who runs every .com nameexample.com’s own servershold the real answer1 ask2 ask“try the .com servers”3 ask“try example.com’s servers”4 ask“it is 93.184.216.34”5 the numbertwo referrals, one answerNames are for people, numbers are for routers. DNS is the worldwide phone book between them, with no single owner.Each answer says how long it may be remembered: 60 – 300 seconds if you need to switch servers fast, days if it never changes.

DNS is a worldwide, cached lookup with no central owner. Your resolver walks the tree once and remembers the answer.

DNS · Going deeper

DNS resolution walking root, TLD, authoritative Client Recursive resolver Root . TLD .com Authoritative stub resolver cache hits answer most queries A? www.example.com 93.184.216.34 .com NS? referral → .com example.com NS? referral → ns1.example.com www.example.com A? 93.184.216.34 (TTL 300) Cache effects Every step is cached at the resolver. Most production queries are cache hits — never touch the authoritative.

Records, lifetimes and privacy

A records map a name to IPv4, AAAA to IPv6, CNAME is an alias, MX names mail servers, TXT holds proofs. Plain DNS is unencrypted and unauthenticated: DNS over HTTPS hides the question, and DNSSEC signs the answer. Treat a TTL as a minimum time to propagate, not a promise.

Idempotency and retries

A request can arrive twice, so make repeating it harmless

Every web exchange has the same shapePOST /pay HTTP/1.1Content-Type: application/json{ "order": 42, "amount": 20 }HTTP/1.1 201 CreatedContent-Type: application/json{ "receipt": "r_981" }request: a verb (GET reads, POST submits), headers, a bodyreply: 2xx worked · 3xx look elsewhere · 4xx your mistake · 5xx the server’sThe trap: a request can arrive twiceClientServerPOST /paycharges 20✗ reply lostno reply:try againPOST /paycharges 20 againThe fix: make repeats harmlessSend a unique key with therequest. The server sees thesame key twice and returns thefirst result, charging once.Idempotency-Key: 7f3a…GET, PUT and DELETE are safe torepeat by design. POST is not.

HTTP is a verb, headers and a body, answered by a status code. Because retries happen, safe systems make requests idempotent.

Idempotency and retries · Going deeper · 1 of 2

Anatomy of an HTTP request POST /api/v1/orders HTTP/1.1 Host: api.example.com Authorization: Bearer eyJhbGciOiJSUzI1NiIs… Content-Type: application/json Content-Length: 47 Accept-Encoding: gzip, br {"sku":"WIDGET-001","qty":3,"customer":1234} request line: method · target · version headers: cross-cutting metadata blank line, then optional body

Anatomy of a request

Most engineering lives in the headers: Authorization for credentials, Content-Type for format, Cache-Control for lifetime, If-None-Match for a conditional fetch that returns 304 instead of the body. Caching is the hardest part: many “works for me” bugs trace back to a cache header.

Idempotency and retries · Going deeper · 2 of 2

HTTP/1.1, HTTP/2, HTTP/3 under one packet loss HTTP/1.1 over TCP+TLS HTTP/2 over TCP+TLS HTTP/3 over QUIC (UDP) TCP TLS 1.3 req 1 req 2 req 3 req 4 3 RTT to first byte; sequential requests; one in flight at a time browsers open 6 parallel TCP connections per origin to compensate TCP TLS 1.3 streams 1–N interleaved packet loss stall 3 RTT to first byte; concurrent streams over one TCP connection but TCP delivers in order: one lost packet stalls every stream QUIC (TLS bundled) streams 1–N packet loss other streams keep flowing 1 RTT to first byte (0 RTT on resumption); per-stream loss recovery

Three wire formats, one meaning

HTTP/1.1 is text, one request at a time per connection, so browsers open six. HTTP/2 interleaves many streams on one TCP connection, but one lost packet stalls them all. HTTP/3 runs on QUIC, where a loss stalls only its own stream.

Check yourself

ClientServerPOST /pay?no reply came back

A client’s payment request times out. It cannot tell whether the server processed it. What makes a retry safe?

TLS key exchange

TLS lets two strangers agree a secret key in public

Your browserServerhello + my public value Ahello + my public value B + certificate + signaturesecret a + B→ key Ssecret b + A→ the same key SAnyone watchingsees A and B, not Severything from here is encrypted with Sone round trip, then fast encryption for every byteIs this really the server?example.comcertificate, valid 90 dayssigned byIntermediate authorityvalid for yearssigned byRoot authorityone of about 150 your browser trustsone broken link and the browser refusesTLS does four jobs in one handshake: agree a key in public, encrypt, detect tampering, and prove who the server is.The secrets a and b are thrown away afterwards, so stealing the server’s key next year cannot unlock today’s traffic (forward secrecy).It hides what you say, not who you talk to: the site’s name is still visible to the network.

TLS is the S in HTTPS: one round trip to agree a key and prove the server’s identity, then everything is encrypted.

TLS key exchange · Going deeper · 1 of 2

TLS 1.3 1-RTT handshake Client Server ClientHello cipher suites · ephemeral key · random ServerHello ephemeral key · {cert, signature, Finished} encrypted 1 RTT later · keys established · transcript verified {Finished} + Application Data first encrypted application byte at 1 RTT

The TLS 1.3 handshake

The key exchange is elliptic-curve Diffie-Hellman; bulk data then uses AES-GCM or ChaCha20 at gigabits per second. Resumed connections can send data on the first packet (0-RTT), which is faster but replayable, so only safe for idempotent requests.

TLS key exchange · Going deeper · 2 of 2

TLS certificate chain validation Leaf cert Intermediate CA Root CA CN = example.com valid 90 days signed by → e.g. Let's Encrypt R3 valid years signed by → e.g. ISRG Root X1 self-signed in OS trust store verifies verifies Browser walk: 1. Verify the leaf's signature using the intermediate's public key. 2. Verify the intermediate's signature using the root's public key. 3. Confirm the root is in the trust store, the leaf name matches the host, nothing has expired or been revoked.

Walking the certificate chain

A certificate is a public key, a name and a signature. Leaves last about 90 days, intermediates years, roots decades. A stolen intermediate means thousands of certificates to reissue; a stolen root could impersonate any site until browsers ship an update.

Check yourself

video frames, in the order they were sent12456losteach frame is useful for about a fifth of a second

On a live video call, one packet is lost. What should the transport do?

Summary

Opening one web page uses every layer, and costs four round trips

You press Enter on https://example.com, 70 ms away. Time until the first byte of the page, to scale:0 ms70 ms140 ms210 msMost sites todayHTTP/2 on TCPDNSTCP handshakea reliable streamTLS handshakeprivate, and provenHTTPrequest, reply230 msNewer sitesHTTP/3 on QUICDNSQUIC + TLS togetherone trip for bothHTTPrequest, reply160 msThree round trips before anything useful, then one. Distance is paid for again on every trip.What each layer hides from the one aboveIP“any machine on Earth isreachable by number”TCP or QUIC“nothing is lost, doubledor out of order”TLS“nobody can read orchange this on the way”DNS and HTTP“a name is enough; hereis what I want”

Each layer hides one failure of the wire so the layer above can pretend. Next: instructions, the languages programs are written in.

Lesson complete

You finished “Networking”

Answer the questions along the way to see your score here.

Open in a new tab