The parts of a system
What is a system that serves millions made of?
A large system is a few standard parts: load balancer, app servers, cache, database, queue and CDN. Requirements decide which ones you need.
Before any detail, here is the whole picture this course fills in. A request leaves a user and reaches a load balancer, which shares requests between many identical app servers. The servers keep their data in a database. Two parts exist to save work. A cache keeps recent answers in memory, so the database is asked less often. A content delivery network keeps copies of files close to the user. Work that can wait goes on a queue, and workers take it from there. Around all of these sit the questions every part must answer: how available, what happens when it fails, how it is secured, what it costs. Each box names its lesson. None of these parts is needed until a requirement says so, which is why the course starts with requirements.
Requirements
What it must do is easy; how well it must do it shapes the design
Functional requirements say what the system does; non-functional requirements say how well, and they decide the architecture.
Every design starts with a brief, and the brief is never enough. Here is one: a photo-sharing app. The first thing to pull out is what it must do. Upload a photo. Follow people. See a feed. Like and comment. Search. These are the functional requirements, and almost any design meets them. The second list is how well it must do those things. Ten million users a day, doubling each year. The feed in under three hundred milliseconds. Available ninety-nine point nine percent of the time. No accepted upload ever lost. And personal data kept in the European Union. This list is what decides the servers, the zones, the database and the region. It is where the architecture comes from.
Constraints · Going deeper
Constraints are fixed facts, and each one removes options
A constraint is not negotiable: budget, deadline, team skills, law. Five constraints here leave two of six designs.
Quality attributes
A requirement without a number cannot be designed for or tested
Each quality attribute becomes a target with a number, a condition and a way to measure it.
People describe what they want in words. Architects need numbers. “It must be fast.” Fast for whom, at what load? Write it as: the feed returns within three hundred milliseconds for ninety-nine percent of requests, at ten thousand requests a second. “Always up” becomes ninety-nine point nine percent of minutes each month, measured at the load balancer. “It must scale” becomes: handles three times today’s peak with no redesign. “Secure” becomes encryption, multi-factor sign-in for staff, and logged access. And “never lose a photo” becomes: an upload is acknowledged only after it is stored in two zones. Each line can now be designed for, and each can be tested.
Back-of-envelope estimates
Daily users become requests per second with one division
300 million requests a day is about 3,472 a second. Design for the peak, here three times the average.
Before choosing a single service, turn the brief into numbers. Ten million daily users, each opening the feed thirty times, is three hundred million requests a day. A day has eighty-six thousand four hundred seconds, so that is about 3,472 requests a second on average. Traffic peaks in the evening, at perhaps three times the average: about 10,417 a second. That is the number to design for. Uploads are far fewer. Two million a day is twenty-three a second, sixty-nine at the peak. So reads outnumber writes a hundred and fifty to one. This is a read-heavy system, and that single fact will shape the cache, the database and the feed.
Back-of-envelope estimates · Going deeper
Six conversions, and the bandwidth the feed needs
Thumbnails alone send 150 TB a day, about 14 gigabits a second: bandwidth, not requests, calls for a CDN.
Storage growth
Data only grows: size the store for year three, not launch day
Year one stores 2.6 PB; with uploads doubling yearly, five years hold 81 PB.
Storage is the estimate people forget. Each upload keeps the original and its resized copies, about three point six megabytes. Two million a day is seven terabytes a day, and 2.6 petabytes in the first year. The brief says users double each year, so year two adds twice as much. By the end of year three the total is 18 petabytes. By year five, 81. Nothing is deleted, and growth compounds. That rules out any store that must be resized by hand, and it makes a plan for old, rarely opened photos part of the design from the start.
Latency numbers
A network round trip costs thousands of memory reads
A round trip inside a zone takes about 0.5 ms, between zones 1 to 2 ms, across an ocean about 150 ms.
Speed targets need a feel for what things cost. This scale goes up by ten at every step. Reading from memory: a hundred nanoseconds. A random read from a local solid-state disk: about twenty microseconds, two hundred times slower. A round trip to another server in the same data centre: half a millisecond. To another zone in the same region: one to two milliseconds. An old spinning disk seeking: five. Across the United States: sixty-five milliseconds. And Europe to California: a hundred and fifty, more than the hundred milliseconds that feels instant. So the hops in a request, and the distance to the user, matter far more than the speed of any one server.
Latency budget · Going deeper
Split the target between the hops, and each part has a number
A latency budget divides the 300 ms target between every hop, and keeps a margin for bad days.
Percentiles
The average hides the slow requests; the 99th percentile shows them
The median here is 72 ms and the average 91 ms, but the p99 is 520 ms: one request in a hundred is slower.
Response times are not one number but a spread. Here are ten thousand requests. Half finish within 72 milliseconds: that is the median, the fiftieth percentile. The average is a little higher and looks healthy. Ninety-five percent finish within 182. But one in every hundred takes more than 520 milliseconds. That is the ninety-ninth percentile, the tail. Those slow requests belong to real people, often the busiest ones. So a speed requirement always names a percentile, and three hundred milliseconds means three hundred for ninety-nine percent of requests.
The tail at scale · Going deeper
Wait for many services, and the rare slow reply becomes common
A page that waits on 100 services is slowed by a p99 reply 63% of the time.
Trade-offs
Every choice gives something up; write down what and why
A decision record keeps the context, the options, the decision and its consequences for each significant choice.
Architecture is a series of trade-offs, and good architects write them down. Here is one decision for the photo app. The context: a hundred and fifty reads for every upload, a three hundred millisecond target, and about two hundred people followed. Two options. Build the feed when it is opened, by querying everyone followed: simple, but two hundred lookups on every open. Or build it when someone posts, by adding the photo to every follower’s list: opening becomes one lookup, at the cost of 400 million inserts a day. The decision: fan out on write, except for very large accounts. And the consequences, which become the next pieces of the design: a queue, workers, and a key-value store.
How the exam asks · Going deeper
An exam question is a requirement plus one deciding phrase
Find the qualifier, such as least operational overhead or most cost-effective, and it chooses between workable answers.
Lesson complete
You finished “Requirements and estimates”
Nicely done.