Availability

Each extra nine allows ten times less downtime

Availabilitydown per 30-day monthdown per year99%7.2 hours3 days 16 hours99.5%3.6 hours43 hours 48 minutes99.9%43 min8 hours 46 minutes99.95%22 min4 hours 23 minutes99.99%4 min53 minutes99.999%26 s5 minutesBars on a logarithmic scale: each full nine is ten times shorter.Availability is uptime over total time, and every extra nine allows ten times less downtime.99.9% sounds near perfect and allows 43 min of downtime every month. Most agreements are monthly, so one badafternoon can spend a whole month’s allowance. The target comes from the business: what an hour down costs, andwho notices. Each nine costs more than the last, so pick the lowest number the business can live with.

Availability is uptime divided by total time. 99.9% allows 43 min of downtime a month.

Service level agreements · Going deeper

Provider SLAs promise more when you spread across zones

ServiceAWSAzureGoogle CloudOne virtual machineEC2 instance: 99.5%one VM, premium disks: 99.9%one instance: 99.9%VMs in two or more zonesregion level: 99.99%across zones: 99.99%multiple zones: 99.99%Managed database, multi-zoneRDS Multi-AZ: 99.95%SQL Database (zones): 99.995%Cloud SQL HA: 99.99%Object storage (availability)S3 Standard: 99.9%LRS/ZRS hot: 99.9%multi-region bucket: 99.95%Provider SLAs reward spreading out: the same machine is promised more when it has a twin in another zone.A service level agreement is a promise with a penalty: if the provider misses it, you receive a service credit, apercentage of that month’s bill, and only if you claim it. It is not compensation for lost business. Your owndesign decides what users actually get. Figures as published in 2026; they change, so check the current SLA pages.Note that durability (is the data still there?) and availability (can I reach it now?) are different promises.

An SLA is a promise backed by service credits. One VM is promised about 99.5 to 99.9%; VMs across zones 99.99%.

Error budgets

An SLO turns availability into a budget you can spend

SLI: what is measuredgood requests ÷ all requestsSLO: the internal target99.9% over 30 daysSLA: the promise to customers99.5%, with credits if missed43 min0budget for the month: 0.1% of 30 daysday 0day 10day 20day 30a bad deploya zone outage: 23 minutes5 minutes left: keep shippingAn SLO turns availability into a budget: the downtime you may spend this month.The SLI measures what users experience; the SLO is the team’s target for it; the SLA, looser than the SLO, is thecontract. At 99.9% the error budget is 43 minutes a month. While budget remains, the team ships features; whenit runs out, changes freeze and reliability work comes first. Unreliability becomes a number to manage, not a mood.

An SLI measures, an SLO targets, an SLA promises. A 99.9% SLO is an error budget of 43 minutes a month.

Parts in series

A chain is less available than its weakest link

Every part must work, so multiplyavailability so farDNS× 100%100% · 0 seconds a yearLoad balancer× 99.99%99.99% · 53 minutes a yearWeb servers× 99.95%99.94% · 5 hours 15 minutes a yearDatabase× 99.95%99.89% · 9 hours 38 minutes a yearPayment API× 99.9%99.79% · 18 hours 23 minutes a yearbar scale: 99.7% at the left edge, 100% at the rightParts in a chain multiply. The whole is less available than its weakest part.Five parts, each at 99.9% or better, give 99.79%: about 18 hours 23 minutes down a year, worse than any one of them.Every dependency on a request’s path, including other teams’ services and outside APIs, adds itself to the chain.So the fewer hard dependencies on the critical path, the better; anything optional should fail without failing the page.

Parts that must all work multiply: five parts at 99.9% or better give 99.79%.

Redundancy

Copies side by side multiply up, if they fail independently

1 copyeach 99%1 − 0.0199%3 days 16 hours a year2 copieseach 99%1 − 0.01×0.0199.99%53 minutes a year3 copieseach 99%1 − 0.01×0.01×0.0199.9999%32 seconds a yearRedundant copies multiply the chance of failure, so availability climbs fast: if failures are independent.The system is down only when every copy is down at once. Two copies at 99% give 99.99%; three give 99.9999%.The catch is the word independent. Two servers in the same rack, zone, or running the same bad release fail together,and then two copies are no better than one. Redundancy works only across the things that fail: zones, versions, regions.

With independent copies the system fails only if all fail: two at 99% give 99.99%.

Correlated failure · Going deeper

Copies that share something fail together

Shared thingHow it takes down every copyThe fixSame rack or hostpower or a switch failsspread instances: placement groups (spread), availability setsSame zonea data centre loses power or networkrun in at least two zonesSame releasea bad deploy reaches every copydeploy one zone or a few percent at a timeSame dependencya shared database or DNScache, degrade gracefully, keep a fallbackSame regiona regional control-plane eventdisaster recovery in a second regionSame credentialsone account deleted or compromisedseparate accounts, backups in a vaultCopies fail together when they share something. Find the shared thing and split it.The parallel formula assumes independence, and real outages are mostly correlated: a whole zone, a bad config pushedeverywhere, a shared dependency. Each row is a layer of the same idea, called a fault isolation boundary. Therack and zone rows cost little; the region and account rows are disaster recovery, priced by lesson 10.

Racks, zones, releases, dependencies, regions and accounts are fault isolation boundaries: spread copies across them.

Single points of failure

Find every part that exists only once

RegionZone aDNS1NAT253461DNS with one providermanaged DNS is spread worldwide; one record type is fine2One NAT gatewayprivate servers lose the internet if its zone fails: one per zone3One database instanceno standby: a crash is an outage and a restore4All servers in one zonea zone failure takes every one of them5Sessions on one serverlose the server, log everyone out (lesson 3)6One person with the keysthe only admin is on holidayA single point of failure is any part that exists only once. Find each one, then double it or accept it.Walk the request from the user to the data and ask of each box: what if this disappears? The answer is often athing nobody drew: the NAT gateway, the session store, the deploy pipeline, the one person with the password.Doubling is not always worth it; then write the risk down, with how long recovery takes.

A single point of failure is any part whose loss stops the system. Walk the request and ask of each part: what if it disappears?

Failover

A standby helps only if the switch to it is quick and automatic

0 s30 s60 s90 sprimary failsDetecthealth checks fail 3 times, 10 s apartPromotethe standby becomes the primaryRedirectDNS name moves; clients reconnectserving againTypical managed figures: one standby (RDS Multi-AZ instance) 60 to 120 s; two readable standbys (Multi-AZ cluster) under 35 s; Aurora with a replica about 30 s.A standby helps only when the switch to it is automatic and quick. Failover time is downtime.Failover has three parts: noticing the failure, promoting the standby, and moving clients to it. Each takes seconds,and a whole failover counts against the error budget. A proxy that holds connections open (RDS Proxy) shortens thelast part; a standby that is already serving reads shortens the second. A failover nobody has tested is a hope.

Failover time is detection plus promotion plus redirecting clients, and all of it is downtime.

Active-active and active-passive · Going deeper

Stateless tiers run all copies at once; databases usually cannot

Active-passiveactivestandby, idle✓simple; no conflicts between copies✗half the capacity waits; failover takes timeActive-activeactiveactive✓all capacity used; losing one is a dip, not an outage✗both must agree on shared dataStateless servers run active-active. Databases usually run active-passive, because writes must agree.Web and application servers keep no data of their own, so any number can serve at once behind a balancer.A database primary accepts writes and a standby follows it; promoting the standby is the failover. Writing in twoplaces at once is possible (multi-primary, global tables) but needs a rule for conflicts, covered in lesson 6.

Active-active uses every copy; active-passive keeps a standby, because writes must agree.

Zone capacity

To survive a lost zone, the others must already carry the load

2 zones: 12 servers in each, 24 in all100% more than the 12 the peak needsZone aZone bZone a lost: 12 servers remain, the peak needs 123 zones: 6 servers in each, 18 in all50% more than the 12 the peak needsZone aZone bZone cZone a lost: 12 servers remain, the peak needs 12To survive a zone loss, the zones that remain must already hold the full load. More zones waste less.With 12 servers needed at peak, two zones need 12 each (24 in all, double); three zones need 6 each (18, half as much again).This is static stability: no scaling has to happen during the failure, when the control plane may be busy too.Spare capacity is the price of availability, and spreading across more zones is the cheapest way to pay it.

With 12 servers needed, two zones must run 24 and three zones 18: spare capacity is the price of availability.

Lesson complete

You finished “Availability”

Nicely done.

Open in a new tab