Solution Architecture reference

126 commands, numbers and terms from the 13 lessons, one line each. The right-hand column opens the page that teaches it.

1Requirements and estimates

Parts of a large systemLoad balancer, app servers, cache, database, queue and workers, CDN; each added when a requirement needs itThe parts of a system
Functional vs non-functionalWhat it must do / how well it must do itRequirements
ConstraintA fixed fact (budget, region, team, law) that removes optionsConstraints
Quality attributeNeeds a number: "p99 under 300 ms", not "fast"Quality attributes
Requests per secondDaily requests ÷ 86,400 (about 100,000 s); peak is a few times thatBack-of-envelope estimates
Storage growthRows per day × row size × days; size for year threeStorage growth
Latency orderMemory ≪ SSD ≪ same-zone network ≪ cross-region round tripLatency numbers
Latency budgetSplit the target between the hopsLatency budget
p50, p99Typical request / the slow one in a hundred; the average hides itPercentiles
Tail at scaleWait on many services and the rare slow reply becomes commonThe tail at scale
Decision recordWhat you chose, what it gives up, and whyTrade-offs
Exam questionA requirement plus one deciding phraseHow the exam asks

2Availability

99.9% / 99.99% / 99.999%About 8.8 h / 53 min / 5 min down per yearAvailability
SLAProvider's promise, higher across zones than for one instanceService level agreements
SLO and error budgetTarget, and the downtime it allows you to spendError budgets
SeriesA × B: a chain is less available than its weakest linkParts in series
Parallel1 − (1 − A)ⁿ: copies multiply up if they fail independentlyRedundancy
Shared fateCopies sharing a zone, power or config fail togetherCorrelated failure
Single point of failureAny part that exists only onceSingle points of failure
FailoverHelps only if detection and switch are quick and automaticFailover
Active-active vs active-passiveStateless tiers run all copies; databases usually one writerActive-active and active-passive
N+1 zonesSurvivors must already carry the full loadZone capacity

3Scaling

Scale up (vertical)Bigger machine: simple, has a ceiling, one failure loses allVertical scaling
Scale out (horizontal)More machines: no ceiling, a failure costs one shareHorizontal scaling
Stateless serviceState moved out, so any server can be copied or removedStateless services
Session store or signed tokenKeep sessions off the serverSessions
Sticky sessionsTies a user to one server; avoidSessions
Auto scaling groupFollows load between a minimum and a maximumAuto scaling
Target tracking, step, scheduled, predictiveWays to decide when to scaleScaling policies
Target tracking arithmeticServers × metric ÷ target, rounded up; the metric must fall as servers are addedTarget tracking
Scale set, managed instance groupAzure’s and Google’s Auto Scaling groupAuto scaling on three clouds
Scaling lagReactive scaling arrives late; schedule known peaksScaling lag
Scale-in: draining and lifecycle hookDeregistration delay 300 s, then Terminating:Wait for up to an hour by defaultScale-in

4Load balancing

Load balancerOne address in front, only healthy servers behindLoad balancers
Cross-zone load balancingEvery node uses every zone’s targets; always on for ALB, off by default for NLBCross-zone balancing
Layer 4 vs layer 7Forwards connections (NLB) / reads requests (ALB)Layer 4 and layer 7
Round robin vs least connectionsFair to requests / fair to serversBalancing algorithms
ALB or NLBHTTP features and WAF, or static IP, UDP, PrivateLink and raw throughputALB and NLB
Listener ruleRoute by host and path to target groupsListener rules
Health checkUnhealthy and healthy thresholds take servers out and backHealth checks
DNS routingPicks the region before the first packetDNS routing
Simple, weighted, latency, failover, geolocationRoute 53 policies and their exam cluesRoute 53 policies
AnycastOne address worldwide, failover without DNSGlobal routing
Front Door, global Application LBAzure’s and Google’s worldwide HTTP entry pointsLoad balancers on three clouds

5Caching and content delivery

CacheAnswers repeated reads from memoryCaching
Cache layersBrowser, CDN, application, databaseLayers of caching
Hit ratioThe last few percent of hits cut database load the mostHit ratio
Cache-asideCheck cache; on a miss read the database and fill the cacheCache-aside
Write-through, write-behind, read-throughThe other patternsCaching patterns
Redis vs MemcachedData structures and replication / simple and multithreadedCaching patterns
TTLCaps how stale data can getTTL and invalidation
Invalidate on writeDelete the key when the data changesTTL and invalidation
StampedeHot key expires: let one request rebuild itCache stampede
CDNEdge copies near users; the origin sees only missesContent delivery networks
Cache-Control: max-age=31536000, immutableFor versioned file names; keep the page short-livedHTTP caching

6Databases at scale

Choose by access patternHow data is read and written decides the databaseChoosing a database
Read replicaMore read capacity, a moment behindRead replicas
Scaling orderFix queries, pool connections, cache, add replicas, scale up; shard lastScaling order
Multi-AZ standbySynchronous copy in another zone, loses nothing on failoverMulti-AZ
Standby vs replicaFor staying up / for reading moreStandby or replica
Aurora storageSix copies across three zones, shared by every replicaAurora storage
ShardingSplit by key across machines; pick a key that spreads loadPartitioning and sharding
RCU, WCUDynamoDB read and write units, rounded up by item sizeDynamoDB capacity
Read-after-writeRead from a copy and you may read the pastConsistency

7Queues and events

Tight couplingA direct call ties two services' speed and uptimeCoupling
QueueAbsorbs a spike; workers catch up at their paceQueues
Standard vs FIFOScale, best-effort order / order per message groupStandard and FIFO
Visibility timeoutMessage hides while worked on, returns if the worker failsMessage lifecycle
Dead-letter queueWhere messages go after too many failed triesMessage lifecycle
Peek-lock, ack deadlineService Bus’s and Pub/Sub’s visibility timeout: 1 minute and 10 s by defaultQueues on three clouds
Redrive policydeadLetterTargetArn plus maxReceiveCount on the source queue; visibility timeout six times the function timeoutRedrive policy
Idempotency keyMessages can arrive twice; handle them safely twiceIdempotency
Transactional outboxEvent stored in the same transaction as the data, published afterwards by a relayTransactional outbox
Queue vs topicOne worker per message / every subscriber gets a copyPublish and subscribe
Fan-outTopic delivering to several queuesPublish and subscribe
Event busRoutes each event by its content (EventBridge)Event buses
Queue, topic, bus, stream, workflowWhich to pickChoosing the tool

8Monoliths, microservices and serverless

Monolith, microservices, serverlessWhere the boundaries fallApplication shapes
ConcurrencyRequests per second × durationFunctions
Cold startA new instance pays for starting; warm ones do notCold starts
Reserved and provisioned concurrencyA free guarantee and cap; and paid, pre-initialised instancesConcurrency controls
Serverless costNothing when idle; steady heavy traffic is cheaper on serversServerless cost
API gatewayAuth, throttling, routing, before your code runsAPI gateways
REST API or HTTP APIREST has caching, usage plans, WAF and private endpoints; HTTP is cheaper and simplerREST and HTTP APIs
Fargate, Cloud Run, Container AppsManaged containers without serversManaged containers
Container Apps, Cloud RunAzure’s and Google’s serverless containers; both scale to zeroCompute on three clouds
Service boundaryA business capability with its own dataService boundaries
Monolith firstSplit when team and scale need itMonolith first

9Designing for failure

TimeoutWithout one, a slow dependency holds every threadTimeouts
BulkheadEach dependency gets its own poolBulkheads
Retry stormRetries multiply through layersRetries
Exponential backoff with jitterWait longer each try, with randomnessBackoff and jitter
Circuit breakerClosed, open, half-open: stop calling a failing dependencyCircuit breakers
Graceful degradationServe a smaller answer instead of an errorGraceful degradation
Load sheddingReject the excess so the rest succeedLoad shedding
Chaos engineeringBreak things on purpose, with a hypothesisChaos engineering
Canary, blue-greenRelease in small steps you can roll backSafe deployments

10Disaster recovery

RPOHow much data you may loseRPO and RTO
RTOHow long you may be downRPO and RTO
HA vs DRSurvives a failed part / survives losing a region or the dataHigh availability and disaster recovery
Backup and restore, pilot light, warm standby, active-activeCost against recovery timeDisaster recovery strategies
Replication vs backupCopies mistakes / goes back in timeReplication and backups
3-2-1, immutable backupsOther region and account, locked, restores testedAWS Backup
Failover orderData first, then trafficCross-region failover
DNS failover timeDetection plus the record's TTLDNS failover timing
Game dayRun the plan; an untested plan is a guessTesting recovery

11Security and network design

Public and private subnetsOnly the load balancer faces the internetVPC design
NAT gatewayOutbound internet for private subnetsVPC design
VPC endpointReach AWS services privately, not through NATNAT gateways and VPC endpoints
Security group chainingEach tier accepts only the tier in frontSecurity groups
Security group vs NACLStateful, per instance, allow only / stateless, per subnet, allow and denyNetwork ACLs
Envelope encryptionData key encrypts data; KMS key encrypts the data keyEncryption and keys
Secrets ManagerSecrets out of code, rotatedSecrets Manager
Implicit denyDenied unless a policy allows; explicit deny winsIAM and least privilege
WAF, Shield, CDNStop attacks at the edgeWAF and Shield
Transit gatewayHub for many networks instead of a meshHybrid and multi-VPC networking

12Cost, operations and reviewing a design

Right-sizingMatch instance size to measured useElasticity and right-sizing
Data transfer costFree in; charged out to the internet, across zones and across RegionsData transfer
On-demand, Savings Plans, SpotPeaks / committed floor / work that can waitPurchase options
Storage tiersRarely read data in infrequent-access or archive classesStorage classes
Savings Plan or Reserved InstanceCommit a spend (flexible) or a configuration (rigid); only zonal RIs and Capacity Reservations hold capacityCommitments compared
Metrics, logs, tracesSomething is wrong / what happened / where the time wentObservability
SLO and error budget"Reliable enough" as a number to spendSLOs and error budgets
CloudTrail, Config, BudgetsWho changed it, what it looked like, and whether spend will pass the limitTools on three clouds
Six pillarsOperational excellence, security, reliability, performance, cost, sustainabilityThe Well-Architected Framework
Design reviewTurns "is it good?" into a ranked list of risksDesign review

NeuroqAI · Solution Architecture · neuroqai.com/courses/solution-architecture/reference/