Compute

Four ways to run code, from a whole machine to a single function

Virtual machineyou hand overa whole operating systemit starts inabout a minuteyou payper second it is onContaineryou hand overan app and its librariesit starts inabout a secondyou payper second it runsPlatformyou hand overyour source codeit starts inminutes, per deployyou payper instance it keeps upFunctionyou hand overone functionit starts inmillisecondsyou payper requestless for you to manage, smaller and shorter-lived unitsCompute is the part of the cloud that runs your code, and it comes in four sizes of “yours”.All four end up on the same physical servers. They differ in how much you bring, how fast a new copy starts and what themeter counts. Most real systems use two or three of them at once.

Compute is rented as virtual machines, containers, platforms or functions. Each step hands over more and starts faster.

Virtual machines

A virtual machine is a slice of a physical server

yours8 CPUs, 32 GBcustomer B4 / 16customer C16 CPUs, 64 GBD2 / 8E2 / 8virtual machines: each sees only its own sliceHypervisor: divides the hardware and keeps the slices apartOne physical server: 32 CPUs, 128 GB of memoryWidths are to scale: each slice’s width is its share of the 32 CPUs.A virtual machine is a fixed slice of a physical server: so many CPUs, so much memory, yours while it runs.You choose its size, its operating system image, its zone and its disk. It boots in about a minute and behaves like a realcomputer you can log in to. It is billed by the second for as long as it is on, busy or idle. Stop it and the compute chargestops; the disk keeps costing until you delete it.

A virtual machine gives you fixed CPUs and memory with your own operating system, billed per second while it is on.

Virtual machines · Going deeper · 1 of 3

Reading a machine type: family, generation, size

m5.largefamilym: generalpurposegen.5thsize2 CPUs, 8 GBAzureStandard_D2s_v5D family, 2 CPUs,s: fast-disk capable,version 5Google Cloude2-standard-2E2 series, standardbalance of CPU andmemory, 2 CPUsEach size up doubles the machine and the pricelarge: 2 CPUs, 8 GB$0.096 an hourxlarge: 4 CPUs, 16 GB$0.192 an hour2xlarge: 8 CPUs, 32 GB$0.384 an hour4xlarge: 16 CPUs, 64 GB$0.768 an hourThe name of a machine type is its specification: family, generation, size.Sizes are fixed menu items, not sliders, and each step doubles everything. Choosing the smallest size that does the job,and checking again later, is called right-sizing. Example prices: AWS m5 family, N. Virginia, 2025.

m5.large is general purpose, fifth generation, 2 CPUs and 8 GB. Each size up doubles the resources and the price. Picking the smallest that works is right-sizing.

Virtual machines · Going deeper · 2 of 3

Five families for five kinds of work

FamilyWhat it has more ofTypical workAWSAzureGoogleGeneral purposebalanced CPU and memoryweb servers, small databasesM, TD, BE2, N2Compute optimisedmore CPU per GB of memorybatch jobs, encoding, game serversCFC3, H3Memory optimisedmore memory per CPUin-memory databases, cachesR, XE, MM3Storage optimisedfast local disksdata warehouses, search indexesI, DLZ3AcceleratedGPUs attachedtraining AI models, renderingP, GNC, NDA3, G2Families trade CPU, memory, disk and GPUs against each other. Match the family to the bottleneck.A workload short of memory wastes money on a compute-optimised machine, and the reverse. General purpose is the defaultuntil measurements say otherwise. AWS’s T family and Azure’s B family are burstable: cheap machines that save up creditwhile idle and spend it on short bursts, which suits small sites and test servers.

General purpose, compute optimised, memory optimised, storage optimised and accelerated. Pick by what the workload runs short of.

Virtual machines · Going deeper · 3 of 3

Four ways to pay for the same machine

The machine is identical in every case. What changes is the promise you make, and the discount you get for it. Lesson 9 covers the arithmetic.

On demandno promise; full price; stop any second
Committedpromise 1 or 3 years of use; up to about 70% off
Spotspare capacity; up to 90% off; can be taken back at short notice
Dedicateda physical server to yourself; costs more
  • On demand is the default and suits anything new, short-lived or unpredictable.
  • Commitments are called Savings Plans and Reserved Instances on AWS, reservations and savings plans on Azure, and committed use discounts on Google Cloud. They suit the steady base load that is always there.
  • Spot machines are the provider’s unsold capacity. AWS gives a two-minute warning before taking one back; Azure and Google give about thirty seconds. They suit work that can be interrupted and restarted: batch jobs, rendering, test runs.
  • Dedicated hosts exist for licences counted per physical processor, and for rules that forbid sharing hardware with other customers.
A common exam pattern: “fault-tolerant batch job, lowest cost” is spot. “Steady database for three years” is a commitment. “Licence tied to physical cores” is a dedicated host.

Autoscaling

A group of identical machines that grows and shrinks

The rule you writekeep average CPUnear 50%minimum 2, maximum 8A group of identical machines, spread over zonesminimum: always onadded while busyroom up to the maximuma machine that fails its health check is replaced automaticallyAutoscaling: you state a target, and the group adds and removes machines to hold it.This is the elasticity of lesson 1, made concrete. The minimum keeps the service alive, the maximum caps the bill, and the rulemoves the count between them. A load balancer in front spreads requests over whatever machines exist at that moment.It needs machines that are interchangeable: nothing important may live only on one of them.

An autoscaling group holds a target, such as 50% CPU, by adding and removing machines between a minimum and a maximum.

Autoscaling · Going deeper

Image, template, group: the same three parts everywhere

AWSAzureGoogle CloudThe groupAuto Scaling groupVM Scale SetManaged instance groupThe template for new machineslaunch templatescale set modelinstance templateThe machine imageAMIVM imageimageFour kinds of scaling ruleTarget trackinghold a number, like a thermostat:CPU at 50%Stepadd 2 at 70% CPU,add 4 at 90%Scheduledmore machines at 9 amon weekdaysPredictivelearn the daily patternand scale ahead of itThe same three parts on every provider: an image, a template and a group with a rule.The image is a snapshot of a prepared disk, with the operating system and your software installed. The template adds the sizeand settings. The group stamps out copies. New machines take a minute or two to boot, so rules add capacity earlyand remove it slowly.

Auto Scaling groups on AWS, Virtual Machine Scale Sets on Azure and managed instance groups on Google Cloud all stamp machines from a template.

Containers

A container packs the app with what it needs and shares the OS

Three virtual machinesAppLibrariesGuest OSAppLibrariesGuest OSAppLibrariesGuest OSHypervisorPhysical servereach carries a whole operating system:gigabytes in size, about a minute to bootThree containersAppLibrariesAppLibrariesAppLibrariesOne shared operating systemVirtual machinePhysical servereach carries only the app and what it needs:megabytes in size, about a second to startA container packs an app with its libraries and shares the operating system underneath.Because the package holds everything the app needs, it runs the same on a laptop, a test server and in production.Because it does not hold an operating system, it is small and starts fast, so one machine can run dozens. Containers sharethe host’s kernel, so their isolation is lighter than a virtual machine’s.

A container is an app plus its libraries in one package. It has no operating system of its own, so it is small and starts fast.

Containers · Going deeper

From recipe to registry to running: the life of an image

Dockerfilea recipe: start fromPython 3.12, add my codebuildImagea sealed package,tagged shop:1.4pushRegistryECR · ACRArtifact Registrya laptopa test serverproductionpullBuild once, run anywhere: the image is the unit you ship.An image is built from a short recipe, stored in a registry under a version tag, and pulled by whatever is going to run it.A running copy of an image is a container. Because the same sealed package runs at every stage, “it worked on my machine”stops being a category of bug. The registries are ECR on AWS, ACR on Azure and Artifact Registry on Google Cloud.

A Dockerfile builds an image; the image is pushed to a registry; anything that pulls it runs an identical container.

Container orchestration

An orchestrator keeps your containers running across many machines

Kubernetesyou declare:“run 6 copiesof shop:1.4”it keeps that trueMachine 1Machine 2Machine 3machine failedthe two lost copies are restarted on the machines that remain (purple)An orchestrator keeps the number of containers you asked for running, whatever happens to the machines.You describe the result you want, not the steps. Kubernetes places containers on machines, restarts the ones that die,replaces versions gradually and spreads traffic across the copies. Every provider sells it managed: EKS on AWS, AKS on Azure,GKE on Google Cloud. A group of containers that run together is called a pod.

Kubernetes takes a description, such as six copies of this image, and keeps it true as machines come and go.

Container orchestration · Going deeper

Container services, and whether you manage the machines

AWSAzureGoogle CloudManaged KubernetesEKSAKSGKEThe provider’s own, simpler orchestratorECSnonenoneContainers with no machines to manageFargateContainer AppsCloud RunImage registryECRACRArtifact RegistryTwo questions choose a container service: Kubernetes or not, and do you manage the machines?With plain EKS, AKS or GKE you still own a fleet of virtual machines that the containers run on, and you patch and size it.The serverless options remove that fleet: you hand over an image, say how much CPU and memory it needs, and pay per secondit runs. Cloud Run and Container Apps can also scale to zero when no requests arrive.

EKS, AKS and GKE are managed Kubernetes. Fargate, Container Apps and Cloud Run run containers with no machines for you to manage.

Serverless functions

A function runs only when something happens, and bills by the millisecond

a web requesta file uploadeda message queueda timerEventsA function: runs for each event, then is gone10 runs of 200 ms eachidle: no server, no chargeA virtual machine doing the same workbilled for all 60 seconds0 s20 s40 s60 sOne minute, to scale: each purple line is 200 milliseconds wide.A function exists only while it handles an event: here 2 seconds are billed out of 60.You upload code for one task and name the event that triggers it. The provider starts a copy when the event arrives, runsas many copies as there are events, and removes them after. This is what serverless means: no machine to size, patch orleave running. AWS Lambda, Azure Functions and Cloud Run functions are the three.

A serverless function starts on an event, runs your code, and disappears. Between events nothing runs and nothing is billed.

Serverless functions · Going deeper · 1 of 2

Cold starts and time limits

First request after a quiet spell: a cold startfetch your codestart the runtimeyour code runsextra delay: from about 100 ms to over a secondRequests that follow soon after: warmthe copy is kept ready for a few minutes and reusedAWS Lambda’s limits15 minuteslongest single run10 GBmost memory per copy1,000copies at once, by defaultAzure Functions and Cloud Runfunctions have limits of the samekind; the numbers differ.The price of “nothing running” is that the first request must wait for something to start.Cold starts matter for user-facing requests and are irrelevant for background work. Keeping a few copies permanentlywarm removes them, for a fee. The time limit matters more: a function suits a task that finishes in seconds or minutes.Anything long-running, or needing a steady connection, belongs in a container or a virtual machine.

The first request after idle waits for a cold start. AWS Lambda allows 15 minutes per run and up to 10 GB of memory.

Serverless functions · Going deeper · 2 of 2

When a function costs less than a machine, and when it does not

A month of requests, each 200 ms at 512 MB1 million requests$1.8710 million requests$18.67100 million requests$186.67one virtual machine, always on: $70.08Functions are almost free when traffic is light or spiky, and lose to an always-on machine under steady heavy load.Computed from AWS Lambda list prices, 2025: $0.20 per million requests plus $0.0000166667 per GB-second, before thefree allowance of 1 million requests a month. The machine is the 2 CPU, 8 GB example from lesson 1 at $0.096 an hour;whether one machine could carry 100 million requests depends on the work, so treat the line as a guide, not a rule.

At a million requests a month a function costs under two dollars. At a hundred million it costs more than a machine that is always on.

Platform as a service

A platform turns “here is my code” into a running website

Youpush codeThe platform does the restbuilds itstarts serversbalances loadscales on trafficpatches the OSno machine for you to log in toa live URLAWS Elastic Beanstalk · Azure App Service · Google App EngineA platform turns “here is my code” into a running, load-balanced, patched website.It is the PaaS of lesson 2 applied to web applications and APIs. You choose a language runtime and a size; the platformhandles the build, the servers, HTTPS and scaling. You give up choices about the operating system and the network.When a standard runtime is all you need, this is the least work for a site that is always on.

An application platform builds, deploys, balances, scales and patches for you. You bring only the code.

Choosing compute

Take the least you have to manage

A new copy starts inYou look afterBilledReach for it whenFunctionmillisecondsonly the codeper requestshort tasks set off by events; spiky trafficPlatformminutes to deploycode and settingsper instancea standard website or API, always onContainerabout a secondthe image; maybe the clusterper second it runsmany services that must be portableVirtual machineabout a minutethe OS and all that is on itper second it is onold software, licences, special OS, GPUseach row down: more control, and more to manageStart at the top row and move down only when something forces you to.A task under 15 minutes that reacts to an event: a function. An ordinary web application: a platform. Several servicesthat must run the same everywhere: containers. Software that needs its own operating system setup: a virtual machine.The rows mix freely, and a typical system has a website on a platform, a few functions and one stubborn machine.

Function for short event-driven tasks, platform for standard web apps, containers for many portable services, machines for full control.

Lesson complete

You finished “Compute”

Nicely done.

Open in a new tab