Compute
Four ways to run code, from a whole machine to a single function
Compute is rented as virtual machines, containers, platforms or functions. Each step hands over more and starts faster.
Compute means the part of the cloud that actually runs your code. It comes in four forms. A virtual machine: you hand over a whole operating system with everything installed. It starts in about a minute, and you pay for every second it is switched on. A container: you hand over just the application and its libraries. It starts in about a second. A platform: you hand over source code, and it comes back as a running website. A function: you hand over one function. It starts in milliseconds, runs when something happens, and you pay per request. Left to right, you manage less, and the thing you rent gets smaller and shorter-lived.
Virtual machines
A virtual machine is a slice of a physical server
A virtual machine gives you fixed CPUs and memory with your own operating system, billed per second while it is on.
Start with the oldest and most flexible form. At the bottom is one physical server. Software called a hypervisor divides it up. Each slice is a virtual machine: a fixed number of processors and a fixed amount of memory, with its own operating system, behaving exactly like a separate computer. The purple one is yours, a quarter of this server. The others belong to strangers, and none can see the rest. You pick the size from a menu, pick an operating system, and it boots in about a minute. From then on you are charged by the second, whether it is working or idle.
Virtual machines · Going deeper · 1 of 3
Reading a machine type: family, generation, size
m5.large is general purpose, fifth generation, 2 CPUs and 8 GB. Each size up doubles the resources and the price. Picking the smallest that works is right-sizing.
Virtual machines · Going deeper · 2 of 3
Five families for five kinds of work
General purpose, compute optimised, memory optimised, storage optimised and accelerated. Pick by what the workload runs short of.
Virtual machines · Going deeper · 3 of 3
Four ways to pay for the same machine
The machine is identical in every case. What changes is the promise you make, and the discount you get for it. Lesson 9 covers the arithmetic.
The machine is identical in every case. What changes is the promise you make, and the discount you get for it. Lesson 9 covers the arithmetic. On demand is the default and suits anything new, short-lived or unpredictable. Commitments are called Savings Plans and Reserved Instances on AWS, reservations and savings plans on Azure, and committed use discounts on Google Cloud. They suit the steady base load that is always there. Spot machines are the provider’s unsold capacity. AWS gives a two-minute warning before taking one back; Azure and Google give about thirty seconds. They suit work that can be interrupted and restarted: batch jobs, rendering, test runs. Dedicated hosts exist for licences counted per physical processor, and for rules that forbid sharing hardware with other customers. A common exam pattern: “fault-tolerant batch job, lowest cost” is spot. “Steady database for three years” is a commitment. “Licence tied to physical cores” is a dedicated host.
- On demand is the default and suits anything new, short-lived or unpredictable.
- Commitments are called Savings Plans and Reserved Instances on AWS, reservations and savings plans on Azure, and committed use discounts on Google Cloud. They suit the steady base load that is always there.
- Spot machines are the provider’s unsold capacity. AWS gives a two-minute warning before taking one back; Azure and Google give about thirty seconds. They suit work that can be interrupted and restarted: batch jobs, rendering, test runs.
- Dedicated hosts exist for licences counted per physical processor, and for rules that forbid sharing hardware with other customers.
Autoscaling
A group of identical machines that grows and shrinks
An autoscaling group holds a target, such as 50% CPU, by adding and removing machines between a minimum and a maximum.
One machine is rarely the answer. A group is. You write a rule. Here: keep average processor use near fifty percent, never fewer than two machines, never more than eight. Two machines always run. That is the minimum, and it keeps the service alive if one fails. When traffic rises and the processors pass fifty percent, the group adds machines. When traffic falls, it removes them. The empty slots are room it may still use, and the maximum is your guard against a runaway bill. If any machine stops answering its health check, the group replaces it without being asked.
Autoscaling · Going deeper
Image, template, group: the same three parts everywhere
Auto Scaling groups on AWS, Virtual Machine Scale Sets on Azure and managed instance groups on Google Cloud all stamp machines from a template.
Containers
A container packs the app with what it needs and shares the OS
A container is an app plus its libraries in one package. It has no operating system of its own, so it is small and starts fast.
Virtual machines have a cost. Each one carries a complete operating system. Three applications means three operating systems to store, boot and patch. Each is gigabytes in size and takes about a minute to start. A container keeps only the part that differs: the application and its libraries. All the containers on a machine share one operating system underneath. So a container is megabytes, not gigabytes, and starts in about a second. And because the package contains everything the application needs, it behaves the same wherever it runs.
Containers · Going deeper
From recipe to registry to running: the life of an image
A Dockerfile builds an image; the image is pushed to a registry; anything that pulls it runs an identical container.
Container orchestration
An orchestrator keeps your containers running across many machines
Kubernetes takes a description, such as six copies of this image, and keeps it true as machines come and go.
A handful of containers you can start by hand. Hundreds need a manager. That manager is called an orchestrator, and the standard one is Kubernetes. You do not give it steps. You declare a result: run six copies of this image. It spreads them over the machines it has. Now one machine fails, and two copies vanish with it. Kubernetes notices that four is not six, and starts two more on the machines that remain. Nobody was paged. The same mechanism handles new versions, replacing copies a few at a time, and scaling.
Container orchestration · Going deeper
Container services, and whether you manage the machines
EKS, AKS and GKE are managed Kubernetes. Fargate, Container Apps and Cloud Run run containers with no machines for you to manage.
Serverless functions
A function runs only when something happens, and bills by the millisecond
A serverless function starts on an event, runs your code, and disappears. Between events nothing runs and nothing is billed.
The smallest unit of compute is a single function. It is tied to an event: a web request arrives, a file is uploaded, a message lands in a queue, or a timer fires. When the event happens, the provider starts your function, runs it, and removes it. On this one-minute timeline it ran ten times, for a fifth of a second each. Two seconds of compute in total, and that is all you pay for. The rest of the minute there is no server at all. A virtual machine doing the same work has to stay on for the full sixty seconds, waiting. This is what people mean by serverless.
Serverless functions · Going deeper · 1 of 2
Cold starts and time limits
The first request after idle waits for a cold start. AWS Lambda allows 15 minutes per run and up to 10 GB of memory.
Serverless functions · Going deeper · 2 of 2
When a function costs less than a machine, and when it does not
At a million requests a month a function costs under two dollars. At a hundred million it costs more than a machine that is always on.
Platform as a service
A platform turns “here is my code” into a running website
An application platform builds, deploys, balances, scales and patches for you. You bring only the code.
Between containers and functions sits the simplest option for an ordinary website. You push your code. The platform builds it, starts servers for it, puts a load balancer in front, adds servers when traffic grows, and keeps the operating system patched. You never log in to anything. What comes back is a live address with your application behind it. On Amazon this is Elastic Beanstalk, on Azure it is App Service, on Google Cloud it is App Engine. You give up choices about the operating system and the network, and in return there is very little left to do.
Choosing compute
Take the least you have to manage
Function for short event-driven tasks, platform for standard web apps, containers for many portable services, machines for full control.
To choose, read this table from the top and stop at the first row that fits. A function, if the task is short and set off by an event. A platform, if it is a standard website or programming interface that must always be on. Containers, if you have many services in different languages that need to run the same way everywhere. A virtual machine, when the software demands its own operating system setup, a particular licence, or special hardware. Each row down gives you more control and more to look after. Real systems use several rows at once, and that is normal.