Everything in a computer is built from one part: the transistor. Voltage on its gate decides whether current may pass.
Last lesson, a bit was a decision. This lesson builds the machine that makes those decisions. It starts with one part. A transistor has three connections. When the voltage on the gate is low, nothing passes from source to drain. That is a zero. When the gate voltage is high, current flows. That is a one. It is a switch with no moving parts, worked by another wire. What makes it special is scale. A gate is about twenty nanometres long, it switches in about a picosecond, and a single chip holds one to two hundred billion of them.
Transistors · Going deeper
Inside the switch
This is a MOSFET. It has only two stable outputs, so a noise margin separates them and a bit can pass through millions of stages without degrading. The build has changed (planar, FinFET, gate-all-around) but the contract has not: voltage in, current allowed or denied.
Logic gates and adders
Wire switches together and they do logic, then arithmetic
A logic gate is a handful of transistors with a fixed rule. Combine gates and the rule becomes addition.
One switch decides one thing. Wire two together and you get rules. Put two switches in a row, and current gets through only when both are on. That is an AND gate. Put them side by side, and either one is enough. That is an OR gate. Now combine gates. An AND gate and an XOR gate together add two bits. One plus one gives sum zero, carry one: binary for two. Chain sixty-four of these and the chip adds sixty-four bit numbers. This is the point where the machine stops being electrical and becomes mathematical.
Logic gates and adders · Going deeper · 1 of 2
The four common gates
AND, OR, NOT and XOR are shorthand for the patterns that appear most. NAND alone, or NOR alone, is functionally complete: any Boolean expression can be built from one repeated gate. A truth table is the contract a gate keeps.
Logic gates and adders · Going deeper · 2 of 2
From half-adder to 64-bit adder
Two half-adders and an OR make a full adder, which also takes a carry in. Chained naively, the carry ripples through all 64 columns. Real designs use carry-lookahead to finish in about log₂(64) = 6 gate delays.
Clocked memory
Memory is a circuit that only listens when the clock ticks
A flip-flop copies its input at each clock tick and holds it until the next. That is how a chip remembers, and why it has a clock.
Gates forget. Same inputs, same output, with no past. To run a program, the machine must remember. Here is a clock: a signal that rises and falls at a steady beat. Below it, an input that changes whenever it likes. A flip-flop looks at its input only at the instant the clock rises, copies it, and holds that value until the next rise. Whatever the input does between ticks is ignored. That one rule keeps billions of parts in step. Sixty-four flip-flops side by side make a register. At three gigahertz, each tick lasts as long as light takes to travel ten centimetres.
Clocked memory · Going deeper
Why edges make design possible
A D flip-flop has one data input, one clock input and one output. Because changes between rising edges are ignored, a designer only has to make each stage finish within one cycle. Flip-flops plus logic make a state machine: every counter and controller in a chip.
The instruction cycle
A processor is one loop: fetch, decode, execute, write back
A CPU core reads one instruction, does the tiny thing it names, stores the result and moves to the next. Billions of times a second.
Add an adder, some registers and a clock, and you can build the loop that runs every program. The program counter holds an address. The core fetches the instruction stored there. It decodes the bits: this one says add, using registers one and two, result into register three. It executes: the adder computes five plus seven. It writes twelve into register three, and moves the counter to the next address. Then it does it again, billions of times a second. Everything else in a modern chip exists to make this loop go faster while looking exactly the same from outside.
The instruction cycle · Going deeper · 1 of 2
The four pieces of a core
Registers: 16 to 32 named slots, the fastest memory there is. The ALU does the maths. The control unit decodes and pulls the right wires. Buses move the bits. Every microcontroller still looks like this.
The instruction cycle · Going deeper · 2 of 2
The contract with software
An instruction set architecture says which instructions exist and how they are encoded. x86-64 has variable-length instructions of 1 to 15 bytes; Arm and RISC-V use fixed 32-bit ones. Everything above the ISA is portable; everything below is not.
Pipelining
A pipeline overlaps instructions, like an assembly line
Pipelining starts the next instruction before the last one finishes. Each still takes five ticks, but one completes every tick.
Running one instruction at a time wastes most of the chip. Here three instructions go one after another. Each passes through five stages, so three of them take fifteen ticks. Now overlap them. As soon as the first instruction leaves the fetch stage, the second enters it. By tick five, five instructions are in flight, each at a different stage. From then on one instruction finishes every tick. Five times the work, from the same gates and the same clock. But it only works if the chip always knows which instruction comes next.
Pipelining · Going deeper · 1 of 2
The classic five stages
IF fetch, ID decode, EX execute, MEM memory access, WB write back. Latency per instruction is unchanged; throughput is what scales. Modern cores go further and fetch and retire 4 to 8 instructions per cycle.
Pipelining · Going deeper · 2 of 2
When one instruction needs the last one’s result
If instruction 2 reads a register instruction 1 is still computing, the naive fix is to stall. The real fix is forwarding: route the result straight from one execute stage to the next. Out-of-order cores go further, keeping 200 or more instructions in flight and committing them in program order.
Check yourself
With a five-stage pipeline, how long does a single instruction take from start to finish?
Each instruction still passes through all five stages. The gain is that five are in progress at once.
Branch prediction
The chip guesses at every fork, and is right 95% of the time
Branch prediction keeps the pipeline full by betting on which way an “if” will go. A wrong bet costs 15 to 20 ticks of discarded work.
A pipeline must fetch the next instruction every tick. But at an if, the next instruction depends on a result that is not ready yet. So the chip guesses. If the guess is right, the work it did ahead of time is kept, and no time is lost. If the guess is wrong, everything it started on the wrong path is struck out. The pipeline is emptied and refilled from the right path, which costs fifteen to twenty ticks. It pays because the guesses are good: ninety-five or more out of a hundred. One catch. The discarded work still leaves traces behind, and reading those traces is the Spectre attack.
Branch prediction · Going deeper
How the guess is made
The simplest predictor is a 2-bit counter per branch with four states, from strongly not taken to strongly taken. One surprise does not flip it; two in a row do. Real predictors add history tables. A loop that runs ten times mispredicts once, at the exit.
Memory hierarchy
Memory is a pyramid because fast memory has to be small
Main memory is about 150 ticks away, so a cache keeps recent data close. Programs that reuse nearby data run far faster.
The loop can run an instruction every tick, but only if its data is at hand. The bars show how long one read takes, to scale. Registers answer at once. The first cache takes four ticks, the second twelve, the third forty. Each level is bigger and slower than the one above. Main memory takes about a hundred and fifty ticks. In that time the core could have run hundreds of instructions. An SSD is a thousand times further again. So the chip keeps copies of recently used data in the small fast levels, and when you ask for one byte it brings the sixty-four bytes around it. Code that walks through memory in order can run ten times faster than code that jumps about.
Memory hierarchy · Going deeper · 1 of 4
Sizes and waits at each level
Each level is roughly 5 to 30 times larger and 3 to 10 times slower than the one above. One cycle is about 0.3 ns at 3 GHz. A register access is more than a million times faster than a disk seek.
Memory hierarchy · Going deeper · 2 of 4
Locality: why the bet pays
Temporal locality: what was used is used again soon. Spatial locality: its neighbours are next. Both are properties of programs, not hardware. An L3 miss rate above a few percent is usually fixable.
Memory hierarchy · Going deeper · 3 of 4
Two cores, one cache line
When cores share a 64-byte line, the MESI protocol keeps their copies consistent. If two threads write different bytes of the same line, it ping-pongs between caches: false sharing, which can make a loop 10 times slower. Pad hot per-thread fields onto separate lines.
Memory hierarchy · Going deeper · 4 of 4
Translating addresses: the TLB
Programs see virtual addresses; RAM is indexed by physical ones. The TLB caches recent translations. A hit costs one cycle; a miss walks a four-level page table, four more RAM reads. Huge pages of 2 MB cover 512 ordinary 4 KB pages with one entry.
Check yourself
Two loops add up the same grid of numbers. Which is faster on a real machine?
Each read brings in 64 neighbouring bytes. Loop A uses them; loop B jumps to a new row every step and keeps waiting on main memory.
Multicore
Clocks stopped getting faster in 2005, so chips grew more cores
Transistors kept shrinking, but heat capped the clock: the power wall. Since then, more speed means more cores, not faster ones.
For decades, every new chip was simply faster. The number of transistors on a chip has kept climbing, hundreds of thousands of times since nineteen eighty-five. Clock speed climbed with it, and then stopped. The reason is heat. Power grows with voltage squared times clock speed, and voltage could not fall any further. Past about four gigahertz, a chip makes more heat than can be taken away. So the extra transistors went sideways: eight to thirty-two cores in a desktop, up to a hundred and ninety-two in a server. A single thread barely got faster. Using all those cores became the programmer’s job.
Multicore · Going deeper · 1 of 2
Why the wall is where it is
Dynamic power in CMOS is roughly C · V² · f. While voltage fell with each generation, power per transistor fell too. Leakage put a floor under voltage around 2005, and raising f at fixed V meant power no 1 cm² of silicon could shed.
Multicore · Going deeper · 2 of 2
Many cores, and memory that is not uniform
Cores share the L3 cache and coordinate through atomic operations. With more than one socket, memory is NUMA: local RAM is about 80 ns away, the other socket’s 120 to 150 ns. Where a thread runs and where its pages were first touched can change throughput by 1.5 to 2 times.
Check yourself
A program runs as a single thread. You move it from a 4-core chip to a 64-core chip with the same clock speed. What happens?
One thread uses one core. More cores only help programs written to split their work.
CPUs and GPUs
A CPU has a few clever lanes; a GPU has thousands of simple ones
No single shape of silicon wins. A GPU gives up cleverness per lane to run the same maths on thousands of numbers at once.
More cores is one way to grow sideways. There is a more extreme one. A CPU core spends most of its transistors on being clever: guessing forks, reordering work, caching. That is right for ordinary programs, which do one thing after another. A GPU makes the opposite trade. About ten thousand very simple lanes, in groups of thirty-two, all performing the same step on different numbers. For graphics and AI, where the same sum runs over millions of values, that gives five to twenty times the work per watt. An accelerator goes further still and does one job only. That is why a phone chip contains all three.
CPUs and GPUs · Going deeper · 1 of 2
Latency cores and throughput cores
SIMD is this idea inside a CPU: 4 to 16 lanes in a vector unit. SIMT is the GPU at scale, with warps of 32 lanes sharing one fetch unit. Branchy, irregular code belongs on the CPU; regular tensor and pixel maths on the GPU.
CPUs and GPUs · Going deeper · 2 of 2
A chip that never fetches an instruction
A systolic array is a grid of multiply-add cells that data flows through, one element per tick. Google’s TPU v4 has a 128 × 128 grid: 16,384 multiply-adds per cycle. It does matrix multiplication superbly and almost nothing else.
Summary
A computer is one trick repeated at six scales
Each layer hides the one below it. Next: orchestration, the software that shares this one machine among many programs.
Here is the whole machine. A transistor is a switch. A few switches make a gate that does logic. Gates make adders that calculate and registers that remember. Put those on a clock and you have a core running its loop three billion times a second. A pyramid of caches keeps it fed. And since two thousand and five, chips have grown sideways into many cores and special lanes. Each layer hides the one below. Next lesson: the operating system, which shares this one machine among many programs.
Summary · Going deeper
Where a cached line may live
A direct-mapped cache gives each address one slot: fast, but two hot addresses can evict each other forever. Real L1 caches are 8-way set-associative and L3 is often 16-way. The interfaces on this page are fixed by standards: the x86, Arm and RISC-V manuals, JEDEC for memory, PCI Express for the buses.
Lesson complete
You finished “Machines”
Answer the questions along the way to see your score here.