Machine learning fits a function to examples. Training is paid once; inference is paid on every request.
Everything so far in this course was software where a person wrote the rules. That works when the rule is short: if the salary is above fifty thousand and tenure above two years, approve. But nobody can write the rule that tells a cat from a dog, pixel by pixel. Machine learning turns the arrangement around. You supply examples, inputs paired with the right outputs, and an optimiser adjusts a huge pile of numbers called weights until the function fits. It is still just arithmetic. No human wrote the constants. This has two phases with wildly different costs. Training happens once: weeks on thousands of processors, tens to hundreds of millions of dollars for a frontier model. Inference happens on every request: one pass through the arithmetic, milliseconds, billions of times. That recurring bill is what production is about.
Learning from examples · Going deeper
Training versus inference
Training iterates over the data nudging every parameter to reduce error. Inference is a single forward pass with no optimiser. Weights are shipped once and cached, and the inference path is heavily tuned. Train/serve skew is the quiet failure: inputs drift (a new locale, a recalibrated sensor) and quality degrades without any error. Monitor input statistics, not only output accuracy.
Kinds of learning
The feedback you have decides the kind of learning you can do
Supervised learns from labels, unsupervised from structure, reinforcement from reward.
Learning needs a signal that says how well you are doing, and there are three kinds. Supervised learning has labelled pairs: this photo is a cat, this email is spam. The label is the entire teaching signal, so the model can be no better than its labels. Unsupervised learning has no labels at all. It finds the shape of the data: which points group together, what is typical. Reinforcement learning has no dataset. An agent acts, the world returns a reward, and it adjusts to earn more. It needs fast trials and a clear reward. A modern language model uses all three in turn: first predicting the next word of raw text, where the text labels itself; then labelled instruction and answer pairs; then rewards from human preferences.
Kinds of learning · Going deeper
Three supervision signals
Supervised learning fits f(X) ≈ Y: classification, spam detection, translation. Unsupervised learns the input distribution itself: clusters, projections, density. LLM pre-training is self-supervised, since the label is extracted from the data. Reinforcement learning learns a policy that maximises cumulative reward; it powers game-playing systems and the human-feedback stage of LLM training.
Gradient descent
Learning is walking downhill on the error, one small step at a time
Gradient descent: move each weight a little in the direction that reduces the error, and repeat.
How does an optimiser find good weights? Take the smallest case: one weight. This curve shows the error for each value of that weight. It is lowest at three, but the optimiser cannot see the curve. It can only feel the slope where it stands. Start at five, where the error is four. The slope there is four, and positive: error rises to the right. So step left. With a learning rate of zero point one, the step is zero point four, landing at four point six, where the error is two point five six. Then four point two eight. Then four point zero two. Notice the steps shrinking. That is because the slope flattens near the bottom. After about twenty steps the weight is three, to three decimal places. A real network does exactly this, for billions of weights at the same time.
Gradient descent · Going deeper
The training loop
Nobody computes the slope over the whole dataset. Stochastic gradient descent uses a batch of 32 to a few thousand examples per step: noisier and far faster. Adam gives each weight its own effective learning rate and is the default for transformers. One pass over the data is an epoch; frontier language models do not finish even one.
Check yourself
The error is lowest at w = 3. You stand at w = 0, where the slope is −6. With a learning rate of 0.1, where does one step take you?
New w = w − rate × slope = 0 − 0.1 × (−6) = 0.6. A negative slope means error falls to the right, so the step goes right, and only a small way.
Neural networks and backpropagation
A network is layers of sums; one backward pass blames every weight
A neural network stacks simple layers. Backpropagation computes every weight’s slope in a single sweep.
One weight is easy. A function with billions needs a shape, and the shape is a network: layers of units, each a weighted sum of the layer before. Every line here is one learned weight. Training runs in two directions. Forward: the inputs flow through the layers and out comes a prediction. Compare it with the right answer to get the error. Backward: the error flows back through the same connections, and each weight receives its share of the blame. That backward sweep is backpropagation, and it yields every slope in one pass. Here it is for a single neuron. Input two, right answer five. The prediction is two, so the error is nine. The backward pass gives slopes of minus twelve and minus six. Nudge both weights, predict again: two point three, error seven point two nine. Lower. Repeat a few billion times.
Neural networks and backpropagation · Going deeper
Layers, nonlinearity and depth
A layer multiplies by a weight matrix, adds a bias and applies a nonlinearity such as ReLU. Stacked linear layers alone collapse into one. Deep networks multiply many slopes together, which can vanish or explode; residual connections, normalisation and careful initialisation are what made networks deeper than about ten layers trainable. A hidden layer in a modern LLM is several thousand units wide.
Overfitting
Two error curves tell you whether a model learned or memorised
Compare training error with error on unseen data. Overfitting and underfitting need opposite fixes.
A model that scores perfectly on its training examples may have learned nothing useful. It may have memorised them. So some data is held back, never trained on, and both errors are plotted as training proceeds. A good fit: both fall together and level off close to each other. The pattern was learned. Overfitting: the training error keeps falling toward zero, but the error on unseen data turns and climbs. From the dot onward the model is memorising. The fix is more data, or stopping at the dot. Underfitting: both stall high. The model is too small or has not trained long enough. The fixes are opposite, which is why you read the curves first.
Overfitting · Going deeper
Reading loss curves
Overfitting wants more data, regularisation, dropout or early stopping. Underfitting wants a larger model, better features or a longer schedule. Vision and tabular models run dozens of epochs and rely on early stopping to know when to quit.
Check yourself
These are a model’s two error curves. What should the team do?
Falling training error with rising unseen error is overfitting. Longer training or a bigger model would make it worse.
Model architectures
Each architecture assumes something; the transformer assumes least
CNNs assume locality, RNNs assume order, transformers let every position see every other, in parallel.
Real data has structure. Pixels next to each other are related; earlier words constrain later ones. Architectures differ in which structure they build in. A convolutional network slides one small filter, three pixels by three, across the whole image. The same edge detector works in the corner and in the centre, so it needs far fewer weights. It assumes that nearby pixels belong together. A recurrent network reads a sequence one step at a time, handing a memory forward. It assumes order matters. Its fatal weakness: step five must wait for step four, so it cannot use parallel hardware. A transformer drops the chain. Every token looks at every other token directly, all at once. It assumes almost nothing, and it runs fully in parallel. That is the design behind every large language model.
Model architectures · Going deeper
Convolution: one kernel, shared
A convolutional layer shares a small kernel across every position, and pooling between layers lets stacked small kernels see large regions. CNNs dominated vision through the 2010s. LSTMs and GRUs added gates that let recurrent networks carry signal over hundreds of steps, but training stayed sequential.
Attention
Attention: every word weighs every other word, all at once
Attention builds each token’s meaning as a weighted mix of all the others. Its cost grows with length squared.
Here is the transformer’s central operation. Take the sentence: the cat sat because it was tired. What does “it” refer to? To represent that word well, the model has to pull in meaning from elsewhere in the sentence. Attention does this with weights. For the word “it”, each other word gets a score for how relevant it is, and the scores are scaled to sum to one. Here most of the weight lands on “cat”. The new representation of “it” is that weighted mix. Every token does this with every other token, in a single parallel operation. That is its strength and its cost. A thousand tokens means a million pairs. Two thousand means four million. Double the text and the work quadruples.
Attention · Going deeper · 1 of 3
Scaled dot-product attention
Each position produces a query, a key and a value. Weights are softmax(Q · Kᵀ / √d), and the output is the weighted sum of values. For two tokens with identity projections the weights come out as 0.67 and 0.33. The cost is O(n²) in sequence length, which most long-context engineering attacks.
Attention · Going deeper · 2 of 3
Many heads
Multi-head attention runs several heads in parallel, each in a smaller subspace, and concatenates them. Without supervision one head ends up tracking syntax, another which pronoun refers to what, another distance. The parameter count is unchanged.
Attention · Going deeper · 3 of 3
Telling positions apart
Attention by itself ignores order: shuffle the tokens and the outputs shuffle with them. Positional encoding marks where each token sits, by adding a unique vector per position or, in newer models, rotating queries and keys by an angle proportional to position.
Next-token prediction
A language model predicts one token, then does it again
An LLM is a transformer trained to predict the next token. Generation is that prediction in a loop.
A large language model is a transformer trained on one task. Text comes in and is cut into tokens: pieces of words, each stored as a number. An English word is about one point three tokens. The tokens pass through the model. Out comes a probability for every token in the vocabulary, all hundred thousand or so. After “the capital of France is”, the token “Paris” gets ninety-two percent. One token is picked and appended to the text, and the whole model runs again for the next. That is all generation is. What is remarkable is what that one objective produces when trained on trillions of tokens: grammar, facts, reasoning patterns and working code, none of them specified by hand.
Next-token prediction · Going deeper · 1 of 2
Tokenisation
Byte-pair encoding starts from single bytes and repeatedly merges the most frequent adjacent pair. On a corpus of low ×5, lower ×2, newest ×6 the first merge is (w, e), seen 8 times. After thousands of merges common words are single tokens and rare strings stay split. The table is frozen at training time; every prompt and every bill passes through it.
Next-token prediction · Going deeper · 2 of 2
The causal mask
Position i may attend only to positions up to i. Blocked scores are set to minus infinity so they become zero weight. Without the mask, training would leak the answer. The same machinery without the mask gives a bidirectional model such as BERT, used for embeddings.
From predictor to assistant
Four stages turn a text predictor into an assistant
Pre-training gives knowledge; instruction and preference tuning give behaviour; serving makes it affordable.
A model trained only to continue text does not know it is in a conversation. Four stages get from there to a product. Pre-training: predict the next token over one to fifteen trillion tokens of text. Ask this base model for the capital of France and it may simply continue with another question, because that is what text often does. Instruction tuning: tens of thousands of examples of a prompt followed by a good answer. Now it answers: Paris. Preference tuning: people compare pairs of answers and choose the better one, and the model is pushed toward what they prefer. It becomes more helpful and learns to decline harmful requests. Then serving, the engineering to run it cheaply. Look at the bar: at this scale, all the tuning data together is too thin to draw. The knowledge comes from pre-training; the manners come from the rest.
From predictor to assistant · Going deeper
The training stack in detail
Instruction tuning is supervised fine-tuning on curated demonstrations. Preference tuning trains on human comparisons, either through a reward model (RLHF) or directly (DPO). Prompting is the interface on top: system prompts, few-shot examples and step-by-step triggers, where small wording changes can swing accuracy by tens of percent.
Cheaper inference
Three techniques make the same model far cheaper to run
The KV cache, quantisation and speculative decoding cut inference cost without changing the model.
Every generated token is a pass through every layer of the model, so serving cost is the constraint. Three techniques attack it. First, cache. To produce token eight, the naive approach redoes the work for the seven before it. But earlier tokens never change, so keep their results and compute only the new one. Second, store each weight in fewer bits. A model with seventy billion weights takes a hundred and forty gigabytes at sixteen bits. At four bits, thirty-five. Quality drops by a few percent, and the model moves from a datacentre to a laptop. Third, draft and check. A small model drafts four tokens cheaply; the big model checks all four in a single pass and keeps the ones it agrees with. Two to three times faster, with identical output.
Cheaper inference · Going deeper · 1 of 3
The KV cache
The keys and values that attention computes for past tokens are fixed once computed. Caching them cuts decoding from O(n²) to O(n) per new token. Serving stacks such as vLLM make managing this cache, its paging, sharing and eviction, their central design problem.
Cheaper inference · Going deeper · 2 of 3
Quantisation
Weights are re-encoded from 16-bit floats into 8 or 4 bits with scale factors found in a short calibration step. Memory falls 2 to 4 times and speed rises similarly on suitable hardware. It changes neither architecture nor training data.
Cheaper inference · Going deeper · 3 of 3
Speculative decoding
The verifier accepts the longest prefix of the draft it agrees with, so the output is identical to running it alone. Checking four candidates in parallel costs about the same as generating one. It pairs cleanly with quantisation and cache reuse.
Hallucination
Likely is not the same as true, so models can be fluent and wrong
Hallucination is structural: the model predicts what is likely, with no internal marker for what it knows.
Now the limits, stated plainly. Ask for a fact the model saw thousands of times and nearly all the probability lands on the right token. Ask for a fact it never saw, say a court ruling, and it still produces a distribution: three plausible case names, properly formatted, all invented. It picks one and continues with complete fluency. Both answers were produced the same way, as the likeliest continuation. Nothing inside the model distinguishes “I know this” from “this fits the pattern”. That is hallucination, and it comes from the objective itself. It is one of four standing limits. Open-ended work has no single right answer to grade against. Every token costs a full pass through the model. And clean data beats a cleverer architecture.
Hallucination · Going deeper · 1 of 2
The honest limits
Mitigations stack: retrieval to ground claims, citations so users can verify, output constrained to a schema, sampling several times and keeping the agreed answer. None eliminates hallucination. Given a fixed budget, more high-quality tokens often beat more parameters, and a few thousand careful fine-tuning examples often beat tens of thousands of mediocre ones.
Hallucination · Going deeper · 2 of 2
Four shapes of evaluation
Closed tasks can be scored for accuracy on held-out data. Open-ended tasks cannot. Public benchmark rankings correlate weakly with usefulness in a given product, and a model used as judge prefers length and its own style. Most teams combine a benchmark for regressions, a domain set written from real prompts, and an end-to-end harness.
Check yourself
A model cites a court case that does not exist, in perfect legal format. What best explains it?
There is no database inside. The model continues text with what is statistically likely, and a plausible case name is likely. Grounding in retrieved sources reduces this.
Context and tools
Extend a frozen model with context, or with a loop of tools
RAG puts the right passages in the prompt. An agent lets the model act, observe and repeat.
A trained model is frozen. It knows nothing about your documents, last week, or the state of any live system. Two patterns extend it without retraining. The first supplies knowledge at question time. Search your own documents for the passages closest to the question, paste the best five to twenty into the prompt, and let the model answer from them. That fixes the cutoff date and private data. It is not memory: the next request starts from nothing. The second lets the model act. It thinks about the next step, calls a tool, reads the result, and repeats until an outside check says the job is done. The catch is arithmetic. If each step is ninety-five percent reliable, five steps finish cleanly seventy-seven percent of the time. Twenty steps, thirty-six. Fifty steps, eight. The next lesson is about building both of these well.
Context and tools · Going deeper · 1 of 2
A retrieval pipeline
Documents are cut into chunks of a few hundred tokens, each embedded once as a vector of 768 to 4,096 numbers and stored. At query time the question is embedded and the nearest chunks fetched. Chunk size, mixing keyword with vector search, and a reranker that scores question and passage together are the main levers. Failure shifts from “the model did not know” to “the system did not retrieve it”.
Context and tools · Going deeper · 2 of 2
The agent loop
The model emits a structured tool call, the host runs it and returns the output into the next prompt. Each round is a full model call over the whole history. Failure modes compound: long-horizon error, drifting beyond the task, wrong arguments, runaway cost. Completion must be checked against outside signals such as passing tests, not the model’s own report.
Summary
Machine learning is a bet that old patterns still hold
Data, training, frozen weights, inference, and the engineering around them. Next: AI engineering.
Here is the lesson on one line. Data goes into training, a single expensive walk downhill on the error. Out come the weights: a frozen function with a cutoff date. Inference runs that function one token at a time, on every request. Around the model sits the engineering that makes it useful: retrieved context, tools in a loop, and tests that check what comes out. And four problems remain unsolved: hallucination, evaluation, cost and data quality. Underneath it all is a bet, that patterns in old data still hold for new data. Next, the engineering that makes that bet pay off in a product.
Summary · Going deeper
The intelligence pipeline
Training and inference are wildly asymmetric in cost, and most failures still come from the data, not the architecture. Standards here are mostly de facto: seminal papers, open formats such as ONNX, Safetensors and GGUF, model cards for documentation, and for governance the NIST AI RMF, the EU AI Act and ISO/IEC 42001.
Lesson complete
You finished “Intelligence”
Answer the questions along the way to see your score here.