Learning from examples

Instead of writing the rule, fit it from examples

Classical software: a person writes the ruleinputsalary, tenureruleif salary > 50000 …approveworks while the rules are short enough to write downMachine learning: examples in, the rule comes outexamplesphoto → “cat”trainingadjust weightsa functionbillions of weightsfor rules nobody can write: cat or dog, from raw pixelsTraining: onceweeks on thousands of GPUstens to hundreds of millions of dollarsweightsInference: every request, billions of timeseach one a single pass through the same arithmetic: milliseconds on one GPUA model is a function whose constants were found by an optimiser instead of written by a programmer.Training is the one-time cost; inference is the bill that recurs with every request, so production economics are inferenceeconomics. And the function only holds for data like its training data: when real inputs drift, quality falls with no error raised.

Machine learning fits a function to examples. Training is paid once; inference is paid on every request.

Learning from examples · Going deeper

Training and inference — same model, very different costs Training Inference find the pattern · run once apply the pattern · run many times Training data Architecture Optimizer (SGD / Adam) forward · loss · backprop · update Trained weights ≈10²³–10²⁵ FLOPs · weeks · thousands of GPUs $1M–$100M for a frontier model New input Trained weights Forward pass no gradients · no updates Prediction ≈10⁸–10¹¹ FLOPs / call · milliseconds $0.0001–$0.10 per call · billions of calls training is a one-time investment; inference is the recurring bill

Training versus inference

Training iterates over the data nudging every parameter to reduce error. Inference is a single forward pass with no optimiser. Weights are shipped once and cached, and the inference path is heavily tuned. Train/serve skew is the quiet failure: inputs drift (a new locale, a recalibrated sensor) and quality degrades without any error. Monitor input statistics, not only output accuracy.

Kinds of learning

The feedback you have decides the kind of learning you can do

Supervisedphotocatphotodogemailspamlearn from labelled pairsthe labels’ quality caps the model’sUnsupervisedno labels: find the structuregroups, compression, what is typicalReinforcementAgentWorldactionrewardact, be rewarded, adjustneeds fast trials and a clear rewardA modern language model uses all three, in this orderpredict the next word of raw textlabelled instruction → answer pairsrewarded by human preferencesThe kind of feedback you have decides the kind of learning you can do.If you have labelled examples, use them. Reaching for reinforcement learning when labelled data would do is one of the mostexpensive mistakes a team can make: without a fast simulator and a well-shaped reward it converges slowly or not at all.

Supervised learns from labels, unsupervised from structure, reinforcement from reward.

Kinds of learning · Going deeper

Three learning modes — different supervision signals, different problems Supervised Unsupervised Reinforcement labeled examples structure of the data trial · error · reward (X, Y) pairs cat → "cat" dog → "dog" cat → "cat" learn f(X) ≈ Y classification regression · translation learn P(X) · find clusters clustering · PCA · embeddings state s action a reward r games · robotics · RLHF most production ML is supervised; the most interesting frontier is the boundary between the three

Three supervision signals

Supervised learning fits f(X) ≈ Y: classification, spam detection, translation. Unsupervised learns the input distribution itself: clusters, projections, density. LLM pre-training is self-supervised, since the label is extracted from the data. Reinforcement learning learns a policy that maximises cumulative reward; it powers game-playing systems and the human-feedback stage of LLM training.

Gradient descent

Learning is walking downhill on the error, one small step at a time

w = 1w = 2w = 3w = 4w = 5errorlowest error: w = 3start: w = 5, error 44.6, error 2.564.284.02new w = w − rate × slopeslope at w = 5: 2 × (5 − 3) = 4rate 0.1: 5 − 0.1 × 4 = 4.6error falls from 4 to 2.56The slope flattens near the bottom,so the steps shrink by themselves.After about 20 steps: w ≈ 3.001Training is this walk downhill: measure the slope of the error, take a small step against it, repeat.A real network does the same for billions of weights at once, one slope per weight, on a small batch of examples per step.The learning rate sets the step size: too large overshoots the valley, too small never arrives. Typical: 0.001 to 0.00001.

Gradient descent: move each weight a little in the direction that reduces the error, and repeat.

Gradient descent · Going deeper

The training loop — one epoch is one pass over the data, broken into mini-batches for epoch in 1…E: shuffle dataset · split into mini-batches one epoch = one full pass over the training set for each mini-batch: Forward Loss Backward Optimizer step x → ŷ L(ŷ, y) ∂L/∂w (chain rule) w ← w − η · ∂L/∂w batch size learning rate η epochs E 32 · 256 · 4096 10⁻³ to 10⁻⁵ 1–100 (vision) · ≈1 (LLM) three knobs do most of the work — everything else is regularization and schedule

The training loop

Nobody computes the slope over the whole dataset. Stochastic gradient descent uses a batch of 32 to a few thousand examples per step: noisier and far faster. Adam gives each weight its own effective learning rate and is the default for transformers. One pass over the data is an epoch; frontier language models do not finish even one.

Check yourself

you: w = 0w = 3w = 6w = 0

The error is lowest at w = 3. You stand at w = 0, where the slope is −6. With a learning rate of 0.1, where does one step take you?

Neural networks and backpropagation

A network is layers of sums; one backward pass blames every weight

inputshidden layerprediction1 forward: inputs become a predictionerrorvs the right answer2 backward: each weight’s share of the errorOne neuron, one stepprediction = w × x + bx = 2, right answer 5, w = 1, b = 0forward: 1 × 2 + 0 = 2error: (2 − 5)² = 9backward: slope for w = −12, for b = −6update (rate 0.01): w = 1.12, b = 0.06again: prediction 2.30, error 7.29A network is layers of simple sums. Backpropagation finds every weight’s share of the error in one backward pass.Between layers sits a simple bend, such as “negative becomes zero”. Without it any stack of layers collapses into one.With it, depth composes features: edges into textures, textures into shapes, shapes into objects.

A neural network stacks simple layers. Backpropagation computes every weight’s slope in a single sweep.

Neural networks and backpropagation · Going deeper

A small feed-forward network — forward pass and backpropagation Input Hidden (ReLU) Output (softmax) forward pass · y = softmax(W₂ · ReLU(W₁ · x + b₁) + b₂) backward pass · ∂L/∂w via chain rule · update w ← w − η · ∂L/∂w W₁, b₁ W₂, b₂

Layers, nonlinearity and depth

A layer multiplies by a weight matrix, adds a bias and applies a nonlinearity such as ReLU. Stacked linear layers alone collapse into one. Deep networks multiply many slopes together, which can vanish or explode; residual connections, normalisation and careful initialisation are what made networks deeper than about ten layers trainable. A hidden layer in a modern LLM is several thousand units wide.

Overfitting

Two error curves tell you whether a model learned or memorised

error on the training dataerror on data the model has never seentraining time →Good fitboth fall together and level offthe pattern was learnedOverfitmemorised the training examplesfix: more data, or stop at the dotUnderfitboth stall highfix: a bigger model, or train longerJudge a model only on data it has never seen. The gap between the two curves tells you what is wrong.The two failures need opposite fixes, so read the shape first: more training makes overfitting worse and underfitting better.Mistaking one for the other wastes the next training run.

Compare training error with error on unseen data. Overfitting and underfitting need opposite fixes.

Overfitting · Going deeper

Training vs validation loss — three classic shapes Well-fit Overfitting Underfitting loss epochs epochs epochs train val (close) early stop val rises train falls both stuck high loss model has capacity data covers task capacity too high or data too small capacity too low or features too weak solid line: training loss · dashed: validation loss · the gap between them is the generalisation error

Reading loss curves

Overfitting wants more data, regularisation, dropout or early stopping. Underfitting wants a larger model, better features or a longer schedule. Vision and tabular models run dozens of epochs and rely on early stopping to know when to quit.

Check yourself

training dataunseen datatraining time →

These are a model’s two error curves. What should the team do?

Model architectures

Each architecture assumes something; the transformer assumes least

Convolutional networkone small 3×3 filter, reused everywhereassumes nearby pixels belong togetherfar fewer weights; still runs real-time visionRecurrent network12345a memory handed from step to stepone step at a timeassumes order mattersstep 5 waits for step 4: no parallelismTransformer12345every token looks at every other, at onceassumes almost nothing about structurefully parallel: exactly what GPUs do bestEach design builds in an assumption about the data. The transformer won by assuming least and running in parallel.A plain network must discover from scratch that neighbouring pixels or words are related. The right built-in assumption makeslearning fast. Recurrent networks could not use parallel hardware, which is the limit attention removed in 2017.

CNNs assume locality, RNNs assume order, transformers let every position see every other, in parallel.

Model architectures · Going deeper

A convolutional layer: shared kernel slides over the input grid Input image (5×5) Kernel (3×3) Feature map (3×3) w₀ w₁ w₂ w₃ w₄ w₅ w₆ w₇ w₈ slide kernel over every position · output[i,j] = Σ kernel · input[i:i+3, j:j+3] 9 weights apply to all positions · translation equivariance for free stack: conv → ReLU → pool → conv → … → fully-connected → class scores

Convolution: one kernel, shared

A convolutional layer shares a small kernel across every position, and pooling between layers lets stacked small kernels see large regions. CNNs dominated vision through the 2010s. LSTMs and GRUs added gates that let recurrent networks carry signal over hundreds of steps, but training stayed sequential.

Attention

Attention: every word weighs every other word, all at once

How much the word “it” draws from each word. Weights here are illustrative; they always sum to 1.Thecatsatbecauseitwastired0.030.580.080.040.100.050.12Every token does this with every other token, in one parallel operation:1,000 tokens → 1 million pairs · 2,000 tokens → 4 million · 200,000 tokens → 40 billionAttention lets each word gather meaning from the words that matter to it, wherever they sit in the text.Each token asks a question (a query); every token offers a label (a key) and a content (a value). Matching scores becomethe weights. Several heads run side by side and learn different relations unprompted. Doubling the text quadruples the work.

Attention builds each token’s meaning as a weighted mix of all the others. Its cost grows with length squared.

Attention · Going deeper · 1 of 3

Scaled dot-product attention — one head Input X Q · Kᵀ / √d_k Output [n × d_model] Q K V [n × d_k] [n × d_k] [n × d_v] softmax [n × n] attn · V [n × d_v] Three learned projections turn the input into queries, keys, values. Q · Kᵀ produces an n × n score matrix; softmax turns rows into probabilities. Each output row is a weighted sum of all V rows — every position attends to every other in one shot. cost: O(n² · d) per layer · multi-head runs h such heads in parallel and concatenates

Scaled dot-product attention

Each position produces a query, a key and a value. Weights are softmax(Q · Kᵀ / √d), and the output is the weighted sum of values. For two tokens with identity projections the weights come out as 0.67 and 0.33. The cost is O(n²) in sequence length, which most long-context engineering attacks.

Attention · Going deeper · 2 of 3

Multi-head attention — H parallel heads, each in a smaller subspace, concatenated Input X [n × d_model] head 1 head 2 head 3 head 4 Q,K,V Q,K,V Q,K,V Q,K,V … H = 8…128 syntax coref distance topic [n × d_k] [n × d_k] [n × d_k] [n × d_k] concat (head 1, head 2, …, head H) [n × H · d_k] · output projection W_o Output

Many heads

Multi-head attention runs several heads in parallel, each in a smaller subspace, and concatenates them. Without supervision one head ends up tracking syntax, another which pronoun refers to what, another distance. The parameter count is unchanged.

Attention · Going deeper · 3 of 3

Positional encoding — every position becomes a unique vector that gets combined with the token embedding Token embedding Positional encoding Sum (input to layer 1) E(▁the) E(▁cat) E(▁sat) d_model d_model d_model PE(0) PE(1) PE(2) E + PE(0) E + PE(1) E + PE(2) PE(pos, 2i) = sin(pos / 10000^(2i/d)) · PE(pos, 2i+1) = cos(...) low frequencies encode coarse position · high frequencies encode neighbour-level offsets RoPE rotates Q and K by angle proportional to position · ALiBi adds a per-distance bias to the score

Telling positions apart

Attention by itself ignores order: shuffle the tokens and the outputs shuffle with them. Positional encoding marks where each token sits, by adding a unique vector per position or, in newer models, rotating queries and keys by an angle proportional to position.

Next-token prediction

A language model predicts one token, then does it again

“The capital of France is”the text so far5 tokensword pieces, as numbersmodela transformera probability for every token“ Paris”0.92“ a”0.03“ the”0.02… and about 100,000 otherspick one, append it to the text, run the whole model again1 English word ≈ 1.3 tokensrare words split into smaller piecesvocabulary: 30,000 – 200,000 tokensfixed at training time, frozen afterwardstrained on 1 – 15 trillion tokenswith one goal: predict what comes nextA language model does one thing: given the text so far, give a probability for every possible next token.One token is picked and the model runs again. Grammar, facts, reasoning and code all emerge from that one objective atenough scale. During training a mask hides everything to the right of each position, so the model can never peek ahead.

An LLM is a transformer trained to predict the next token. Generation is that prediction in a loop.

Next-token prediction · Going deeper · 1 of 2

Byte-Pair Encoding — text becomes a sequence of subword token ids Input text Tokenizer (BPE merges) Token ids "the unfortunate cat" 19 characters merge most frequent pair repeat until vocab target ▁ marks word boundary [464, 22001, 9379] 3 ids ▁the ▁un fort unate ▁cat 464 555 3319 4470 9379 common words = 1 token · rare words split into subwords English ≈ 1.3 tokens / word · code is denser · CJK is sparser · emoji can take 3–4 tokens

Tokenisation

Byte-pair encoding starts from single bytes and repeatedly merges the most frequent adjacent pair. On a corpus of low ×5, lower ×2, newest ×6 the first merge is (w, e), seen 8 times. After thousands of merges common words are single tokens and rare strings stay split. The table is frozen at training time; every prompt and every bill passes through it.

Next-token prediction · Going deeper · 2 of 2

Causal attention mask — lower-triangular pattern that prevents peeking at future tokens Attention scores (Q · Kᵀ) After causal mask + softmax t₀ t₁ t₂ t₃ t₄ t₀ t₁ t₂ t₃ t₄ 2.10.41.10.71.5 0.32.40.81.00.6 0.91.22.00.51.3 0.40.61.42.20.9 0.71.10.51.32.5 1.0 0.10.9 0.20.30.5 0.10.20.30.4 0.10.20.10.30.3 −∞−∞−∞−∞ −∞−∞−∞ −∞−∞ −∞ t₀ t₁ t₂ t₃ t₄ t₀ t₁ t₂ t₃ t₄ all positions visible — encoder behaviour row i sums to 1 over columns 0…i only decoder-only models (GPT, Llama, Mistral) apply this mask in every layer · encoder-only models (BERT) do not same architecture, one mask — different model family

The causal mask

Position i may attend only to positions up to i. Blocked scores are set to minus infinity so they become zero weight. Without the mask, training would leak the answer. The same machinery without the mask gives a bidirectional model such as BERT, used for embeddings.

From predictor to assistant

Four stages turn a text predictor into an assistant

Pre-training1 – 15 trillion tokens of textgoal: predict the next tokenInstruction tuningtens of thousands ofprompt → answer examplesPreference tuningtens of thousands of human“A is better than B” choicesServingcompressed and cached,run billions of timesAsked “What is the capital of France?”, the model after each stage:continues the text, perhaps withanother questionanswers:“Paris.”answers helpfully, and declinesharmful requestsgives the same answers,cheaply enough to shipData used, drawn to scalepre-training text. All the tuning data together would be a few millionths of this bar: too thin to draw.Pre-training supplies the knowledge. Two small rounds of tuning turn a text-continuer into an assistant.Only a few labs can afford pre-training. The later stages are why one base model can become a chat assistant, a coding modelor a cautious product variant. Every model also has a cutoff date, and knows nothing after its training data ends.

Pre-training gives knowledge; instruction and preference tuning give behaviour; serving makes it affordable.

From predictor to assistant · Going deeper

The four-stage training stack of a modern instruction-tuned LLM Pre-train Instruction tune Preference tune Deploy next-token on text unsupervised SFT on (prompt, response) supervised RLHF / DPO human comparisons quantize · KV cache speculative decode ≈10T tokens ≈10–100K examples ≈10–100K comparisons runtime → a base model that completes text → follows instructions in a chat format → aligned to preferences helpful · harmless → low-latency low-cost serving ≈10²⁵ FLOPs ≈10²² FLOPs ≈10²² FLOPs per request weeks · 10K GPUs days · 100s GPUs days · 100s GPUs milliseconds 99% of the compute is in pre-training; 99% of the personality comes from the last two stages

The training stack in detail

Instruction tuning is supervised fine-tuning on curated demonstrations. Preference tuning trains on human comparisons, either through a reward model (RLHF) or directly (DPO). Prompting is the interface on top: system prompts, few-shot examples and step-by-step triggers, where small wording changes can swing accuracy by tens of percent.

Cheaper inference

Three techniques make the same model far cheaper to run

Cache past workwithout: redo every earlier tokenwith a cache: compute only the new onework per new token: n² → nearlier tokens never change, so keep their resultsFewer bits per weighta model with 70 billion weights16-bit140 GB8-bit70 GB4-bit35 GB4× less memory, 2 – 4× fasterquality lost at 4-bit: a few percentDraft small, check biga small model drafts four tokensthe big model checks all four in one passkeepkeepkeepredo2 – 3× faster, identical output60 – 80% of drafted tokens are acceptedEach generated token is a pass through every layer. A long agent task costs cents to dollars, times millions of users.Training happens once; every answer is paid for. Three techniques make the same model far cheaper to run.None of them changes what the model knows, and they combine. Quantisation is what moves a 70-billion-weight model froma datacentre GPU to a high-end laptop.

The KV cache, quantisation and speculative decoding cut inference cost without changing the model.

Cheaper inference · Going deeper · 1 of 3

KV cache — turns O(n²) decode into O(n) per new token Without KV cache With KV cache t₀ t₁ t₂ t₃ t₄ to predict t₅: recompute K,V for t₀…t₄ — every step step 1: 1 token step 2: 2 tokens step 5: 5 tokens total: 1+2+3+4+5 = O(n²) work K,V₀ K,V₁ K,V₂ K,V₃ t₄ cached each new token: read all cached K,V · compute one new K,V total: O(n) per new token cache memory grows linearly with context — KV cache dominates GPU memory at long contexts at 128K context, a 70B model can need ≈10 GB just for KV per request

The KV cache

The keys and values that attention computes for past tokens are fixed once computed. Caching them cuts decoding from O(n²) to O(n) per new token. Serving stacks such as vLLM make managing this cache, its paging, sharing and eviction, their central design problem.

Cheaper inference · Going deeper · 2 of 3

Quantization — same weights, fewer bits per number FP16 (baseline) INT8 INT4 16 bits / weight 8 bits / weight 4 bits / weight −1.4271 0.8103 2.5546 −1.43 · scale 0.81 · scale 2.55 · scale −1.4 · scale 0.8 · scale 2.5 · scale 70 B params → 140 GB datacentre GPU territory 70 B params → 70 GB single H100 / dual A100 70 B params → 35 GB consumer GPU / laptop 2× smaller 2× smaller per-channel calibration learns scale factors that bound quantization error precision loss is real but small · perplexity rises a few percent at INT4 · most users cannot tell activations and KV cache can be quantized too — different trade-offs, same idea

Quantisation

Weights are re-encoded from 16-bit floats into 8 or 4 bits with scale factors found in a short calibration step. Memory falls 2 to 4 times and speed rises similarly on suitable hardware. It changes neither architecture nor training data.

Cheaper inference · Going deeper · 3 of 3

Speculative decoding — small draft model proposes, big verifier checks in parallel Draft model small · cheap · fast propose 4 tokens the cat sat on Verifier (target model) one parallel forward pass over all 4 candidates the ok cat ok sat ok on no accept 3 · re-decode 1 guarantee: output identical to running verifier alone

Speculative decoding

The verifier accepts the longest prefix of the draft it agrees with, so the output is identical to running it alone. Checking four candidates in parallel costs about the same as generating one. It pairs cleanly with quantisation and cache reuse.

Hallucination

Likely is not the same as true, so models can be fluent and wrong

A fact it saw thousands of times“The capital of France is …”“ Paris”0.92“ Lyon”0.03“ the”0.02A fact it never saw“The 2019 ruling on this point was …”“ Smith v. Jones”0.31“ Brown v. Lee”0.27“ Adams v. Hill”0.22Both are produced the same way: the likeliest continuation. Nothing inside marks which one is known.Hallucinationfluent and false;reduced, never removedEvaluationopen-ended tasks haveno single right answerCostevery token is a passthrough the whole modelData qualityclean data beatsa cleverer architectureA language model optimises for what is likely, not for what is true. So it can be fluent and wrong.Grounding answers in retrieved sources, requiring citations and constraining the output all lower the rate; none brings itto zero. The model is the easy part: the hard work is the data, the evaluation and the system around it.

Hallucination is structural: the model predicts what is likely, with no internal marker for what it knows.

Hallucination · Going deeper · 1 of 2

The honest limits of modern ML — the unfixed bugs of the field Hallucination Evaluation gap Inference cost Data quality plausible-sounding but wrong cause: trained for likelihood, not truth mitigation: RAG · cite · constrained decoding no ground truth on open-ended tasks benchmarks weak predictors of utility mitigation: LLM-judge + human side-by-side ≈10⁸–10¹¹ FLOPs per token scales with users × tokens per request mitigation: quantize · distill · cache curation beats architecture "the model is the easy part" the actual bottleneck none of these are research bugs waiting to be fixed; they are properties of the current paradigm

The honest limits

Mitigations stack: retrieval to ground claims, citations so users can verify, output constrained to a schema, sampling several times and keeping the agreed answer. None eliminates hallucination. Given a fixed budget, more high-quality tokens often beat more parameters, and a few thousand careful fine-tuning examples often beat tens of thousands of mediocre ones.

Hallucination · Going deeper · 2 of 2

The four shapes of LLM evaluation — what counts as a correct answer Multiple choice Generation Preference Agentic MMLU · ARC · GPQA TriviaQA · HellaSwag pick A / B / C / D log-likelihood scoring cheap · automatable ceiling effects · saturated HumanEval · MBPP GSM8K · MATH · BIG-bench execute output · check vs reference / unit tests narrow but objective code & math · not chat LMSYS Arena · MT-Bench AlpacaEval · Chatbot Arena human or LLM picks A vs B side-by-side captures "feel" slow · biased · drifty SWE-bench τ-bench · WebArena end-to-end task success in env closest to product expensive · noisy no single benchmark predicts product quality · ship with your own task-specific eval benchmarks rank models · only your eval ranks them on your task

Four shapes of evaluation

Closed tasks can be scored for accuracy on held-out data. Open-ended tasks cannot. Public benchmark rankings correlate weakly with usefulness in a given product, and a model used as judge prefers length and its own style. Most teams combine a benchmark for regressions, a domain set written from real prompts, and an end-to-end harness.

Check yourself

what the model produced for the next wordscase name A0.31case name B0.27case name C0.22

A model cites a court case that does not exist, in perfect legal format. What best explains it?

Context and tools

Extend a frozen model with context, or with a loop of tools

Give it knowledge at question timequestionsearchyour documentsmodelanswers from themthe 5 – 20 closest passages are pasted into the promptfixes the cutoff date and private data, with no retrainingnot a memory: the next request starts from nothingLet it act in a loopthinknext stepactcall a toolobserveread the resultrepeat until an outside check says donechance of a flawless run when each step is 95% right5 steps77%20 steps36%50 steps8%Two ways to extend a frozen model: put the right text in front of it, or let it use tools in a loop.Both are engineering around the model, not changes to it, and both fail in ways engineers can debug: the wrong passageretrieved, an error compounding over steps. The next lesson builds both properly.

RAG puts the right passages in the prompt. An agent lets the model act, observe and repeat.

Context and tools · Going deeper · 1 of 2

A RAG pipeline — query-time retrieval grounds the LLM in private or fresh data User query Embed → vector ∈ ℝᵈ Vector search top-k nearest Prompt = system + query + top-k chunks "answer the question using only this context" LLM renders an answer Answer + citations Knowledge store chunk 1 chunk 2 chunk N vec₁ vec₂ vec_N ANN index vector search hits the ANN index, returns top-k chunks embeddings + vector DB do the heavy lifting · the LLM just renders the answer in fluent prose

A retrieval pipeline

Documents are cut into chunks of a few hundred tokens, each embedded once as a vector of 768 to 4,096 numbers and stored. At query time the question is embedded and the nearest chunks fetched. Chunk size, mixing keyword with vector search, and a reranker that scores question and passage together are the main levers. Failure shifts from “the model did not know” to “the system did not retrieve it”.

Context and tools · Going deeper · 2 of 2

An agent loop — think · act · observe · repeat LLM (think) plan · choose tool · or finish Tool / API (act) search · code · DB · HTTP Observation tool output → back into prompt Memory history · scratchpad · vector store act observe append read finish · return answer termination: max steps · explicit done · cost cap

The agent loop

The model emits a structured tool call, the host runs it and returns the output into the next prompt. Each round is a full model call over the whole history. Failure modes compound: long-horizon error, drifting beyond the task, wrong arguments, runaway cost. Completion must be checked against outside signals such as passing tests, not the model’s own report.

Summary

Machine learning is a bet that old patterns still hold

Dataquality beats architectureTrainingdownhill on the error, onceWeightsfrozen, with a cutoff dateInferenceone token at a timeAround the modelretrieved context, tools in a loop, and tests that check the outputStill unsolvedhallucination, evaluation, cost, data qualityMachine learning is a bet that patterns in old data still hold for new data. Checking that bet is the real work.Anchors: backpropagation (1986), “Attention Is All You Need” (2017), and for governance the NIST AI Risk ManagementFramework and the EU AI Act.

Data, training, frozen weights, inference, and the engineering around them. Next: AI engineering.

Summary · Going deeper

The intelligence pipeline — data, training, inference, and the augmentation patterns above it Data Train Eval Deploy collect · clean · label fit weights · GPU days held-out · benchmark ship weights · serve Inference new input → forward pass → prediction feedback Retrieval-augmented generation Agents embed query · vector search stuff top-k chunks into prompt think → call tool → observe loop until done The honest limits hallucination plausible-wrong evaluation gap no ground truth inference cost per-token compute data quality the real bottleneck a model is a function with billions of learned numbers — everything above is what you do with one

The intelligence pipeline

Training and inference are wildly asymmetric in cost, and most failures still come from the data, not the architecture. Standards here are mostly de facto: seminal papers, open formats such as ONNX, Safetensors and GGUF, model cards for documentation, and for governance the NIST AI RMF, the EU AI Act and ISO/IEC 42001.

Lesson complete

You finished “Intelligence”

Answer the questions along the way to see your score here.

Open in a new tab