Prompt anatomy

A reliable prompt has a task, context, an example and a format

RoleYou are a support analyst for an online bank.who the model should beTaskClassify the customer message below by topic and urgency.one clear instructionContextUrgent means money is missing or the account is locked.what it cannot guessExamplesMessage: "Card blocked abroad" → {"topic":"card","urgent":true}the pattern to copyInput<message>I was charged twice for rent</message>the data, fenced offOutput formatReply with JSON only: {"topic": string, "urgent": boolean}exactly what to returnA reliable prompt states the task, the context, an example and the exact output wanted.The model knows nothing about your situation except what is in the prompt. Say who it is, what to do, the facts it needs, what agood answer looks like, and the format. Fence the input with tags or quotes so instructions and data cannot be confused.Most poor answers come from a missing part, not from a weak model.

Role, task, context, examples, fenced input, output format.

Prompt anatomy · Going deeper

Prompting techniques and when to use each

TechniqueHowBest forZero-shotinstruction onlysimple, common tasksFew-shot2 to 5 worked examplesa format or style to copyChain of thoughtask for steps before the answermaths, logic, multi-stepPrompt chainingsplit a job into calls, output to inputlong or mixed tasksNegative promptsay what to avoidimages; unwanted contentSelf-consistencysample several answers, take the majorityhard reasoning, at more costTechniques are tools for specific gaps, not rituals to add to every prompt.Start with a clear zero-shot prompt. Add examples when the format drifts, steps when the reasoning fails, and chaining when oneprompt is doing too many jobs. Prompt templates keep the fixed parts in code and insert the variable input, so every call isbuilt the same way and can be versioned and tested.

Add examples, steps or chaining only where the plain prompt fails.

System prompt and messages

The system prompt sets the rules; messages carry the chat

{ "system": "You are a support analyst. Answer in under 50 words. Never give account numbers.", "messages": [ { "role": "user", "content": "Why was I charged twice?" }, { "role": "assistant", "content": "A pending hold ..." }, { "role": "user", "content": "When will it drop off?" } ] }system: rules for the whole conversation, set by the appuser: what the person typedassistant: the model’s earlier repliesthe newest user turn: what to answer nowthe whole list is sent on every requestThe system prompt sets rules for every turn; the messages carry the conversation.Apps send a list of messages with roles. The system prompt, written by the developer, sets persona, scope, tone and limits, and theuser cannot see or change it. Because the model is stateless, the app resends earlier turns each time; that is how it seems to remember.Field names differ slightly between providers; the shape is the same.

System for standing rules; user and assistant turns resent every time.

API differences · Going deeper

Same three parts in every chat API, under different field names

APISystem promptTurn rolesCap on output lengthAnthropic Messagestop-level "system"user, assistantmax_tokens (required)OpenAI Chat Completionsa message, role "system" or "developer"user, assistantmax_completion_tokensGoogle Gemini generateContent"system_instruction"user, modelgenerationConfig.maxOutputTokensAmazon Bedrock Conversetop-level "system" listuser, assistantinferenceConfig.maxTokensTool results differ too: OpenAI uses a message with role "tool"; Anthropic and Bedrock put a tool result block inside a user turn.Bedrock Converse gives one request shape for every model it hosts, so switching model is a change of model id.Every chat API has the same three parts; only the field names change.Standing rules, alternating turns and a cap on the reply are in all of them. The differences bite when you port code: a systemmessage left inside the turn list is rejected or ignored by an API that wants it at the top level, and the assistant role is calledmodel on Gemini. Keep the provider call behind one small function of your own, and the rest of the app does not care.

System prompt, turns and an output cap: check each API's field names when porting.

Few-shot prompting

Examples teach the format faster than any description

Zero-shotClassify the sentiment of this review:"Arrived late, but the quality is superb."Model:The sentiment is mixed. The customer isunhappy with delivery but pleased with ...Few-shot: three examples first"Broke in a week." → negative"Exactly as described." → positive"Fine, nothing special." → neutral"Arrived late, but the quality is superb." →Model:positiveExamples teach the format and the judgement faster than any description.Without examples the model answers the question its own way, in prose. Three labelled examples show the labels allowed, theoutput shape and how borderline cases are judged. Pick varied examples, including a hard one, and keep their format identical.One example is one-shot; several are few-shot.

Zero-shot: instruction only. Few-shot: a few worked examples first.

Step-by-step reasoning

Asking for the steps first improves multi-step answers

Pens are 3 for $2. Notebooks are $4, buy 3 get 1 free.What do 12 pens and 4 notebooks cost?Answer at once$24.✗treats the free notebook as a discount on all fourThink step by step, then answer12 pens = 4 groups of 3 → 4 × $2 = $8Buy 3 notebooks, get the 4th free → 3 × $4 = $12Total: $8 + $12 = $20Asking for the steps before the answer improves multi-step reasoning.A model writes one token at a time, so the reasoning it writes down becomes context for the answer that follows. Reasoning modelsdo this internally ("thinking" or "extended reasoning") and can be given a budget for it. It costs more tokens and time, so use itfor maths, logic and planning, not for simple lookups. For automated use, ask for the steps and the final answer in separate fields.

Write the reasoning first, then the answer. It costs tokens; use it where steps matter.

Structured output

When code reads the answer, ask for JSON against a schema

Schema the app sends{ "type": "object", "properties": { "topic": { "enum": ["card","payment","login"] }, "urgent": { "type": "boolean" } }, "required": ["topic","urgent"] }Model output{ "topic": "payment", "urgent": true }App validates✓valid JSON✓topic in the list✓both fields presenton failure: retry once withthe error, then flag for reviewStructured output modes (JSON schema, tool definitions) constrain decoding so the reply always parses; still validate the values.When code will read the answer, ask for JSON against a schema, and validate it.Free text breaks the program that parses it the first time the model adds "Sure! Here is the JSON:". A schema fixes the field names,types and allowed values. Most model APIs can enforce one, so the output always parses; your code still checks that the values makesense before acting on them.

A schema fixes fields and types. Validate before acting on the values.

Validate and retry · Going deeper

Five ways a JSON reply fails, and the loop that catches them

for attempt in range(2): reply = ask_model(prompt) try: data = json.loads(reply) jsonschema.validate(data, schema) break except (json.JSONDecodeError, jsonschema.ValidationError) as e: prompt += f"\nInvalid: {e}"else: flag_for_review(reply)Five replies that fail, and what the two libraries printSure! Here is the JSON: {"topic": ...prose around the JSONJSONDecodeError: Expecting value: line 1 column 1 (char 0){"topic": "payment", "urgent": trcut off at the token limitJSONDecodeError: Expecting value: line 1 column 32 (char 31){"topic": "billing", "urgent": true}a value outside the enumValidationError: 'billing' is not one of ['card', 'payment', 'login']{"topic": "payment", "urgent": "yes"}the wrong typeValidationError: 'yes' is not of type 'boolean'{"topic": "payment"}a missing fieldValidationError: 'urgent' is a required propertyParse, validate against the schema, feed the error back once, then hand off: never act on an unchecked reply.The schema is the one on the main page. The error text is specific enough to send back to the model as the retry instruction. Thesecond failure is different: the reply hit the output-token limit, so a retry fails the same way until the limit is raised. Aschema-enforced output mode removes the first, third, fourth and fifth, not the second, and never a valid but wrong value.

Parse, validate, retry once with the error, then flag for review.

Prompt failures

Most bad answers trace back to the prompt

FailureLooks likeFixVague task"Summarise this"say length, audience, focusMissing contextanswers from general knowledgepaste the facts, or use RAGConflicting rules"be brief" and "explain fully"one priority, statedFormat driftextra prose around the JSONschema mode, an exampleMade-up factsconfident, wrong detailsground in sources; allow "I don’t know"Instructions in datatext in a document gives ordersfence input; never trust it (lesson 13)One prompt, five jobsquality drops on eachchain smaller promptsMost bad answers trace back to a vague, conflicting or overloaded prompt.Read a failure as feedback on the prompt first. Is the instruction clear to a new colleague? Is the needed fact in the context? Doesone rule contradict another? Allowing the model to say it does not know is one of the cheapest ways to cut invented answers.

Fix the prompt first: clear task, needed context, one priority, a format.

Testing prompts

Version prompts like code and test every change

50 saved test messages with known correct labels, run against every prompt versionv1first draft32/50 = 64%v2+ output schema39/50 = 78%v3+ 3 examples, incl. a hard one45/50 = 90%v4+ "unsure" option for unclear cases47/50 = 94%Re-run the same set whenever the prompt or the model version changes. Illustrative figures.Treat a prompt like code: version it, and test every change against a fixed set of cases.Judging a prompt by a few tries in a chat window is how regressions ship. Keep a test set of real inputs with expected outputs, scoreeach version automatically, and keep what improves the score. Bedrock Prompt Management, Microsoft Foundry and Vertex AI storeversions; their evaluation tools score them. The same set catches silent changes when the provider updates a model.

A fixed test set, a score per version, keep what improves it.

Test set size · Going deeper

Fifty test cases cannot tell 90% from 94%

The same two scores, 90% and 94%, with the range each test set can actually support (95% interval)v3: 45 of 5079% to 96%v4: 47 of 5084% to 98%v3: 450 of 50087% to 92%v4: 470 of 50092% to 96%50 test cases500 test cases70%80%90%100%On 50 cases, two more right answers is not evidence that the prompt improved.At 50 cases the interval is about 14 points wide, so v3 and v4 overlap almost entirely: the gain could be luck. At 500 cases it is 4points wide and the two barely overlap. Small sets catch large regressions, which is still worth having. To choose between closeversions, grow the set, and compare the versions case by case: which inputs flipped tells you more than the total. Wilson intervals.

Put a confidence interval on every score; close versions need hundreds of cases.

Lesson complete

You finished “Prompting”

Nicely done.

Open in a new tab