Prompt engineering is the practice of structuring model inputs so an LLM reliably produces the output you need. In production it is less about clever phrasing and more about engineering discipline: clear instructions, well-delimited context, examples, a fixed output contract, and an evaluation set that tells you whether a change helped.
This page covers prompt structure, core techniques (few-shot, chain-of-thought, self-consistency), structured output, prompt injection defence, and an evaluation-driven workflow. Parent topic: LLM Engineering.
Most chat APIs separate a system prompt (stable instructions, role, constraints) from user turns (per-request content). Keep the system prompt stable to benefit from prompt caching, and put variable data in the user turn.
<document>, <email>) so the model can tell instructions from data.Zero-shot: instruction only. Works for well-known tasks on capable models. Few-shot: include examples; the model imitates their format and decision boundary. Choose diverse examples including edge cases, and vary them so the model does not copy surface details.
Chain-of-thought (CoT): ask the model to reason before answering (or use a model with built-in extended thinking). This improves multi-step arithmetic, logic, and planning, at the cost of latency and tokens. Ask for the reasoning inside <thinking> tags and the answer in <answer> tags so you can parse only the answer.
Self-consistency: sample several CoT paths at non-zero temperature and take the majority answer. Prompt chaining: split a complex task into sequential calls (extract → analyse → write), each with a focused prompt — easier to debug and evaluate than one giant prompt.
When code consumes the output, define a contract. Prefer native mechanisms — JSON mode, tool/function schemas, or constrained decoding — over "please return JSON". See Function Calling / Structured Output.
import json
from anthropic import Anthropic
client = Anthropic()
SYSTEM = """You classify customer support tickets.
Categories: billing, bug, feature_request, account, other.
Return only JSON: {"category": str, "urgency": 1-5, "summary": str (max 20 words)}.
Urgency 5 means data loss or full outage; 1 means a cosmetic question."""
FEW_SHOT = [
{"role": "user", "content": "<ticket>I was charged twice this month.</ticket>"},
{"role": "assistant", "content": '{"category": "billing", "urgency": 3, "summary": "Customer reports a duplicate charge this month."}'},
]
def classify(ticket: str) -> dict:
msg = client.messages.create(
model="claude-sonnet-4-5", # use your current model id
max_tokens=200,
temperature=0,
system=SYSTEM,
messages=FEW_SHOT + [{"role": "user", "content": f"<ticket>{ticket}</ticket>"}],
)
out = json.loads(msg.content[0].text)
assert out["category"] in {"billing", "bug", "feature_request", "account", "other"}
return out
print(classify("The export button crashes the app and I lost my report."))Always validate the parsed result (Pydantic or JSON Schema) and have a retry path that feeds the validation error back to the model.
Any text the model reads — web pages, emails, retrieved documents, tool outputs — can contain instructions. The model cannot reliably distinguish your instructions from injected ones, so treat prompting as a mitigation, not a security boundary.
See also OWASP Top 10 for LLMs.
Prompt changes without measurement are guesswork. Build a fixed evaluation set of real inputs with expected outputs or grading rubrics, score each prompt version (exact match, schema validity, an LLM-as-judge rubric, or human review), and only ship changes that improve the aggregate without regressing important slices.
Change one thing at a time, keep a changelog, and re-run the full set when you switch model versions — a prompt tuned for one model is not guaranteed to transfer. For RAG pipelines see Evaluation (RAGAS); for operations see LLMOps.
Order matters for cost and for quality. Providers that support prompt caching reuse computation for an identical prefix, so place stable material (system instructions, tool definitions, few-shot examples) first and per-request material last. With long documents, a common pattern that works well is to put the documents near the top of the user turn, each wrapped in tags with metadata such as source and date, and to put the question and output instructions after them. For very long inputs, asking the model to first quote the passages relevant to the question, then answer from those quotes, reduces answers built on material that is not in the documents and makes the result easy to verify.
Consider routing support emails into four queues. Version 1 is a single sentence: "Classify this email as billing, technical, account or other." On a 200-email labeled set it scores, say, a modest accuracy, and the confusion matrix shows the errors concentrate on password-reset emails, which humans label account but the model labels technical. Version 2 adds a one-line definition of each queue, including "password resets and login problems are account". Version 3 adds three short examples, one per frequently confused pair, and requires output as a JSON object with a queue field from a fixed enum. Each version is scored on the same set and the change is kept only if it improves the metric without regressing other classes. The specific numbers will vary by model; the method is the point: one change at a time, measured against a fixed set, with the confusion matrix telling you what to change next.
| Setting | Guidance |
|---|---|
| Temperature | Low (0 to 0.3) for extraction, classification and code; higher for brainstorming and varied phrasing. Note that temperature 0 reduces but does not always eliminate variation between runs. |
| Max output tokens | Set a ceiling that fits the expected answer plus headroom; truncated JSON is a common production failure. |
| Stop sequences | Stop at a closing tag to avoid paying for trailing text. |
| Reasoning / thinking budget | Enable for multi-step problems; disable for simple lookups where it adds latency without benefit. |
| Model tier | Prototype on a capable model to learn what is possible, then test whether a smaller, cheaper model passes the same evaluation set. |
Automated prompt optimization is also possible: frameworks such as DSPy treat instructions and few-shot examples as parameters and search over them against a metric, which works well once a reliable evaluation set exists.
Further reading. Brown et al. (2020), "Language Models are Few-Shot Learners"; Wei et al. (2022), "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models", arXiv:2201.11903; Wang et al. (2022), "Self-Consistency Improves Chain of Thought Reasoning"; Kojima et al. (2022), "Large Language Models are Zero-Shot Reasoners"; Yao et al. (2022), "ReAct: Synergizing Reasoning and Acting in Language Models"; Liu et al. (2023), "Lost in the Middle: How Language Models Use Long Contexts"; Greshake et al. (2023), "Not What You've Signed Up For" (indirect prompt injection); the Anthropic and OpenAI prompt engineering guides; OWASP Top 10 for LLMs.