Context Engineering

Prompt engineering is the craft of writing a single, effective instruction: choosing wording, providing examples, specifying an output format, for one call whose input is essentially fixed. It optimizes a static piece of text.

Context engineering is broader and dynamic. It treats the entire contents of the context window at inference time — system instructions, tool definitions, retrieved documents, prior conversation turns, and the current user turn — as a limited, curated resource that must be assembled fresh, or incrementally revised, for every single call an agent makes over a long-running task. A prompt-engineered instruction can still fail if the context around it is wrong: stale retrieved documents, an overflowing conversation history that pushes the actual question out of the model’s effective attention, or tool definitions that eat the budget a retrieved answer needed. Prompt engineering asks "what should I ask the model to do?"; context engineering asks "what does the model need to see, right now, to do it well, and what do I leave out?" The distinction matters operationally: a single well-crafted prompt is written once and reused, while a context-engineering strategy runs on every turn of an agent’s loop, deciding anew what to include, what to compact, and what to discard as the interaction grows. The rest of this page covers the concrete techniques that make up that strategy.

Every request has a hard ceiling (the model’s context window, covered in detail here) and a much lower practical ceiling: past a certain point, filling the window with more tokens degrades response quality and latency even though the request would still technically fit. Context engineering starts by treating the window as a budget to be allocated across competing slots, not a bucket to be filled. A concrete allocation for DocsAssistant, against an assumed 128K-token window and an 8K generation reserve, might look like:

These numbers are illustrative starting points for one application, not fixed rules — the right split depends on the task (an agent that mostly reasons over a long document needs a larger retrieval slot; one that mostly holds a long back-and-forth needs a larger history slot), and should be tuned against real traffic rather than assumed.

The following function allocates a total input budget across slots and truncates each candidate’s content to fit, using a simple word-count stand-in for token counting (len(text.split())); a real deployment should use a proper tokenizer instead, as covered in Tokens & Tokenization:

See Anthropic — Effective context engineering for AI agents.

Rather than pre-loading everything an agent might conceivably need, just-in-time retrieval fetches content only when a step of the task actually calls for it, keeping the retrieved-documents slot small until it is needed and populated with what is relevant right now rather than a broad, stale snapshot taken at the start of a session. For DocsAssistant this means the retrieval call happens per user question, not once at conversation start: each turn’s retrieval query is built from the current question (plus, when useful, a short summary of the recent conversation), so the documents slot always reflects the immediate need instead of accumulating every page ever touched in the session. This is the same underlying pattern that Prompt Augmentation & Generation describes for retrieval-augmented generation — embedding a query, retrieving the top-matching chunks, and inserting them into the prompt — seen here as one specific instance of the general context-engineering problem of deciding, per turn, what belongs in the window. Just-in-time retrieval also composes with agentic tool use: instead of a single retrieval call, the agent can be given a search_docs tool it invokes explicitly whenever a step needs more information, so the retrieval decision (and its resulting token cost) is made by the model in context rather than paid on every turn regardless of need.

Even with just-in-time retrieval keeping the documents slot lean, the conversation-history slot grows monotonically as a session continues, and eventually exceeds its budget. Compaction addresses this by replacing older turns with a shorter summary once a threshold is crossed, rather than either dropping them outright (losing information the agent may still need) or letting the slot overflow (starving other slots or exceeding the window). A simple compaction step for DocsAssistant: once the history slot’s token count crosses its budget, call the model itself to summarize everything except the most recent few turns, then replace that older span with the summary in place:

See Anthropic Messages API reference.

Compaction discards detail by design; some information should instead be preserved verbatim across an arbitrarily long session, independent of what gets summarized away. Structured note-taking addresses this by giving the agent a small, durable scratchpad — a distinct slot from the conversation history — that it writes to explicitly (a running list of open questions, decisions already made, document sections already checked) rather than relying on the raw transcript to still contain that information after several rounds of compaction. For DocsAssistant, this might be a short running list such as "user is on version 3.x", "already checked the authentication page, no answer found", or "user prefers curl examples over SDK code" — facts that a plain chat-history summary might compress away but that materially change how later turns should behave. Because the notes slot is deliberately small and explicitly written (not accumulated automatically), it stays cheap and reliable even across a session long enough that the conversation history has been compacted several times over.

A minimal version keeps the notes as a small JSON object in application state, lets the model add to it through a save_note tool, and injects the current notes into the system prompt on every call:

See the "Structured note-taking" section of Anthropic — Effective context engineering for AI agents for the pattern.

A single agent’s context window is a shared resource across every sub-task it performs, which means a long or noisy sub-task (searching many documents, running many tool calls to narrow down an answer) can consume most of the budget before the main task is even addressed. Sub-agent isolation solves this by delegating a bounded sub-task to a separate agent invocation with its own, fresh context window, and returning only that sub-agent’s result — not its full working transcript — to the parent’s context. For DocsAssistant, a question that requires searching across many unrelated sections of the documentation could be delegated to a sub-agent whose own context fills up with search queries and candidate passages, while the parent agent’s context only ever receives that sub-agent’s final synthesized answer, keeping the parent’s own budget stable regardless of how much exploration the sub-task needed. This trades an extra model call (latency and cost) for keeping the parent’s context clean, which is worthwhile specifically when a sub-task’s process is large but its result is small.

In its simplest form the sub-agent is a separate call with its own fresh messages list, and only its final summary text is returned into the parent’s history (a real sub-agent would also run its own tool loop inside run_subagent, and all of that transcript stays local to the function):

See the "Sub-agent architectures" section of Anthropic — Effective context engineering for AI agents for when this trade-off pays off.

Progressive disclosure is the principle that an agent’s available capabilities should occupy context in proportion to how relevant they are right now, not how many exist in total: a short, always-loaded description of each capability, with its full instructions loaded into context only once that capability is actually triggered. Anthropic’s Agent Skills follow exactly this pattern: a skill’s name and description (roughly 100 tokens) stay resident so the model can decide whether it applies, while the skill’s full body — its actual step-by-step instructions, reference files, and scripts — is read into context only once the skill is triggered, and any bundled scripts run without their code ever entering context at all. This is what lets an agent have access to dozens of skills without a proportional context cost: unused skills cost only their metadata line, not their full content. The planned Customizing AI Workflows sub-section covers writing these skill definitions in more depth; for now, the relevant point is that progressive disclosure is context engineering applied to an agent’s own capability set, the same budgeting discipline as the rest of this page applied to "what instructions does the model see" rather than "what documents does it see." The same discipline applies to writing the instruction files themselves — keeping a CLAUDE.md or SKILL.md direct, structured, and scoped to what a given task actually needs rather than exhaustive — which the ai-catalog project’s guidance on authoring effective instructions covers in more detail (Authoring effective instructions).

The skill format makes the split concrete: only the YAML frontmatter of a SKILL.md file is loaded up front, and the Markdown body below it is read only once the description matches the task:

See the Anthropic Agent Skills documentation for the SKILL.md format and its levels of progressive loading.

Where content sits within the assembled context, not just whether it fits the budget, measurably affects how well a model uses it. Liu et al. found that model performance on long-context tasks is markedly better when the relevant information sits at the very start or the very end of the context than when it is buried in the middle — a primacy effect (content early in context) and a recency effect (content near the end, closest to the actual question) both outperform the middle, with accuracy degrading the deeper into the middle of a long context the needed information sits, a pattern commonly called "lost in the middle" (see References). For DocsAssistant this has direct, practical consequences for how the assembled context above should be ordered, not just sized: retrieved documents should be ranked so the most relevant passages sit closest to the actual user question rather than first-retrieved-first-placed; a compacted summary of older conversation should sit before, not interleaved with, the most recent (and most relevant) turns; and system instructions that must be followed reliably benefit from reinforcement near the end of the context, close to where generation begins, rather than relying solely on their position at the very start. Ordering is a free lever in the sense that it costs no extra tokens — it is worth tuning independently of, and in addition to, the budget allocation covered earlier on this page.

A small assembly function applies these rules: summary first, retrieved chunks sorted so the highest-scoring chunk sits last, then the recent turns, a short reminder of the key instruction, and the question at the very end:

See the lost-in-the-middle paper (References) for the position effects this ordering works around.