Prompting Techniques

Chain-of-thought (CoT) prompting asks the model to lay out intermediate reasoning steps before its final answer, instead of jumping straight to a conclusion. Wei et al. showed this alone — with no change to the model’s weights — substantially improves accuracy on tasks that require multi-step arithmetic, logic or common-sense reasoning (see References). The classic form adds an instruction like "think step by step" (or a handful of worked examples that themselves show the reasoning) to a prompt that would otherwise ask only for the final answer.

Manual CoT prompting matters less with a reasoning model (Reasoning Models), because those models already generate an internal reasoning stream by default — adding "think step by step" on top of a model that already thinks adaptively is usually redundant, and can even conflict with the model’s own trained reasoning process. The practical rule: reach for explicit CoT instructions on a non-reasoning or budget-constrained model, and prefer the model’s own thinking/effort controls once a reasoning model is in play.

A minimal explicit-CoT prompt for DocsAssistant asks for the reasoning inside one tag and the final answer inside another, so the caller can log the reasoning and show only the answer:

See the chain-of-thought prompting paper for the technique, and the Anthropic prompting best practices — "Leverage thinking & interleaved thinking capabilities" for when to prefer the model’s own adaptive thinking instead.

Prompt chaining breaks one large, unreliable task into a sequence of smaller prompts, each handling one stage, with the output of one step feeding into the input of the next as a separate API call. This trades latency (multiple round trips instead of one) for reliability: each individual step is easier for the model to get right than the combined task would be in a single pass, and each step’s output can be logged, validated or branched on independently.

For DocsAssistant, a two-step chain separates extracting the facts relevant to a question from answering using only those facts — this keeps the answering step from having to also decide what in a long page is relevant, and makes the extracted facts independently inspectable (a wrong final answer is much easier to debug when you can see whether the extraction step already went wrong):

See the Anthropic prompting best practices — "Chain complex prompts" for this pattern, including the self-correction variant (generate a draft, review it against criteria, refine it) as the most common chaining shape.

Task decomposition is the same idea applied before any prompting happens at all: instead of chaining generic steps, the sequence of steps is itself designed around the task’s actual structure (for example, "find the relevant page" → "extract the relevant section" → "answer" → "cite the section" for DocsAssistant), so each prompt in the chain has a narrow, well-defined job rather than being a generic "do part of the work" instruction.

Sampling & Decoding already introduced self-consistency as a majority-vote variant of best-of-N: sample several independent completions at a temperature above 0 and take the most common final answer, instead of trusting any single sample. Wang et al. found this consistently outperforms plain greedy decoding on tasks with a single correct answer, precisely because CoT reasoning at a non-zero temperature can wander down a wrong reasoning path on any given sample, but a wrong path is unlikely to be the same wrong path across several independent samples (see References).

The following extends that idea into a small, complete voting loop: it samples N completions to a factual yes/no-style DocsAssistant question and returns the majority answer, along with the vote breakdown so a caller can see how close the vote was:

See the Wang et al. self-consistency paper (References) for the underlying method; the code above is an ordinary application of the messages.create call already covered on the Sampling & Decoding page, sampled N times.

Tree-of-thought (ToT) prompting generalizes chain-of-thought from a single linear reasoning path into a search over several candidate reasoning paths: at each step the model (or a controller loop around it) generates multiple possible "next thoughts", evaluates or scores each partial path, and expands only the most promising ones — backtracking away from a path that a later step judges unlikely to lead anywhere, much like a search algorithm exploring a tree of states rather than committing to the first branch taken. Yao et al. introduced this as a way to solve problems where a single reasoning chain is prone to getting stuck (planning puzzles, tasks with multiple valid-looking but ultimately dead-end approaches) by giving the model an explicit way to compare and prune alternative approaches instead of best-of-N’s simpler post-hoc pick-the-best-completion (see References). For DocsAssistant, a ToT-style approach could apply to answering an ambiguous question that plausibly matches several different pages: generate one candidate answer per plausible source page, have the model (or a lightweight scoring prompt) rate how well each is actually supported by its cited page, and keep only the best-supported candidate rather than committing to whichever page was retrieved first.

Because ToT needs the surrounding search/evaluation loop as much as it needs the prompt text, it is usually implemented as an explicit orchestration loop around several prompts rather than a single prompt — the loop itself is out of scope for this page, but the prompting piece is: each "generate a candidate" and "score a candidate" step is an ordinary, focused prompt of the kind covered elsewhere on this page (task decomposition for the generation step, self-critique below for the scoring step).

A minimal, one-level version of that loop for DocsAssistant generates one candidate per plausible page, scores each candidate against its page, and keeps only the best-supported one (a full ToT search would repeat the generate-and-score step at every level of the tree and prune low-scoring branches):

See the tree-of-thoughts paper (References) for the full search formulation, including breadth-first and depth-first expansion of the candidate tree.

Self-critique (also called reflection) asks the model to evaluate its own prior output against explicit criteria, then produce an improved version informed by that critique — typically as two or three separate calls: generate an answer, critique it (does it cite a real section, does it actually answer the question asked, is anything unsupported by the retrieved page), then revise based on the critique. Unlike self-consistency (which trusts a majority across otherwise-equal samples), self-critique relies on the model being a reasonably reliable grader of its own or another model’s output, which holds up well for checklist-style criteria (did it cite something, is it a complete sentence) and less well for judging its own factual accuracy on ambiguous questions, where a critique pass can just as easily "correct" a right answer into a wrong one.

For DocsAssistant, a lightweight reflection pass over a drafted answer might ask specifically: "Does this answer cite an actual heading from the source page? Does it avoid claiming anything the page doesn’t say?" — narrow, checkable questions the model can grade far more reliably than an open-ended "is this a good answer?" A minimal draft, critique and revise chain for that pass takes three separate calls:

See the Anthropic prompting best practices — "Chain complex prompts" for this self-correction chain, and the Self-Refine paper for the iterative feedback-and-refine method it is based on.

Metaprompting uses the model itself to draft or improve a prompt, rather than a human iterating on the prompt text by hand: given a task description (and, ideally, a handful of example inputs/outputs or failure cases), the model proposes a candidate prompt, which is then tested against real inputs like any other prompt. This is especially useful as a starting point when there is no first draft yet — Anthropic ships a metaprompt notebook in the Claude Cookbooks for exactly this (see References) — and as an iteration tool once a prompt is already failing on specific cases: feeding the model the failing input, the current prompt, and the wrong output, and asking it to propose a revised prompt, turns prompt debugging into another chained step rather than manual trial and error.

A minimal metaprompting call for that second use case sends the current prompt, a failing input and the wrong output, and asks for a revised prompt:

See Anthropic’s metaprompt notebook in the Claude Cookbooks for a complete prompt-generator metaprompt, and the automatic prompt engineer paper for model-driven prompt search as a research method.

Metaprompting still needs the same evaluation discipline as any other prompt change: a generated or revised prompt is a candidate, checked against a held-out set of real questions (Managing Prompts covers testing a prompt against a golden set) before it replaces a working prompt in production.

Role prompting assigns the model a persona or area of expertise, most naturally through the system parameter rather than folded into the first user message, since the system prompt applies to the whole conversation rather than reading as one more instruction embedded in the user’s question. A role does not grant the model new knowledge or capabilities it does not already have, but it does reliably shift tone, level of detail and default assumptions — a role like "an experienced technical writer reviewing internal documentation for accuracy" nudges DocsAssistant’s answers toward being terser and more willing to flag when a page’s wording is ambiguous, compared to a generic "helpful assistant" framing with no role at all.

See the Anthropic prompting best practices guide (References) for role prompting via the system parameter alongside the other general techniques it covers.

The same role prompt in Java uses the Spring AI 2.0 ChatClient fluent API, where .system(…​) sets the role and .user(…​) carries the question and its context:

See the Spring AI ChatClient reference for ChatClient.Builder, the prompt().system(…​).user(…​).call().content() chain and its other options.

Prefilling supplies the beginning of the assistant’s own turn (instead of only a user message) and has the model continue generating from that partial text, which is a direct way to force an output to start in a particular shape — for example, opening a JSON object so the model has no room to add introductory prose before it.

This technique has a real, current caveat that changes which model an example can target: as of 2026-09-28, Anthropic’s own migration guidance states that prefilled responses on the last assistant turn are no longer supported starting with the Claude 4.6 model generation onward — which includes claude-sonnet-5 — and a request that prefills the last assistant turn against one of those models returns a 400 error. Anthropic’s stated reasoning is that current models follow direct formatting instructions and Anthropic’s own Structured Outputs feature reliably enough that most of prefill’s former use cases (forcing JSON, skipping a preamble) no longer need it. The example below therefore targets claude-haiku-4-5, an earlier generation that still accepts prefill, and DocsAssistant’s own prompts should prefer explicit formatting instructions or structured outputs over prefill wherever the target model is claude-sonnet-5 or later:

See the Anthropic prompting best practices — "Migrating away from prefilled responses" (References) for the exact models affected and the recommended replacements (structured outputs, direct system-prompt instructions, or tool calling) on current-generation models.

ReAct (Reason + Act) interleaves reasoning steps with tool calls and their results, rather than doing all of a task’s reasoning up front and only then invoking a tool: the model reasons about what it needs, calls a tool to get it (a search, a lookup, a calculation), observes the result, and reasons again with that new information before either calling another tool or producing a final answer. Yao et al. showed that interleaving reasoning and acting this way reduces the compounding errors that come from committing to a full plan before any real information has come back from a tool (see References). For DocsAssistant, a ReAct-style loop reasons about which page a question is likely answered on, calls a page-lookup tool, reads what came back, and only then either answers or reasons that a different page needs to be looked up next — rather than guessing the single right page once and never revisiting that guess.

With the Messages API, a minimal ReAct loop gives the model a lookup_page tool, runs the tool whenever the response stops with stop_reason == "tool_use", sends the result back as a tool_result block, and repeats until the model answers:

See the Anthropic tool use overview for the tools parameter and the tool_use / tool_result loop, and the ReAct paper (References) for the reason-act-observe pattern itself.

ReAct as covered here is the prompting/reasoning pattern itself: an alternating reason-act-observe loop over one or more tools. How that loop is actually operated in practice — the agent runtime that drives the loop, tracks tool results across turns, and decides when to stop — is covered in more depth on the planned AI Agents sub-section rather than repeated here.

The following table gives one deliberately weak prompt fragment against one improved fragment for each technique above, all phrased against the same kind of DocsAssistant question: