Managing Prompts

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

Examples on this page target Python 3.12 with Jinja2 3.x and pytest 8.x, DSPy 3.4.0 (the current PyPI release, checked 2026-09-28), and Spring AI 2.0’s PromptTemplate and ChatClient APIs, all checked against the vendors' own documentation as of 2026-09-28. Prompt Engineering Fundamentals and Prompting Techniques cover how to write a good prompt; this page covers what happens once that prompt has to live in a codebase, get changed by more than one person, and keep working as the model behind it changes — using the running DocsAssistant example (a retrieval-augmented support assistant introduced on the vector-database pages) as the concrete scenario throughout.

Prompts as code

A prompt that only exists as a string literal inside application code has the same problems any hard-coded, unreviewed, untested piece of logic has: nobody can see what changed between two versions, nobody can roll back a regression, and nobody can reuse the same prompt from two different call sites without copy-pasting it. Treating prompts as code addresses this with the same tools already used for the rest of the codebase:

  • Templates. The literal instruction text is separated from the variables that change per request (the user’s question, retrieved context chunks, conversation history) so the fixed part can be reviewed, diffed and tested independently of the data it is filled with. Jinja2 templates, Python’s own string.Template, and framework- specific template classes (Spring AI’s PromptTemplate, covered below) all serve this role.

  • Versioning. Prompt templates live as files in version control, next to the code that renders them, so a prompt change is an ordinary reviewed pull request with a diff, a commit message, and a git history — not an edit nobody can trace. A prompt registry (below) can additionally track versions that were promoted to production independently of the git history, which matters when a prompt is edited through a UI rather than a code change. Tool definitions (each tool’s name, description and input schema) are prompt text too, since the model reads them on every call, so they belong in the same versioned files and the same golden-set tests as the instruction templates rather than being edited inline in application code.

  • A prompt library. As a project accumulates prompts — one per intent, one per few-shot variant, one per language — a flat pile of ad hoc strings becomes its own maintenance problem. A prompt library is simply a conventional place (a prompts/ directory, one file per prompt, a consistent naming scheme) so a new prompt is added the same way every time and an existing one is easy to find before writing a near-duplicate.

  • Prompt registries. Beyond a library of files, some teams run a prompt registry: a small service or database that stores named, versioned prompts with metadata (which model they were validated against, who approved the latest edit, which environment each version is deployed to) and serves the current production version to the application at runtime. This decouples "ship a new prompt version" from "ship a new application deploy," at the cost of an extra moving part and an extra place state can drift from what is in git — worth it once enough people are editing prompts that a code review and a deploy would otherwise be needed for every wording tweak, not worth it for a single-developer project with a handful of prompts.

Testing prompts: golden sets and regression tests

A prompt is a piece of logic like any other, so it deserves the same discipline: a change that looks like an improvement for the case it was written for can silently regress a case nobody re-checked. The standard technique is a golden set: a fixed collection of representative inputs together with the properties their outputs must satisfy (contains a citation, refuses a specific request, extracts a field correctly, stays under a length limit). Running the current prompt against the golden set after every edit turns "does this still work" from a vague impression into a pass/fail check that can run in CI, the same way a unit test suite catches a regression in ordinary code. The DocsAssistant example below builds the simplest version of this: rendering the prompt template with fixed inputs and asserting the rendered text still contains what the golden set expects, without requiring a live model call for every CI run.

A golden set is necessarily narrow — it only catches regressions on the exact cases it contains, and it says nothing about how a prompt performs on the long tail of real user input. Turning that into a systematic evaluation practice — structured test suites run against live model calls, scoring rubrics, and tracking a prompt’s quality over time rather than a fixed pass/fail set — is covered in depth in LLMOps: Evaluation Methodology and LLMOps: Prompt & Model Management; this page stops at the golden-set regression test that belongs directly alongside the prompt template in the same repository.

DocsAssistant: a Jinja2 template with a pytest golden-set test

The template below is DocsAssistant’s answer-generation prompt, kept as its own file rather than an inline string so it can be diffed, reviewed and tested on its own:

prompts/answer_question.j2
You are DocsAssistant, a support assistant that answers questions using only the documentation
excerpts provided below. If the excerpts do not contain the answer, say so instead of guessing.

<excerpts>
{% for chunk in context_chunks %}
<excerpt source="{{ chunk.source }}">
{{ chunk.text }}
</excerpt>
{% endfor %}
</excerpts>

Question: {{ question }}

Answer in {{ max_sentences }} sentences or fewer, and cite the source of each excerpt you use.

The rendering code loads that file and fills it with per-request variables:

from jinja2 import Environment, FileSystemLoader

env = Environment(loader=FileSystemLoader("prompts"), trim_blocks=True, lstrip_blocks=True)
template = env.get_template("answer_question.j2")

def render_answer_prompt(question: str, context_chunks: list[dict], max_sentences: int = 3) -> str:
    return template.render(
        question=question,
        context_chunks=context_chunks,
        max_sentences=max_sentences,
    )

The golden-set test does not call a model at all — it checks that rendering still produces the instructions and placeholders the rest of the system depends on, so a template edit that accidentally drops the citation instruction or the excerpt loop fails immediately in CI:

import pytest

GOLDEN_CASES = [
    {
        "question": "How do I reset my password?",
        "context_chunks": [{"source": "account-faq.adoc", "text": "Use the 'Forgot password' link."}],
        "max_sentences": 2,
    },
    {
        "question": "What does ef_search control?",
        "context_chunks": [{"source": "vector-rag/hnsw.adoc", "text": "ef_search trades recall for latency."}],
        "max_sentences": 3,
    },
]

@pytest.mark.parametrize("case", GOLDEN_CASES)
def test_rendered_prompt_keeps_required_instructions(case):
    rendered = render_answer_prompt(**case)

    assert "only the documentation excerpts" in rendered
    assert "say so instead of guessing" in rendered
    assert f"Answer in {case['max_sentences']} sentences or fewer" in rendered
    for chunk in case["context_chunks"]:
        assert chunk["source"] in rendered
        assert chunk["text"] in rendered
    assert case["question"] in rendered

A version built on the standard library instead of Jinja2 replaces the template file with a .txt file read into a string.Template and $placeholder substitutions; the golden-set test above is unchanged either way, since it only ever inspects the rendered string, not how it was rendered. See the Jinja2 API documentation for Environment, FileSystemLoader and Template.render, and the Python string.Template documentation for the standard-library alternative.

Provider prompt improvers and generators

Both major hosted providers ship tooling that turns prompt writing into an assisted, iterative task rather than a purely manual one:

  • Anthropic offers a prompt generator and a prompt improver in the Claude Console (https://platform.claude.com). The generator takes a plain description of a task and produces a starting template — typically structured with XML tags, a chain-of-thought section, and variable placeholders for the parts that change per call — following the same techniques Prompt Engineering Fundamentals covers by hand. The improver takes an existing template, including its variables, and strengthens it automatically: adding chain-of-thought reasoning and standardizing and enriching its examples, while keeping the same variables so it drops into the same call site. If an improved prompt comes back with a prefilled assistant turn, drop it before targeting claude-sonnet-5 or another Claude 4.6+ model, where prefilling the last assistant turn returns a 400 error (see Prompting Techniques — Prefilling). As of 2026-09-28, Anthropic’s former prompt-generator and prompt-improver documentation pages under platform.claude.com redirect to the general prompting best practices guide, which does not document either tool, so check the Console itself for their current workflow.

  • OpenAI offers a Prompt Optimizer: a chat interface in the OpenAI dashboard that takes a prompt, refines it according to current prompting best practices, and can also use a dataset’s annotations and graders to target the prompt’s actual failures. As of 2026-09-28 that dataset-backed mode is being deprecated together with the Evals platform it runs on: Evals becomes read-only for existing users on 2026-10-31 and is scheduled to shut down on 2026-11-30 (see the OpenAI Prompt Optimizer guide), so do not build a long-term workflow on the dataset-backed optimizer.

These tools are a good way to get from a blank page to a reasonable first draft, or to try one more optimisation pass on a prompt that has stalled — but the golden-set test above is what actually decides whether an improver’s suggested rewrite is worth keeping, since none of these tools sees DocsAssistant’s own regression cases.

DSPy-style optimisation

Rather than hand-editing a prompt’s wording and re-testing by feel, DSPy takes a programming rather than prompting stance: a task is declared as a typed signature (its inputs and outputs), and one of DSPy’s modules (Predict, and others built on it) turns that signature into an actual prompt sent to the configured model, without the caller writing or tuning that prompt text by hand. DSPy separates the task’s declaration from its prompt so that the framework — and DSPy’s own optimisers, which are out of scope for the minimal example below — can search over different prompt phrasings, few-shot examples or intermediate reasoning steps for that signature automatically, using a metric on a training set instead of a human tweaking wording.

The minimal DocsAssistant example below declares a signature for the same answer-generation task as the Jinja2 template above, then runs it once with dspy.Predict. This is the current dspy.Signature / dspy.Predict shape as of DSPy 3.4.0 (verified against the PyPI release index on 2026-09-28); DSPy’s public API has changed across major versions before, so re-check the installed version’s own docs before relying on this shape unchanged:

import dspy

lm = dspy.LM("anthropic/claude-sonnet-5")
dspy.configure(lm=lm)

class AnswerFromExcerpts(dspy.Signature):
    """Answer a support question using only the given documentation excerpts."""

    question: str = dspy.InputField()
    excerpts: str = dspy.InputField(desc="concatenated documentation excerpts, one per source")
    answer: str = dspy.OutputField(desc="a concise answer that cites which excerpt it used")

answer_question = dspy.Predict(AnswerFromExcerpts)

result = answer_question(
    question="How do I reset my password?",
    excerpts="[account-faq.adoc] Use the 'Forgot password' link.",
)
print(result.answer)

DSPy’s LM reads the underlying provider credentials the same way the vendor SDK it wraps does (for example ANTHROPIC_API_KEY from the environment for an anthropic/…​ model string), so nothing provider-specific needs to change in the signature or module code above to point it at a different backend. See the DSPy documentation for the current Signature, InputField/OutputField and Predict API.

Prompt portability across models

A prompt tuned against one vendor’s model does not automatically perform the same way against another, even when the wording is left unchanged, because several things a prompt implicitly depends on differ per vendor and per model generation:

  • Formatting conventions. Anthropic’s own prompting guidance favors XML tags to delimit sections (instructions, examples, retrieved context); OpenAI’s guidance leans more on Markdown headings and delimiters. A prompt moved from one to the other keeps working in the sense that the model still reads the text, but the structural cues it was tuned to notice may land less reliably.

  • System-role handling. Where the "system" instructions live and how strongly they are enforced relative to the rest of the conversation differs by vendor and has changed across model generations on both major APIs; Prompt Engineering Fundamentals covers the current shape of the system/developer role on each API in detail. A prompt that leans on one vendor’s specific system-role semantics needs that section re-checked, not just copy-pasted, when it moves to another vendor.

  • Token limits. Context-window size and output-token caps differ by model and change between generations (Context Windows & Memory covers this in depth), so a prompt built around "always include the last 20 turns of history" or "always include these five retrieved chunks" can silently overflow a smaller model’s window even though it fit comfortably in the one it was written for.

  • Parameter differences. Sampling and control parameters are not named or scoped identically across vendors — stop_sequences versus stop, max_tokens versus max_completion_tokens, and reasoning models on both major APIs that restrict or remove ordinary sampling parameters entirely (Sampling & Decoding documents the current parameter names and restrictions on each vendor). A prompt-and-parameters pair moved to another vendor or another model generation needs both halves re-verified, not just the prompt text.

In practice, "portable" is a spectrum rather than a binary: a well-structured prompt with clearly labeled sections, explicit instructions rather than implicit conventions, and its sampling parameters kept separate from its wording moves between vendors with less rework than one that leans on a specific vendor’s formatting habits — but the golden-set regression test from earlier in this page is what actually confirms a ported prompt still behaves as intended on the new target, rather than assuming structural similarity is enough.

Spring AI: PromptTemplate loaded from a .st resource

Spring AI’s own PromptTemplate is a JVM-side equivalent of the Jinja2 template above: the instruction text lives in its own resource file (a StringTemplate .st file) rather than as a Java string literal, which keeps it just as reviewable and testable as the Python version:

src/main/resources/prompts/answer-question.st
You are DocsAssistant, a support assistant that answers questions using only the documentation
excerpts provided below. If the excerpts do not contain the answer, say so instead of guessing.

Excerpts:
{excerpts}

Question: {question}

Answer in {maxSentences} sentences or fewer, and cite the source of each excerpt you use.
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.ai.chat.prompt.PromptTemplate;
import org.springframework.core.io.Resource;

import java.util.Map;

public class AnswerService {

    private final PromptTemplate template;
    private final ChatClient chatClient;

    public AnswerService(Resource answerQuestionResource, ChatClient.Builder chatClientBuilder) {
        this.template = new PromptTemplate(answerQuestionResource);
        this.chatClient = chatClientBuilder.build();
    }

    public Prompt buildPrompt(String question, String excerpts, int maxSentences) {
        return template.create(Map.of(
                "question", question,
                "excerpts", excerpts,
                "maxSentences", maxSentences
        ));
    }

    public String answer(String question, String excerpts, int maxSentences) {
        return chatClient.prompt(buildPrompt(question, excerpts, maxSentences))
                .call()
                .content();
    }
}

answerQuestionResource is injected as a @Value("classpath:prompts/answer-question.st") Resource, which is what keeps the prompt text a versioned resource file rather than an inline Java string; buildPrompt stays separate from answer so a golden-set test can check the rendered Prompt without a model call. See the Spring AI prompts reference for PromptTemplate, its Resource-based constructors, and the StringTemplate renderer it uses by default, and the Spring AI ChatClient reference for sending a ready-made Prompt with chatClient.prompt(prompt).call().content().

Related pages

References