Prompt and Model Management

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

Written on 2026-10-01 against MLflow 3.16.1, Langfuse 4.49.0 (Python SDK 4.16.0) and langsmith 0.14.2; the full list is the version baseline on the landing page. Every snippet was written from the official documentation linked after it and was not run: no registry was started and no server was called.

In an LLM application, behaviour lives in more places than code: the prompt text, the model identifier and its sampling parameters, the retrieval settings, the tool definitions and the judge configuration all change what users see. Treating them as configuration edited in a console is how a Friday-afternoon wording tweak becomes an incident. This page applies the discipline that Aryan’s LLMOps and Huyen’s AI Engineering recommend for versioned artefacts (see the bibliography) to prompts and models, using current tooling. Prompt authoring technique is in Managing Prompts; this page is about their lifecycle.

What to version together

A release of the DocsAssistant is a tuple, and any change to a component is a new release:

Component	Recorded as
Prompt template and variables	Registry name and immutable version number.
Model	Provider, exact snapshot identifier (never a moving alias in production), and sampling parameters.
Retrieval configuration	Index version, chunking parameters, embedding model, number of passages.
Tool and MCP definitions	Schema hash or version of each tool the model can call.
Judge and thresholds	Judge model, rubric version and the gate thresholds used to approve the release.

Component

Recorded as

Prompt template and variables

Registry name and immutable version number.

Model

Provider, exact snapshot identifier (never a moving alias in production), and sampling parameters.

Retrieval configuration

Index version, chunking parameters, embedding model, number of passages.

Tool and MCP definitions

Schema hash or version of each tool the model can call.

Judge and thresholds

Judge model, rubric version and the gate thresholds used to approve the release.

Recording the tuple on every trace (see Observability with OpenTelemetry) is what later lets you answer "which prompt produced this answer" and compare cohorts by release.

Prompt registries

A prompt registry stores prompt templates with immutable versions, a human-readable change note, and movable aliases or labels such as production and staging. Application code asks for the alias, so promoting a version is a pointer move, not a deployment. The three tools used in this guide differ mainly in where they sit.

MLflow Prompt Registry LangSmith prompts Langfuse prompt management

	MLflow Prompt Registry	LangSmith prompts	Langfuse prompt management
Versions and aliases	Immutable versions and aliases on registered prompts.	Commits with tags on prompts in the hub.	Immutable versions with labels such as `production`.
Fits best when	You already track experiments and models in MLflow.	You build with LangChain or LangGraph and evaluate in LangSmith.	You want prompts next to traces, with links from a trace to the prompt version used.
Self-hosting	Open source (Apache-2.0).	Hosted service, with a self-hosted option on enterprise plans per its docs.	Open source core that can be self-hosted.

Versions and aliases

Immutable versions and aliases on registered prompts.

Commits with tags on prompts in the hub.

Immutable versions with labels such as production.

Fits best when

You already track experiments and models in MLflow.

You build with LangChain or LangGraph and evaluate in LangSmith.

You want prompts next to traces, with links from a trace to the prompt version used.

Self-hosting

Open source (Apache-2.0).

Hosted service, with a self-hosted option on enterprise plans per its docs.

Open source core that can be self-hosted.

The table is a summary of each product’s documentation on the date above; check the current pages before choosing, because features and editions change.

import mlflow

prompt = mlflow.genai.register_prompt(
    name="docs-assistant-answer",
    template="You answer questions about {{product}} using only the context.\n\nContext:\n{{context}}\n\nQuestion: {{question}}",
    commit_message="Require citations for every claim",
)

mlflow.genai.set_prompt_alias("docs-assistant-answer", alias="staging", version=prompt.version)

loaded = mlflow.genai.load_prompt("prompts:/docs-assistant-answer@production")
text = loaded.format(product="DocsApp", context="...", question="How do I log in?")

Source: MLflow: Prompt Registry.

from langfuse import get_client

langfuse = get_client()

langfuse.create_prompt(
    name="docs-assistant-answer",
    type="text",
    prompt="You answer questions about {{product}} using only the context.\n\nContext:\n{{context}}\n\nQuestion: {{question}}",
    labels=["staging"],
    commit_message="Require citations for every claim",
)

prompt = langfuse.get_prompt("docs-assistant-answer", label="production")
compiled = prompt.compile(product="DocsApp", context="...", question="How do I log in?")

Source: Langfuse: Prompt management. LangSmith offers the same pull-by-tag workflow through its prompt hub; see LangSmith: Manage prompts programmatically. Both Langfuse and MLflow read credentials from the environment (for example LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, MLFLOW_TRACKING_URI), so no secret appears in the code. Registries cache prompts client-side; configure a fallback so a registry outage does not take the application down, and see the incident table in Operational Governance.

Versioning and approvals

A promotion path keeps the speed of editing in a console and the safety of a code review:

Edit creates a new immutable version; nothing in production moves.
Evaluate: the CI job runs the golden set against the candidate version and posts the scores on the pull request or change ticket (see Evaluation in CI).
Approve: a named reviewer who is not the author signs off, with the evaluation link attached.
Promote: move the staging alias, then canary and move production (see Deployment and Release Patterns).
Record: the registry keeps the change note; the change log records who approved and which eval run justified it.

Rollback is moving the alias back to the previous version, which is why versions are immutable and why old versions are never deleted while any release still references them. Two-person approval for prompt changes is proportional to the risk, so apply it to prompts that carry policy (refusals, safety wording, citations) first.

Model catalogue and allow-list

A model catalogue is the inventory of every model the organisation may call; an allow-list is the subset a given application is permitted to use. At minimum each catalogue entry records:

Field	Content
Identifier	Provider and exact snapshot or checkpoint hash.
Purpose and owner	What it is approved for and the named owner (see Operational Governance).
Licence and terms	Licence of open weights or the provider’s usage terms, including data-retention and training-on-inputs settings.
Data classification	The highest data class that may be sent to it, and the region where it runs.
Evaluation evidence	Link to the latest golden-set run and the date.
Lifecycle	Provider deprecation or retirement date, and the planned replacement.

Field

Content

Identifier

Provider and exact snapshot or checkpoint hash.

Purpose and owner

What it is approved for and the named owner (see Operational Governance).

Licence and terms

Licence of open weights or the provider’s usage terms, including data-retention and training-on-inputs settings.

Data classification

The highest data class that may be sent to it, and the region where it runs.

Evaluation evidence

Link to the latest golden-set run and the date.

Lifecycle

Provider deprecation or retirement date, and the planned replacement.

The enforcement point is the gateway: expose only catalogue entries as model names, and reject anything else. In LiteLLM the model_list is that allow-list, and per-key model restrictions narrow it per team (see LiteLLM: Virtual Keys). The cost view of the same inventory is in Cost Management, and the hosted-versus-open-weight decision for local models in Running LLMs Locally.

Reproducibility

Exact bit-for-bit reproduction of a sampled answer is usually impossible: hosted models may change hardware or kernels, and sampling at nonzero temperature is random by design. The achievable goal is reconstructibility: given a trace you can rebuild the exact input, and given a release tuple you can rerun the evaluation and get statistically comparable scores.

Store the fully rendered prompt (or its registry version and variables), model snapshot, parameters, retrieved chunk ids and tool results on each trace, subject to the privacy rules in Data Operations.
Set temperature to zero and pass a seed where the provider documents one, and say in the evaluation report that this reduces but does not eliminate variation.
Repeat evaluation runs and report a mean with an interval, as described in Evaluation Methodology, so that a difference smaller than run-to-run noise is not treated as a regression.
Pin dependencies (client libraries, tokenizers, embedding models) and the index version along with the prompt.

Model and system cards

A model card documents a model: intended use, training data at the level the provider discloses, evaluation results, limitations and licence. Providers and model hubs publish them for their models; you read them when adding an entry to the catalogue (see Hugging Face and On-Device Models for the model-hub format). A system card documents your application, which is what governance reviews actually need. A one-page system card for the DocsAssistant has these sections: purpose and out-of-scope uses, release tuple, data sources and their classification, evaluation results with dates and slices, known failure modes, human oversight and escalation, monitoring and owners, and the change log link. Keep it in the repository next to the prompts so a pull request that changes behaviour also changes the card. Regulatory obligations that may require such documentation are named in Operational Governance.