Operational Governance

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

Written on 2026-10-01 against the tools in the version baseline (MLflow 3.16.1, Langfuse 4.49.0, LiteLLM 1.103.2, Prometheus 3.15.0). This page contains no code that needs running; the one configuration fragment is written from the documentation linked after it and was not run. The organisational practices are paraphrased from Aryan’s _LLMOps, Huyen’s AI Engineering and the Google SRE books (see the bibliography). It is general engineering guidance, not legal or compliance advice.

Every earlier page produces signals and controls: evaluation scores, gates, dashboards, budgets, registries. Governance decides who is accountable for each, what happens when one fires, and how you show afterwards that it happened.

Roles and metric owners

A metric without an owner is decoration. Name a person or team for each, and make the owner the default recipient of its alert. Roles are responsibilities, not job titles; in a small team one person holds several.

Role	Accountable for	Typical metrics owned
Product owner	What the assistant is for, out-of-scope uses, accepting residual risk.	Task success, user feedback, cost per outcome.
Application engineer	Prompts, retrieval and tools; keeping the evaluation gate green.	Golden-set scores, regression-run results, latency.
Platform or SRE	Serving, gateway, observability and on-call.	Availability, TTFT and TPOT, error rate, saturation (see Metrics, Dashboards and Alerting).
Data owner	Source documents, index freshness, personal-data handling (see Data Operations).	Index lag, delete propagation, retention compliance.
Evaluation owner	Golden sets, judge calibration, drift checks (see Drift and Regression Monitoring).	Judge agreement with humans, drift distances.
Model and cost owner	Catalogue, allow-list, budgets (see Prompt and Model Management and Cost Management).	Spend rate, cache hit rate, deprecation calendar.
Security and privacy contact	Threat modelling, abuse handling, legal requests.	Blocked-injection rate, access-control exceptions.

Role

Accountable for

Typical metrics owned

Product owner

What the assistant is for, out-of-scope uses, accepting residual risk.

Task success, user feedback, cost per outcome.

Application engineer

Prompts, retrieval and tools; keeping the evaluation gate green.

Golden-set scores, regression-run results, latency.

Platform or SRE

Serving, gateway, observability and on-call.

Availability, TTFT and TPOT, error rate, saturation (see Metrics, Dashboards and Alerting).

Data owner

Source documents, index freshness, personal-data handling (see Data Operations).

Index lag, delete propagation, retention compliance.

Evaluation owner

Golden sets, judge calibration, drift checks (see Drift and Regression Monitoring).

Judge agreement with humans, drift distances.

Model and cost owner

Catalogue, allow-list, budgets (see Prompt and Model Management and Cost Management).

Spend rate, cache hit rate, deprecation calendar.

Security and privacy contact

Threat modelling, abuse handling, legal requests.

Blocked-injection rate, access-control exceptions.

A short RACI table in the system card (see Prompt and Model Management) settles who approves a prompt change, who can pause the assistant, and who is called at night.

Runbooks

A runbook is the page an on-call engineer opens when an alert fires. Each alert in the dashboards and rules links to one. A useful runbook has the same fields every time: the symptom and the alert that signals it, the first checks (dashboards, trace queries), mitigations ordered by speed and safety, the criteria to escalate, the owner, and the date it was last exercised. Exercise them: a game day in which the provider key is revoked or the index is emptied finds more gaps than reading does.

Hallucination incident

A hallucination incident is a confirmed case where the assistant gave a wrong or fabricated answer that a user could act on, reported by a user, found by sampled judging, or raised by support.

Triage: pull the trace by id; record the question, the release tuple, the retrieved chunks and the answer. Classify as retrieval miss (the right content was not retrieved), grounding failure (retrieved but ignored or distorted), or knowledge gap (not in the corpus).
Contain: if the harm is high, switch the affected topic to a safe fallback ("see the documentation page"), disable the tool or route involved, or roll the prompt alias back (see Deployment and Release Patterns).
Fix: correct the source, retrieval setting or prompt through the normal pull request and gate.
Prevent: add the failing case to the golden set (see Building Eval Sets) so the gate catches a recurrence, and check how many other traces share the pattern.
Review: run a blameless postmortem and note whether detection came from monitoring or from a user.

Detection and correction techniques are covered in Hallucination Detection and Correction.

Provider outage

Symptom	Response
Provider returns errors or timeouts	The gateway retries with backoff, then falls back to the configured secondary model; confirm the fallback appears in traces and dashboards.
Fallback model is also degraded or more costly	Switch to cached or retrieval-only answers; announce the degraded mode; watch the spend rate.
Rate limiting (HTTP 429)	Lower concurrency, shed low-priority traffic (batch and evaluation jobs first), check quota with the provider.
Prompt registry or observability backend down	Application keeps serving from a client-side cached prompt version; telemetry is buffered or dropped, never blocking.
Model snapshot retired or behaviour changed	Follow the re-validation procedure in Drift and Regression Monitoring.

Symptom

Response

Provider returns errors or timeouts

The gateway retries with backoff, then falls back to the configured secondary model; confirm the fallback appears in traces and dashboards.

Fallback model is also degraded or more costly

Switch to cached or retrieval-only answers; announce the degraded mode; watch the spend rate.

Rate limiting (HTTP 429)

Lower concurrency, shed low-priority traffic (batch and evaluation jobs first), check quota with the provider.

Prompt registry or observability backend down

Application keeps serving from a client-side cached prompt version; telemetry is buffered or dropped, never blocking.

Model snapshot retired or behaviour changed

Follow the re-validation procedure in Drift and Regression Monitoring.

router_settings:
  num_retries: 2
  timeout: 30
  fallbacks:
    - docs-assistant: ["docs-assistant-backup"]

Source: LiteLLM: Fallbacks and reliability. The fallback only helps if it has been evaluated: a secondary model that was never run against the golden set is an untested change that activates during an incident. Serving-side failures (out-of-memory, saturated KV cache) are in Serving at Scale.

Severity and communication

Define severities by user impact, not by technical cause, for example: severity 1 for harmful or widely wrong answers or a full outage, severity 2 for a degraded mode or a single-tenant impact, severity 3 for a contained quality regression. Each level sets the response time, who is notified, and whether users or customers are told. Follow the incident-management structure (incident commander, communications, operations) described in the SRE book rather than inventing one.

Audit trail

An audit trail lets someone who was not there reconstruct what happened and who decided. It is assembled from records that earlier pages already create, plus access logs:

Question	Where the answer lives
Which release produced this answer?	Release tuple on the trace (prompt version, model snapshot, index version).
Who approved that release, on what evidence?	Change ticket or pull request linking the evaluation run and the approver.
Who changed a prompt alias or a gateway budget, and when?	Registry and gateway audit logs, exported to append-only storage.
Who looked at raw user traces?	Access logs of the observability platform.
What did the assistant know?	Index manifest and source revision (see Data Operations).
What changed after an incident?	Postmortem, golden-set diff and runbook revision.

Question

Where the answer lives

Which release produced this answer?

Release tuple on the trace (prompt version, model snapshot, index version).

Who approved that release, on what evidence?

Change ticket or pull request linking the evaluation run and the approver.

Who changed a prompt alias or a gateway budget, and when?

Registry and gateway audit logs, exported to append-only storage.

Who looked at raw user traces?

Access logs of the observability platform.

What did the assistant know?

Index manifest and source revision (see Data Operations).

What changed after an incident?

Postmortem, golden-set diff and runbook revision.

Keep audit records for the retention period your obligations require, protect them from edits, and keep personal data in them to the minimum. Applicable obligations vary by jurisdiction and use: the EU AI Act, data-protection law such as the GDPR, and sector rules can all attach documentation and logging requirements to AI systems, and risk frameworks such as the NIST AI Risk Management Framework offer a vocabulary for them. Which of these applies to you is a question for your legal and compliance colleagues. The AI Security & Responsible AI sub-section covers threat modelling, abuse and these regulations in depth (see NIST AI RMF and The EU AI Act); they are named here only so the audit trail is designed with them in mind.

Maturity self-assessment

The maturity levels on the landing page describe three steps from manual prompt editing to automated, continuously evaluated operation. Each level carries the same five supporting measures as well as the technical ones: documentation (a system card), KPIs, a risk register, cost tracking and API documentation, each more complete and more automated at the higher level. Use the checklist below to place a system honestly: a level is reached only when all its items are true and someone has seen them work.

Level	Checklist
0: Ad hoc	[ ] Prompts live in code, notebooks or a console. [ ] Quality is judged by eye. [ ] No traces. [ ] Spend is known only from the provider invoice. [ ] There is no written description of the system (no system card), no agreed KPI, and no list of risks. [ ] The API is whatever the client happens to call, without a written contract.
1: Managed	[ ] Prompts and models are versioned and each production model is pinned. [ ] The golden set gates every prompt, model and index change in CI. [ ] Traces and golden-signal metrics exist, with runbooks for the main alerts. [ ] The gateway enforces keys and budgets. [ ] A system card in the model-card style records intended use, models, data sources and known limits, and is updated with each release. [ ] KPIs (for example judged pass rate, time to first token, adoption) are defined with targets and shown on a dashboard. [ ] A risk register lists each known risk with an owner and a mitigation. [ ] Cost is tracked per model and per day against a budget. [ ] The assistant’s API is documented: request and response contract, error and degradation behaviour, and versioning.
2: Optimised	[ ] Sampled online judging runs and is calibrated against humans. [ ] Feedback flows into the golden set. [ ] Shadow, canary and A/B releases are routine and drift is monitored. [ ] Cost is attributed per tenant. [ ] Metric owners are named, an audit trail answers who approved what, and restore tests and game days run on a schedule. [ ] The system card is produced or checked by the release pipeline from the prompt and model registry, so it cannot drift from what is deployed. [ ] KPIs are tied to SLOs and to a business outcome, and are reviewed on a schedule. [ ] The risk register is reviewed after every incident and linked to the evaluation cases that test each risk. [ ] Cost per successful answer is tracked alongside cost per tenant. [ ] API documentation is versioned with the API, and a deprecation policy is published to consumers.

Level

Checklist

0: Ad hoc

[ ] Prompts live in code, notebooks or a console. [ ] Quality is judged by eye. [ ] No traces. [ ] Spend is known only from the provider invoice. [ ] There is no written description of the system (no system card), no agreed KPI, and no list of risks. [ ] The API is whatever the client happens to call, without a written contract.

1: Managed

[ ] Prompts and models are versioned and each production model is pinned. [ ] The golden set gates every prompt, model and index change in CI. [ ] Traces and golden-signal metrics exist, with runbooks for the main alerts. [ ] The gateway enforces keys and budgets. [ ] A system card in the model-card style records intended use, models, data sources and known limits, and is updated with each release. [ ] KPIs (for example judged pass rate, time to first token, adoption) are defined with targets and shown on a dashboard. [ ] A risk register lists each known risk with an owner and a mitigation. [ ] Cost is tracked per model and per day against a budget. [ ] The assistant’s API is documented: request and response contract, error and degradation behaviour, and versioning.

2: Optimised

[ ] Sampled online judging runs and is calibrated against humans. [ ] Feedback flows into the golden set. [ ] Shadow, canary and A/B releases are routine and drift is monitored. [ ] Cost is attributed per tenant. [ ] Metric owners are named, an audit trail answers who approved what, and restore tests and game days run on a schedule. [ ] The system card is produced or checked by the release pipeline from the prompt and model registry, so it cannot drift from what is deployed. [ ] KPIs are tied to SLOs and to a business outcome, and are reviewed on a schedule. [ ] The risk register is reviewed after every incident and linked to the evaluation cases that test each risk. [ ] Cost per successful answer is tracked alongside cost per tenant. [ ] API documentation is versioned with the API, and a deprecation policy is published to consumers.

Treat the result as a backlog, not a score: pick the cheapest unchecked item with the highest risk and schedule it. Revisit the list when the system, the team or the obligations change.