LLMSecOps

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

Written on 2026-10-01 against the OpenTelemetry GenAI semantic conventions (status: Development, so attribute names can still change), Microsoft Presidio 2.2.364, the OWASP Top 10 for LLM Applications 2026 and NIST AI 600-1 (July 2024); see the version baseline. The code is illustrative, was written from the official documentation and was not run against a live DocsAssistant; no measurements or incident results are reported on this page.

The earlier pages of this section describe individual risks and controls. LLMSecOps is the practice of keeping those controls true after release: models, prompts, retrieval indexes and tools keep changing, and so does the attack surface. The term borrows from DevSecOps and sits on top of the operational practices in the LLMOps section. This page is organised around the lifecycle, the audit that checks it, the telemetry that proves it, and the response when it fails.

Security across the LLMOps lifecycle

Each LLMOps stage adds a distinct security duty. The diagram shows the loop; the table maps each stage to the page that covers the control in depth.

flowchart LR D[Data and index
preparation] --> B[Build and evaluate
prompts, models, tools] B --> R[Release
and deploy] R --> O[Operate
serve and observe] O --> M[Monitor
detect and alert] M --> I[Respond
and remediate] I --> D M -. regression cases .-> B
Stage Security duty Where it is covered

Data and index preparation

Classify sources, strip secrets and personal data before indexing, record provenance, review for poisoning and bias.

RAG and Vector Security, Data Operations

Build and evaluate

Treat prompts and model choices as code, run adversarial and regression suites on every change.

AI Red Teaming, Evaluation in CI

Release and deploy

Verify third-party models and dependencies, gate rollouts, keep a rollback path.

Supply Chain Security, Deployment and Release Patterns

Operate

Enforce guardrails, limits and least privilege at runtime; manage credentials outside code.

Guardrails, Unbounded Consumption, Cost Management

Monitor

Emit security-relevant telemetry, alert on abuse patterns and quality drift.

Observability with OpenTelemetry, Metrics, Dashboards and Alerting, Drift and Regression Monitoring

Respond and remediate

Contain, investigate, fix and feed the case back into the evaluation set.

This page, Operational Governance

Two feedback arrows matter more than the forward flow. A production incident must become a regression case in the build stage, and a drift alert must be able to trigger the same review as a code change would. Without them the lifecycle is a pipeline rather than a loop.

The audit

Aryan’s treatment of LLMOps security (credited in the bibliography) frames a periodic audit of an LLM application as an ordered sequence of steps. The version below is a paraphrase adapted to the DocsAssistant; the order is meaningful because each step consumes the output of the previous one.

Step Question DocsAssistant output

1. Scope

Which models, prompts, indexes, tools, data stores and third parties are in the audit boundary?

An inventory: chat model and version, search_docs and open_issue, the documentation index, the logging pipeline.

2. Threat model

Who can attack it, through which channels, and what would hurt?

The data-flow diagram from Trust Boundaries and Threat Modeling, refreshed.

3. Controls

Which controls exist for each threat, and is each one actually enforced?

A control-to-risk matrix against the OWASP LLM list, with owners.

4. Red team

Do the controls hold against realistic attacks?

Scripted and manual attack runs (see AI Red Teaming); findings with reproduction steps.

5. Data review

What do training, fine-tuning, retrieval and log data contain, and who can read them?

Sampling of indexed pages and stored conversations for secrets, personal data and planted instructions.

6. Bias

Do outputs differ unfairly across user groups or topics?

Slice-based evaluation results and reviewed examples (see Evaluation Methodology).

7. Document

Is the evidence written down in a form an auditor or successor can use?

Findings register, accepted risks with owners and dates, updated model and system documentation.

8. Monitor

Which signals will reveal a control failing between audits?

Alerts and dashboards described below; coverage gaps listed explicitly.

9. Remediate

Who fixes what by when, and how is the fix verified?

Tickets with deadlines, retest of each finding, new regression cases.

Run the audit at least on every major change (new model, new tool, new data source) and on a fixed cadence otherwise. The NIST AI RMF Measure and Manage functions are a convenient way to present the same evidence to a governance board.

Logging prompts and responses

Prompts and responses are the most useful security evidence an LLM application produces and also its most sensitive data: users paste personal data, and system prompts and retrieved documents may contain confidential material. The balance is a design decision, not a default.

Decision Guidance

What to record always

Metadata that is rarely sensitive: request identifier, session and tenant identifiers (pseudonymous), model and version, prompt template version, tool names called, token counts, latency, guardrail verdicts, finish reason.

What to record conditionally

Full prompt and response text, tool arguments and results. Capture on a sampled basis, on flagged requests only, or in a separate restricted store.

Redaction

Detect and mask personal data and secrets before text leaves the process, then store the masked form. Detectors are probabilistic, so redaction reduces risk but does not remove it.

Retention

Short, written, enforced by the storage layer (time-to-live or lifecycle rules). Keep security-event metadata longer than content. Align periods with the applicable privacy law and contract.

Access

Content stores readable by a small named group, with access itself logged; developers debugging from metadata should not need content.

Integrity

Ship to append-only or write-once storage so an attacker who compromises the application cannot erase the trail.

Redaction with Presidio

Microsoft Presidio (2.2.364 at the time of writing) provides an analyzer that finds personal-data entities and an anonymizer that replaces them. The sketch redacts text before it is logged.

from presidio_analyzer import AnalyzerEngine
from presidio_anonymizer import AnonymizerEngine

analyzer = AnalyzerEngine()
anonymizer = AnonymizerEngine()


def redact(text: str) -> str:
    findings = analyzer.analyze(text=text, language="en")
    return anonymizer.anonymize(text=text, analyzer_results=findings).text


logged_prompt = redact(user_prompt)

Derived from the Presidio documentation: https://microsoft.github.io/presidio/getting_started/. The default recognisers target English and common entity types; add custom recognisers for internal identifiers and evaluate recall on your own data before relying on the result. The privacy side of retrieval is discussed in Access Control and Privacy, and what a model may reveal in Sensitive Information and Hidden Context.

SIEM integration

A security information and event management (SIEM) system correlates events from many sources. To make an AI application visible to it, emit events that are structured, consistently named and correlated with the rest of the platform’s traces.

OpenTelemetry GenAI conventions

OpenTelemetry defines semantic conventions for generative AI spans, metrics and events. Using them means the same attribute names work across vendors and the collector can route and filter without custom parsing. Attributes of particular security interest are the operation name, the provider and model requested and returned, token usage, and the error type. The conventions are in Development status, so pin the version in use, and message content capture is opt-in precisely because of the privacy considerations above. The wider tracing setup is covered in Observability with OpenTelemetry.

from opentelemetry import trace

tracer = trace.get_tracer("docsassistant")

with tracer.start_as_current_span("chat docs-model") as span:
    span.set_attribute("gen_ai.operation.name", "chat")
    span.set_attribute("gen_ai.request.model", "docs-model")
    # ... call the model, then record usage and verdicts
    span.set_attribute("gen_ai.usage.input_tokens", input_tokens)
    span.set_attribute("gen_ai.usage.output_tokens", output_tokens)

Derived from the OpenTelemetry GenAI span conventions: https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-spans/ and the Python API at https://opentelemetry.io/docs/languages/python/instrumentation/.

Tracer tracer = openTelemetry.getTracer("docsassistant");

Span span = tracer.spanBuilder("chat docs-model").startSpan();
try (Scope scope = span.makeCurrent()) {
    span.setAttribute("gen_ai.operation.name", "chat");
    span.setAttribute("gen_ai.request.model", "docs-model");
    // ... call the model, then record usage and verdicts
    span.setAttribute("gen_ai.usage.input_tokens", inputTokens);
    span.setAttribute("gen_ai.usage.output_tokens", outputTokens);
} finally {
    span.end();
}

Derived from the OpenTelemetry Java manual instrumentation guide: https://opentelemetry.io/docs/languages/java/instrumentation/. Custom security attributes (for example a guardrail verdict) should use an application-specific namespace so they do not collide with the standard ones.

Structured security events

Spans describe what happened; the SIEM also needs discrete, filterable events. Emit one JSON line or log record per security-relevant decision, with a stable schema and the trace identifier for correlation.

Event Typical fields

Guardrail block or flag

Rule identifier, direction (input or output), action taken, trace identifier. No raw content by default.

Tool call denied or approved

Tool name, caller identity, policy decision, approver where applicable.

Limit exceeded

Limit type (rate, tokens, tool calls), tenant, value, threshold.

Authentication or authorisation failure

Principal, resource, reason code.

Suspected injection or exfiltration

Detector name, score, source channel (user, retrieval, tool result), trace identifier.

Configuration change

What changed (prompt version, model, tool set, guardrail rule), who, when.

Typical detections built on these events include repeated guardrail blocks from one principal, a tool call that follows a retrieval flagged as suspicious, token consumption far above a tenant’s baseline (the cost side is in Unbounded Consumption and Denial of Wallet), and a sudden change in refusal rate. Alert design and routing are covered in Metrics, Dashboards and Alerting, and slower quality shifts in Drift and Regression Monitoring.

AI incident response

AI incidents follow the usual response phases, but containment and evidence differ: the "vulnerability" may be a prompt, an index entry or a model version rather than a code defect, and the behaviour may not reproduce deterministically. Record the exact model version, prompt template, retrieved context and tool results for every incident so it can be replayed.

Incident type Contain Investigate Recover and prevent

Prompt injection with tool misuse

Disable or require approval for the affected write tool; revoke tokens it used.

Trace the originating text (user, document, tool result); list every action taken.

Add the case to the red-team suite; tighten tool scopes and approval rules.

Sensitive data in responses or logs

Stop the leaking path (feature flag, guardrail rule); restrict log access.

Determine source (index, system prompt, conversation memory), affected users and period.

Purge data, rotate exposed secrets, notify per legal obligations, fix ingestion filters.

Poisoned or malicious indexed content

Remove or quarantine the documents; pause re-indexing.

Identify who added them and which answers cited them.

Add provenance and review gates; re-embed affected items.

Compromised model or dependency

Roll back to the previous pinned version; block the artefact.

Compare hashes and signatures; review who pulled it and when.

Verify artefacts before use (see Supply Chain Security).

Cost spike or denial of wallet

Apply stricter rate and token limits; suspend abusive principals.

Break down usage by tenant, route and prompt length.

Enforce budgets and per-request caps; alert on anomalies.

Harmful or false output reaching users

Add a blocking output rule; revert prompt or model change.

Identify the failing input class and whether a recent release caused it.

Add regression cases; review the evaluation gaps (see Evaluation in CI).

Every runbook ends with the same three steps: a blameless review, a regression case in the evaluation set, and a check of whether the incident changes the risk register kept under Operational Governance. Guardrail-level mitigations are in Guardrails.

Secrets management

An LLM application holds unusually valuable credentials: provider API keys that translate directly into spend, tokens for tools that can write, and database or index credentials. The rules are the usual ones, with a few AI-specific consequences.

  • Keep credentials in a secret store and inject them at runtime as environment variables or by managed identity; code, prompts and configuration checked into version control never contain them. The Azure implementation is in Key Vault and Secrets.

  • Prefer short-lived, narrowly scoped credentials per tool over one broad key, and rotate on a schedule and after any suspected exposure.

  • Never place a secret in the prompt or in text the model can read: a model that can see a credential can be tricked into repeating it. Keep tool authentication outside the model, in the tool implementation.

  • Redact secrets from logs and traces with the same pipeline used for personal data, and scan repositories for committed secrets before merge.

  • Use separate keys per environment and per application so a leak can be revoked without an outage elsewhere, and so usage can be attributed.

Application code references them by name only:

import os

api_key = os.environ["MODEL_API_KEY"]

Derived from the Python standard library documentation: https://docs.python.org/3/library/os.html#os.environ. Token handling for the tools themselves is discussed in Excessive Agency and Tool Security.

Where to go next

References