Trust Boundaries and Threat Modeling

A trust boundary is a line across which data or control passes between parties that deserve different levels of trust. In a classic web application the boundaries are the network edge, the service-to-service calls and the database. An LLM application adds boundaries that are easy to miss because they are not network hops: they are places where text written by someone else becomes part of the model’s context.

The parties and data flows to enumerate for every system are these.

The most important consequence is a rule that the later pages keep applying: the model’s authority is the minimum of the authority of everyone whose text is in its context. If the context contains a paragraph from an anonymous uploaded document, then anything the model can do, that anonymous author can attempt to make it do.

STRIDE is Microsoft’s mnemonic for six threat categories: spoofing, tampering, repudiation, information disclosure, denial of service and elevation of privilege. It applies to each element and flow of the diagram above; the LLM-specific reading of each letter is the useful part.

STRIDE looks at elements of a system. MAESTRO (Multi-Agent Environment, Security, Threat, Risk and Outcome), published by the Cloud Security Alliance, organises agentic threats by layer, which suits systems where several agents, tools and data stores interact. It describes seven layers: foundation models, data operations, agent frameworks, deployment and infrastructure, evaluation and observability, security and compliance, and the agent ecosystem, and it asks the modeller to consider threats inside each layer and across layers, for example a poisoned dataset in layer 2 that surfaces as a goal hijack in layer 7. For the DocsAssistant the cross-layer view is what shows that an ingestion pipeline weakness (data operations) becomes an agent action (ecosystem). Use STRIDE for the data-flow diagram and MAESTRO as a checklist that no layer was forgotten; the OWASP Top 10 for Agentic Applications gives names to the results.

MITRE ATLAS is a knowledge base of adversary tactics and techniques against AI systems, modelled on ATT&CK. Recording the ATLAS technique next to each threat lets a security operations team reuse its existing vocabulary and detections. The techniques used in this sub-section were confirmed against the ATLAS data repository on 2026-10-01.

The scope is the five parts listed on the landing page. The assets are the confidential documents in the vector store, the users' identities and conversations, the ticketing system that open_issue writes to, the monthly token budget, and the organisation’s reputation for accurate answers. The attackers considered are an anonymous chat user, an authenticated user trying to read documents above their clearance, an author of a document the assistant ingests, and the operator of a third-party MCP server.

Each path in the flow becomes one row of a threat register. Keep the register in the repository next to the code so it is reviewed with every change; a format such as the following is enough.

The format is the author’s own; the fields are chosen so that each risk can be reported in the vocabulary of any of the frameworks on the framework map. The NIST categories are those of NIST AI 600-1. A threat model is "good enough" when every register row has a control, a test and an owner, and when the residual risk is written down and accepted by a named person rather than silently carried.