Guardrails

Meta publishes safety classifiers as open-weight models. Llama Guard is an LLM fine-tuned to classify a prompt, or a prompt plus a response, against a taxonomy of hazard categories and to answer with a safe or unsafe verdict plus the violated categories. Because the taxonomy is part of the prompt template, a deployment can adapt categories to its policy. Prompt Guard is a much smaller classifier aimed at prompt-injection and jailbreak detection, which is why it is cheap enough to run on every request. Both are models, so they have the false-positive and false-negative behaviour of any classifier and need their own evaluation on your traffic. Check the model cards for the current version, licence terms and the supported languages before adopting either (see Choosing Models and Licences).

ShieldGemma is Google’s family of safety classifiers built on Gemma, trained to judge whether text violates a stated policy (for example sexually explicit content, dangerous content, hate speech and harassment) in user input or in model output. It is used the same way as Llama Guard: send the text with the policy, read back a verdict. Model sizes and policy wording are documented on the model card.

NeMo Guardrails (NVIDIA, 0.24.1 at the time of writing) is a programmable layer around an LLM. A configuration directory holds a config.yml that declares the model and the active rails (input, dialog, retrieval, execution and output), plus optional Colang flows and Python actions. The library wraps the model call, so the application talks to LLMRails instead of to the model.

The self check input and self check output flows ask the main model to judge the text against prompts that you must also define in the configuration (a prompts.yml file); the structure follows the NeMo Guardrails documentation, where the full set of built-in rails, including integrations with Llama Guard and other classifiers, is listed. Self-check rails cost an extra model call per check, which matters for the latency budget below.

Guardrails AI (0.11.0 at the time of writing) takes a validator-centred approach: a Guard composes validators, each of which checks a value and applies an on-fail action such as raising an exception, fixing the value, filtering it or re-asking the model. Validators are distributed through the Guardrails Hub and installed separately, so the set you can use depends on what you install.

The pattern follows the Guardrails AI documentation; validator names, parameters and the accepted on_fail values should be confirmed there for the installed version. Validators that wrap a local model add startup and per-call latency, so measure them before placing them on the request path.

Several providers offer a moderation endpoint that classifies text (and, for some, images) into harm categories with scores and a flagged field. OpenAI’s moderation endpoint and Azure AI Content Safety are the best known; Azure’s prompt-shield capability targets injection attempts specifically. Their advantages are that they need no hosting and are tuned continuously; the costs are an extra network round trip, a dependency on a third party, and a taxonomy you cannot change. Check each provider’s current model names, free-tier terms and data-handling policy, because user text is sent to a service. Moderation endpoints catch harmful categories; they are not injection or data-loss controls unless the provider says so. Azure AI Content Safety’s harm categories, Prompt Shields and groundedness checks are covered with code on Foundry Tools: Content Safety.

Spring AI provides a moderation abstraction with an OpenAI implementation. The pieces named in the reference documentation are OpenAiModerationModel, ModerationPrompt, ModerationResponse and OpenAiModerationOptions, and the model is enabled with the property spring.ai.model.moderation. The documentation page consulted for this page does not state the name of the common interface explicitly: it is expected to be ModerationModel, which another page in this section also uses, but this is unverified against the Spring AI 2.0 Javadoc, so confirm it before writing code against the interface and not the implementation.

The API key is read from the Spring configuration (spring.ai.openai.moderation.api-key, normally bound to an environment variable such as ${OPENAI_API_KEY}). The calls derive from the Spring AI OpenAI moderation reference; openAiModerationModel is an injected bean and userText the incoming message. The accessor chain for the results was simplified here and should be checked against the Javadoc.

LangChain 1.x ships built-in middleware for common guardrails. PIIMiddleware detects a PII type and applies a strategy (documented strategies include redacting, masking, hashing and blocking), ModelCallLimitMiddleware and ToolCallLimitMiddleware bound runaway loops, and HumanInTheLoopMiddleware implements the action-layer approval. Custom middleware covers anything else, as shown in the middleware page.

The middleware names and the overall shape derive from the LangChain guardrails guide and the middleware documentation; parameter names and the accepted strategy values change between releases, so confirm them for the installed version. search_docs and open_issue are the DocsAssistant tools defined elsewhere.

SafeGuardAdvisor blocks a request whose text contains any configured sensitive word and returns a fixed failure response without calling the model. Registered through defaultAdvisors, it is a cheap deterministic input guardrail; a moderation or classifier call can be wrapped in a custom advisor in the same position. Advisor order matters, as described in Advisors.

The builder follows the Spring AI advisors reference; confirm the exact builder methods against the 2.0.x Javadoc. A word list is a speed bump that paraphrase defeats, so treat this as the first and cheapest layer in front of the moderation and tool-level controls.