Structured Output & Tool Calling

Before any vendor offered a schema-aware generation mode, the only way to get JSON out of a model was to ask for it in the prompt — "respond with a JSON object containing these fields, and nothing else" — and then regex- or substring-extract whatever the model wrapped it in ("Here is the JSON you requested: \`{…​}\`"), before finally calling a JSON parser on the extracted span. This still works, and it is still what a bare local model with no schema-aware API falls back to, but it carries three practical problems: the model can wrap the JSON in prose despite instructions not to, it can produce almost-valid JSON (a trailing comma, an unescaped quote inside a string) that a strict parser rejects, and nothing stops it from inventing a field name or omitting a required one. Native structured-output and JSON-schema modes — covered for each vendor below — replace that parse-the-text approach with the vendor constraining which tokens the model is allowed to sample so that the output is JSON, and in the strict variants so that it also matches a specific schema, before your code ever sees it. The regex-extraction approach is worth recognizing when reading older code or a model/runtime that has no schema-aware mode, but for anything built new against a current API, the sections below are the preferred starting point.

Both major hosted vendors expose two tiers of constraint, and the distinction matters for what you can assume downstream:

Locally run models get the equivalent guarantee through a different mechanism, covered next.

Ollama and llama.cpp — Ollama’s own inference engine is in fact built on llama.cpp — expose the same grammar-constrained idea for models you run yourself, without depending on a hosted vendor’s schema-aware endpoint. Apple’s on-device Foundation Models framework applies the same idea in Swift: its @Generable guided generation derives a schema from a structure and constrains decoding to it, as shown in Apple Intelligence and Machine Learning.

Anthropic’s recommended way to get a schema-conformant JSON answer is JSON outputs: pass output_config: \{"format": \{"type": "json_schema", "schema": …​}} on an ordinary messages.create call to /v1/messages. There is no separate endpoint and, as of 2026-09-28, no beta header. The API compiles the schema into grammar-constrained sampling, so the text block Claude returns always parses as JSON that matches the schema — correct types, all required fields present, no extra fields when additionalProperties: false is set:

The Python SDK also offers client.messages.parse(), which takes a Pydantic model as output_format, translates it to output_config.format for you, and returns the validated object as response.parsed_output:

Strict tool use is a separate, complementary guarantee: strict: true on a tool definition constrains the tool parameters Claude sends (tool_use.input) to the tool’s input_schema, rather than the shape of Claude’s answer. Use JSON outputs when the JSON is the answer, and strict tool use when the JSON is a call your code executes — the search_docs example in Tool / function calling as structured output below shows the latter.

See the Anthropic structured outputs documentation for output_config.format, messages.parse() and the supported JSON Schema subset, and the Anthropic strict tool use documentation for strict: true on tools.

OpenAI’s Responses API takes the JSON Schema directly on the request, through text.format with \{"type": "json_schema", …​} and strict: true, rather than routing the extraction through a tool call:

The json_schema format is, per OpenAI’s own documentation, the direct successor to the older \{"type": "json_object"} JSON mode: both guarantee syntactically valid JSON, but only json_schema with strict: true guarantees the result matches the schema’s required fields, types and (when additionalProperties: false is set) exact field set — OpenAI’s guidance is to prefer it over plain JSON mode whenever the target shape is known ahead of time, which it is for every example on this page.

See the OpenAI structured outputs guide for the text.format/json_schema shape (and its Chat Completions equivalent, response_format) and its per-model support.

A JSON-schema mode still only guarantees the wire format is correct; it says nothing about domain rules a plain JSON Schema cannot express (a component value that must be one of this section’s actual component names, a non-empty topics list). Pydantic closes that gap: keep output_config.format on the request so the shape is guaranteed, define the domain rules on a BaseModel, parse each response into it, and treat a ValidationError as a retry signal — feeding the validation error’s own message back to the model so the next attempt has a concrete reason to correct, rather than repeating the same mistake:

The loop reuses PAGE_METADATA_SCHEMA from the Anthropic section rather than PageMetadata.model_json_schema(), because Pydantic’s generated schema omits additionalProperties: false unless the model sets extra="forbid". It combines with the schema-conformance guarantee rather than replacing it: JSON outputs already rule out a missing field or wrong type, so the retries this loop spends are used entirely on the domain rules a JSON Schema cannot express (component_must_be_known, topics_must_not_be_empty) — a narrower, cheaper problem than validating the whole shape from scratch on every attempt. See the Anthropic structured outputs documentation for output_config.format on messages.create.

On the JVM, Jackson plays the role Pydantic’s parsing plays in Python: deserializing a JSON string into a typed object. Spring Boot 4.1 uses Jackson 3, whose packages moved to tools.jackson, whose mapper is the immutable JsonMapper built with JsonMapper.builder(), and whose base exception JacksonException is now unchecked. Domain rules go on the record components as Bean Validation annotations, which a record supports without any compact constructor:

Jackson does not fail closed on a missing field by default: a missing title simply becomes null. Enable DeserializationFeature.FAIL_ON_MISSING_CREATOR_PROPERTIES (and FAIL_ON_NULL_CREATOR_PROPERTIES for an explicit null) to make it raise. Jackson also never looks at the Bean Validation annotations, so the code calls a Validator explicitly after deserializing:

With both features on, a wrong shape raises JacksonException and a broken domain rule shows up as a ConstraintViolation, so the same retry-on-failure pattern from the Pydantic section applies. See the Jackson 3 migration guide for the tools.jackson packages, JsonMapper.builder() and the unchecked JacksonException, and the Hibernate Validator reference for Validator.validate().

Spring AI wraps exactly this parse-and-validate step behind ChatClient’s `entity() call, so application code never touches raw JSON or ObjectMapper directly. Internally it appends the target type’s JSON Schema as format instructions to the prompt (a BeanOutputConverter, covered in more structural depth for the vector-database and RAG scenario in Integrating with Spring AI) and deserializes the response with Jackson, raising the same way ObjectMapper.readValue does on a mismatch:

entity() accepts either a Class<T> (as above) or a ParameterizedTypeReference<T> for a generic target such as List<PageMetadata>. Spring AI’s own documentation describes this conversion as best-effort rather than a schema-conformance guarantee equivalent to Anthropic’s JSON outputs or OpenAI’s strict structured outputs — it depends on the underlying model actually following the appended format instructions — so pairing it with validation (Bean Validation annotations, or a manual check before the value is used) remains worthwhile even though entity() already removes the manual JSON parsing step.

Every example so far asked the model to return structured data as the final answer. Tool calling is the same underlying mechanism — the model still emits a schema-shaped JSON object rather than free text — pointed at a different purpose: instead of the JSON being the answer, it is a request your application executes, whose result is fed back so the model can produce a final, free-text answer. A search_docs tool for DocsAssistant illustrates the full round trip:

The strict: true flag on search_docs_tool is the tool-parameter counterpart of the JSON outputs used for extraction above: it guarantees tool_use.input always has a query string and never a stray extra field, so the application’s tool-executing code can trust the call’s shape without its own defensive parsing. The round trip itself — request, execute, return, final answer — is the same shape regardless of which vendor’s SDK is used; Context Windows & Memory covers what happens to this exchange once several tool calls accumulate across a longer conversation, and the "Best practices for tool definitions" section of Anthropic’s Define tools documentation covers writing the tool description field well enough that the model reaches for the right tool without being forced into it via tool_choice.

Across every technique on this page, "validate then retry with the error fed back" is the same three-step loop regardless of which layer produced the invalid output:

A production system typically bounds the retry loop (three attempts is a common default) and falls back to a human-reviewed queue or a rejection rather than looping indefinitely against a model that keeps failing the same validation.