Structured Output

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

This page’s examples target Python 3.12 with LangChain 1.4.3 / langchain-core 1.6.5 / langchain-classic 1.0.8 and langchain-ollama 1.1.0, using DocsAssistant's Ollama llama3.1:8b chat model, as checked on 2026-09-29 (see the LangChain ecosystem and packages page for the full baseline). Ollama was not available while the page was written, so every snippet was executed against a scripted fake chat model that replays canned AIMessage replies; the printed values are therefore what the LangChain code does with a well-formed (or deliberately malformed) reply, not a measurement of llama3.1:8b.

An LLM application rarely wants prose. It wants a title, a category, a list of citations — something a program can filter, store or route on. Structured output is the set of LangChain features that make the model return data matching a schema, parse it into a Python object and tell you when it does not fit. The wire-level mechanisms behind it (JSON mode, strict JSON Schema, grammar-constrained decoding in Ollama, tool calls as structured output) are explained in Structured Output & Tool Calling; this page is about the LangChain API on top of them. The model side of the story is on Chat Models & Messages, and the agent side on Agents with create_agent.

All examples share one schema, the metadata DocsAssistant extracts from each documentation page:

The PageMetadata schema used throughout this page

from typing import Literal

from pydantic import BaseModel, Field, field_validator


class PageMetadata(BaseModel):
    """Metadata describing one documentation page."""

    title: str = Field(description="The page title, exactly as written")
    component: Literal["vector-rag", "spring-ai", "langchain"] = Field(
        description="Which documentation component the page belongs to")
    level: Literal["beginner", "intermediate", "advanced"] = Field(description="Reader level")
    topics: list[str] = Field(description="Up to five lowercase topics, one to three words each", max_length=5)

    @field_validator("topics")
    @classmethod
    def lowercase_topics(cls, topics: list[str]) -> list[str]:
        bad = [t for t in topics if t != t.lower() or len(t.split()) > 3]
        if bad:
            raise ValueError(f"topics must be lowercase, 1-3 words: {bad}")
        return topics

The class docstring and every Field(description=…) are not decoration: LangChain sends them to the model as part of the schema, so they are the instructions for each field. See Pydantic — Models for the model syntax.

with_structured_output on a chat model

Every chat model has a with_structured_output(schema) method. It returns a new runnable that, given the same input as the model, returns the parsed object instead of an AIMessage. Under the hood LangChain either asks the provider to enforce the schema or forces a tool call whose arguments are the schema, then parses the result.

Pydantic schema: the result is a validated PageMetadata instance

from langchain_core.messages import AIMessage

from models import PageMetadata          # the class above

# In production: model = ChatOllama(model="llama3.1:8b", temperature=0)
# Here `model` replays a scripted tool-call reply so the example runs without Ollama.
model = scripted(AIMessage(content="", tool_calls=[{"name": "PageMetadata", "id": "call_1", "args": {
    "title": "HNSW and IVFFlat Indexes", "component": "vector-rag", "level": "advanced",
    "topics": ["hnsw", "ivfflat", "ann index"]}}]))

extractor = model.with_structured_output(PageMetadata)
meta = extractor.invoke("Extract metadata from: 'HNSW and IVFFlat Indexes -- tuning ef_search ...'")
print(type(meta).__name__, meta)

Running it printed PageMetadata title='HNSW and IVFFlat Indexes' component='vector-rag' level='advanced' topics=['hnsw', 'ivfflat', 'ann index']. scripted is a three-line helper: a GenericFakeChatModel subclass whose bind_tools returns itself, fed with the replies to replay. See LangChain — Models (Structured output).

The schema can be one of three kinds, with different guarantees:

Schema You get back Validation

Schema	You get back	Validation
Pydantic `BaseModel`	An instance of the class	Full: types, `Literal` values, `Field` constraints and your own validators run on every response.
`TypedDict`	A plain `dict`	None at runtime — the schema guides the model, but a wrong value is passed through.
JSON Schema `dict`	A plain `dict`	None in LangChain; the provider may enforce it, LangChain does not.

Pydantic BaseModel

An instance of the class

Full: types, Literal values, Field constraints and your own validators run on every response.

TypedDict

A plain dict

None at runtime — the schema guides the model, but a wrong value is passed through.

JSON Schema dict

A plain dict

None in LangChain; the provider may enforce it, LangChain does not.

The next snippet shows the last two kinds, including the consequence of having no validation:

TypedDict and JSON Schema

from typing import Annotated, Literal

from typing_extensions import TypedDict


class PageMetadataDict(TypedDict):
    """Metadata describing one documentation page."""

    title: Annotated[str, ..., "The page title"]
    component: Annotated[Literal["vector-rag", "spring-ai", "langchain"], ..., "Documentation component"]
    topics: Annotated[list[str], ..., "Up to five lowercase topics"]


json_schema = {
    "title": "PageMetadata",
    "description": "Metadata describing one documentation page.",
    "type": "object",
    "properties": {
        "title": {"type": "string", "description": "The page title"},
        "component": {"type": "string", "enum": ["vector-rag", "spring-ai", "langchain"]},
        "topics": {"type": "array", "items": {"type": "string"}},
    },
    "required": ["title", "component", "topics"],
}

print(model.with_structured_output(PageMetadataDict).invoke("..."))
print(model.with_structured_output(json_schema).invoke("..."))

Both calls returned a plain dict such as {'title': 'pgvector Indexes', 'component': 'vector-rag', 'topics': ['pgvector', 'hnsw']}. When the scripted reply carried 'component': 'nope', the TypedDict variant returned it unchanged, while the Pydantic variant raised a validation error — so use Pydantic whenever a wrong value should never reach the rest of the program, and TypedDict or JSON Schema when you validate elsewhere or need the schema to be shared with another language. A JSON Schema needs a top-level title and description. See LangChain — Models (Structured output) and JSON Schema — Understanding JSON Schema.

Methods and include_raw

with_structured_output also takes provider-dependent options. The method argument selects the mechanism: json_schema (the provider’s dedicated structured-output feature), function_calling (a forced tool call) or json_mode (valid JSON, but the schema must be described in the prompt). ChatOllama in langchain-ollama 1.1.0 accepts all three and defaults to json_schema, which uses Ollama’s constrained decoding — see Running LLMs Locally for what that means for local models, and LangChain — ChatOllama and Ollama — Structured outputs for the provider side.

include_raw=True makes the runnable return a dict with the raw AIMessage (raw), the parsed result (parsed) and the exception if parsing failed (parsing_error), instead of raising. That is how you get token usage next to the parsed object, and how you handle failures without a try block:

include_raw: parsed result, raw message and parsing error

bad = {"name": "PageMetadata", "id": "c3", "args": {
    "title": "pgvector Indexes", "component": "vector-rag", "level": "expert", "topics": ["pgvector"]}}
result = scripted(AIMessage(content="", tool_calls=[bad])) \
    .with_structured_output(PageMetadata, include_raw=True).invoke("...")

print(sorted(result))                                  # ['parsed', 'parsing_error', 'raw']
print(result["parsed"], type(result["parsing_error"]).__name__)

The keys were ['parsed', 'parsing_error', 'raw'], parsed was None and parsing_error was a Pydantic ValidationError for the invalid level. With a valid reply, parsing_error was None and raw an AIMessage. See LangChain — Models (Structured output).

Agent response_format

with_structured_output is for a single model call. An agent built with create_agent loops — it may call tools several times before it answers — so its structured output is requested once, up front, with the response_format argument, and the parsed result is added to the agent’s final state under the structured_response key. response_format accepts a schema type (Pydantic, dataclass or TypedDict), a ToolStrategy, a ProviderStrategy or None. A JSON Schema dict is also accepted bare — LangChain picks the strategy for it the same way as for a schema type, and the result is a plain dict. Wrap it in ToolStrategy or ProviderStrategy only to choose the strategy yourself or to set options such as strict or handle_errors.

If you pass a bare schema type, LangChain picks the strategy for you: ProviderStrategy when the model’s profile says it supports native structured output (as for OpenAI, Anthropic and xAI models), ToolStrategy otherwise. With langchain-ollama 1.1.0 and llama3.1:8b, a bare schema resolved to ToolStrategy in the version checked, so DocsAssistant’s local agent uses tool calling for its structured response. Hosted models such as claude-sonnet-5 or gpt-6-sol use the provider’s native mechanism when their profile advertises it, which is the more reliable option because the provider enforces the schema.

ToolStrategy

ToolStrategy exposes the schema to the model as an extra tool. When the model calls that tool, the agent validates the arguments, stores the result in structured_response and ends. It works with every model that supports tool calling. Beyond the schema it has two options: tool_message_content, the text of the ToolMessage that closes the exchange in the message history, and handle_errors, covered under validation and retries.

create_agent with ToolStrategy: a validation error is fed back and the model retries

from langchain.agents import create_agent
from langchain.agents.structured_output import ToolStrategy

good = {"title": "pgvector Indexes", "component": "vector-rag", "level": "advanced", "topics": ["pgvector"]}
bad = {**good, "level": "expert"}          # not in the Literal
model = scripted(
    AIMessage(content="", tool_calls=[{"name": "PageMetadata", "id": "call_1", "args": bad}]),
    AIMessage(content="", tool_calls=[{"name": "PageMetadata", "id": "call_2", "args": good}]),
)
agent = create_agent(
    model,
    tools=[],
    system_prompt="Extract metadata for the documentation page. Never invent values.",
    response_format=ToolStrategy(PageMetadata, tool_message_content="Metadata captured."),
)
result = agent.invoke({"messages": [{"role": "user", "content": "Describe: 'pgvector Indexes ...'"}]})
print(result["structured_response"])
for m in result["messages"]:
    print(type(m).__name__, "|", str(m.content).replace("\n", " ")[:160])

The final state held title='pgvector Indexes' component='vector-rag' level='advanced' topics=['pgvector']. The message history showed the human message, an AIMessage with the bad call, a ToolMessage starting Error: Failed to parse structured output for tool 'PageMetadata' followed by Pydantic’s explanation, a second AIMessage, and a closing ToolMessage reading Metadata captured. The error message is what lets the model correct itself on the second try. See LangChain — Structured output.

ToolStrategy also accepts a Union of schemas, and the model chooses the one that fits. DocsAssistant uses that to let one agent either extract metadata or recognise a request to report a documentation problem:

A union of schemas: the model picks the matching one

from typing import Union

from pydantic import BaseModel


class OpenIssue(BaseModel):
    """The user wants a documentation problem reported."""
    page: str
    problem: str


model = scripted(AIMessage(content="", tool_calls=[
    {"name": "OpenIssue", "id": "c", "args": {"page": "a.adoc", "problem": "wrong"}}]))
agent = create_agent(model, tools=[], response_format=ToolStrategy(Union[PageMetadata, OpenIssue]))
print(agent.invoke({"messages": [{"role": "user", "content": "x"}]})["structured_response"])

It returned page='a.adoc' problem='wrong', an OpenIssue. If a model calls two of the schema tools in one turn, the agent replies with an error ToolMessage and asks it to pick one. See LangChain — Structured output.

ProviderStrategy

ProviderStrategy asks the provider itself to constrain the response to the schema, so the reply arrives as JSON in the message content rather than as a tool call. Its strict option (LangChain 1.2 and later) requests strict schema adherence from providers that support it, such as OpenAI and xAI. Writing response_format=PageMetadata is equivalent to ProviderStrategy(PageMetadata) when the model supports native structured output, and falls back to tool calling when it does not.

ProviderStrategy: the JSON in the message content is parsed into the schema

import json

from langchain.agents.structured_output import ProviderStrategy

model = scripted(AIMessage(content=json.dumps(good)))        # native JSON output arrives as message text
agent = create_agent(model, tools=[], response_format=ProviderStrategy(PageMetadata, strict=True))
result = agent.invoke({"messages": [{"role": "user", "content": "Describe: 'pgvector Indexes ...'"}]})
print(result["structured_response"])
print([type(m).__name__ for m in result["messages"]])

The parsed PageMetadata was the same as before, and the message list was just ['HumanMessage', 'AIMessage']: no extra tool exchange. In a real deployment the model line becomes create_agent("anthropic:claude-sonnet-5", …) or create_agent("openai:gpt-6-sol", …). Note that create_agent needs a model that supports structured output together with tool calling if the agent also has tools. See LangChain — Structured output.

Output parsers

Before structured-output APIs existed, the way to get typed data was to describe the format in the prompt and parse the reply text with an output parser. Parsers still matter in two situations: the model or endpoint has neither tool calling nor native structured output, or you are building a chain of prompt, model and parser with LCEL and Runnables and want the format instructions in the prompt. langchain_core.output_parsers provides StrOutputParser (message to plain string), JsonOutputParser (reply to a dict, tolerating a fenced JSON block), PydanticOutputParser (reply to a validated model, with get_format_instructions() to put the schema in the prompt) and list and XML parsers. The tool-call parsers such as JsonOutputKeyToolsParser are what with_structured_output uses internally.

PydanticOutputParser in a prompt | model | parser chain

from langchain_core.exceptions import OutputParserException
from langchain_core.language_models.fake_chat_models import GenericFakeChatModel
from langchain_core.output_parsers import JsonOutputParser, PydanticOutputParser, StrOutputParser
from langchain_core.prompts import ChatPromptTemplate

parser = PydanticOutputParser(pydantic_object=PageMetadata)
prompt = ChatPromptTemplate.from_messages([
    ("system", "Extract documentation metadata.\n{format_instructions}"),
    ("human", "{page}"),
]).partial(format_instructions=parser.get_format_instructions())

good_json = '{"title": "pgvector Indexes", "component": "vector-rag", "level": "advanced", "topics": ["pgvector"]}'
model = GenericFakeChatModel(messages=iter([
    AIMessage(content="Sure! Here is the JSON: " + good_json),      # chatty prefix: not parseable
    AIMessage(content="```json\n" + good_json + "\n```"),            # fenced JSON: parsed
]))
chain = prompt | model | parser

try:
    chain.invoke({"page": "pgvector Indexes"})
except OutputParserException as e:
    print("first attempt:", type(e).__name__)
print("second attempt:", chain.invoke({"page": "pgvector Indexes"}))

print((GenericFakeChatModel(messages=iter([AIMessage(content=good_json)])) | JsonOutputParser()).invoke("x"))
print((GenericFakeChatModel(messages=iter([AIMessage(content="plain")])) | StrOutputParser()).invoke("x"))

The first reply, which had prose before the JSON, raised OutputParserException; the second, a fenced JSON block, produced a PageMetadata. JsonOutputParser returned a dict and StrOutputParser returned plain. The lesson is the same as for schemas without validation: a parser only fails after the model has answered, and nothing in the prompt forces the model to comply. Prefer with_structured_output or response_format when the model supports them, and keep parsers for models that do not. See LangChain reference — output parsers.

Validation and retries

Structured output has two failure modes: the reply is not parseable (bad JSON, wrong tool arguments) or it parses but breaks a rule (an unknown component, a topics value in upper case). Pydantic catches both when the schema is a BaseModel; what remains is deciding who retries and with what feedback. LangChain gives three levels.

In an agent, ToolStrategy(handle_errors=…) controls the retry loop, as the earlier snippet showed with the default True: any parsing or validation error becomes a ToolMessage and the model tries again. The parameter also accepts a string (always send that fixed message), an exception type or tuple of types (retry only for those, raise the rest), a callable that turns the exception into the message, or False (raise). The two exception classes you can distinguish are StructuredOutputValidationError and MultipleStructuredOutputsError, both from langchain.agents.structured_output.

A custom handle_errors function, and handle_errors=False

from langchain.agents.structured_output import StructuredOutputValidationError


def explain(error: Exception) -> str:
    if isinstance(error, StructuredOutputValidationError):
        return f"Invalid metadata: {error}. Use only the listed components and levels."
    return f"Error: {error}"


bad = {**good, "component": "kafka"}
agent = create_agent(
    scripted(AIMessage(content="", tool_calls=[{"name": "PageMetadata", "id": "c1", "args": bad}]),
             AIMessage(content="", tool_calls=[{"name": "PageMetadata", "id": "c2", "args": good}])),
    tools=[],
    response_format=ToolStrategy(PageMetadata, handle_errors=explain),
)
result = agent.invoke({"messages": [{"role": "user", "content": "Describe the page."}]})
print(result["structured_response"].component)            # vector-rag, after one corrective round
print(result["messages"][2].content[:120])                # the ToolMessage produced by explain()

The first attempt was rejected with the message built by explain, and the second produced vector-rag. With handle_errors=False the same scripted run raised StructuredOutputValidationError out of invoke, which is what you want when a failure should stop the run rather than cost another model call. See LangChain — Structured output (Error handling).

Around a single model call, include_raw=True hands you the failure so you can decide. A validate-and-repair loop shows the model its own bad answer and the error, which is a much better retry signal than sending the same prompt again:

A repair loop built on include_raw

from langchain_core.messages import HumanMessage


def extract_with_repair(model, page: str, max_attempts: int = 3) -> PageMetadata:
    """Validate the model's answer; on failure show it the error and ask again."""
    structured = model.with_structured_output(PageMetadata, include_raw=True)
    messages = [HumanMessage(f"Extract metadata for this page:\n{page}")]
    for attempt in range(1, max_attempts + 1):
        out = structured.invoke(messages)
        if out["parsing_error"] is None and out["parsed"] is not None:
            return out["parsed"]
        messages += [out["raw"], HumanMessage(
            f"That failed validation ({type(out['parsing_error']).__name__}): "
            f"{str(out['parsing_error']).splitlines()[-2:]}. Fix it and answer again.")]
    raise ValueError(f"no valid metadata after {max_attempts} attempts")

With a scripted model that first answered level: expert and then a valid reply, it returned the valid PageMetadata on the second attempt; with a model that always answered {"title": "x"} it raised ValueError: no valid metadata after 3 attempts. Bounding the attempts matters: an uncapped loop turns a schema the model cannot satisfy into an unbounded bill. The same validate-and-retry pattern, without LangChain, is on Structured Output & Tool Calling. See LangChain — Models (Structured output).

Around a chain, any runnable has .with_retry(…), which re-runs it when an exception matching retry_if_exception_type is raised. Each retry is a fresh call to the model, so it helps with sampling noise (use a temperature above zero) but not with a prompt that the model consistently misreads:

with_retry on a prompt | model | parser chain

model = GenericFakeChatModel(messages=iter([AIMessage(content="no json here"), AIMessage(content=good_json)]))
chain = (prompt | model | parser).with_retry(
    retry_if_exception_type=(OutputParserException,), stop_after_attempt=3, wait_exponential_jitter=False)
print(chain.invoke({"page": "pgvector Indexes"}))

The first reply had no JSON and raised, the retry consumed the second reply, and the chain returned the PageMetadata. The legacy OutputFixingParser and RetryOutputParser, which ask a second model call to repair a bad reply, live in the langchain-classic package (langchain_classic.output_parsers) and are not part of the v1 langchain package; for new code prefer the error-feedback loops above. Retry-related middleware for whole agent runs is covered on Middleware. See LangChain — Structured output and Pydantic — Validators.

DocsAssistant: metadata extraction

DocsAssistant’s corpus (the documentation pages indexed in the pgvector store described on PostgreSQL & pgvector) is only searchable by similarity until each chunk carries filterable metadata. Extracting it with PageMetadata is a good structured-output workload: one call per page, a schema with closed vocabularies, and results that are validated before they are written.

Batch extraction into rows ready for the vector store

pages = {
    "modules/ROOT/pages/database/vector-rag/ann-indexes.adoc": {
        "title": "ANN Indexes", "component": "vector-rag", "level": "advanced", "topics": ["hnsw", "ivfflat"]},
    "modules/ROOT/pages/database/vector-rag/embeddings.adoc": {
        "title": "Embeddings", "component": "vector-rag", "level": "beginner",
        "topics": ["embeddings", "cosine similarity"]},
}
model = scripted(*[AIMessage(content="", tool_calls=[{"name": "PageMetadata", "id": f"c{i}", "args": a}])
                   for i, a in enumerate(pages.values())])
extractor = model.with_structured_output(PageMetadata)

results = extractor.batch([f"Extract metadata:\n{path}" for path in pages], config={"max_concurrency": 1})
rows = [{"source_path": path, **meta.model_dump()} for path, meta in zip(pages, results)]
for row in rows:
    print(row)

Each row printed as a flat dict such as {'source_path': 'modules/ROOT/pages/database/vector-rag/ann-indexes.adoc', 'title': 'ANN Indexes', 'component': 'vector-rag', 'level': 'advanced', 'topics': ['hnsw', 'ivfflat']}. In the real pipeline the prompt contains the page text, the model is ChatOllama(model="llama3.1:8b", temperature=0), max_concurrency matches the number of requests Ollama serves in parallel, and each row’s fields become metadata on the page’s chunks — the component and level columns are exactly what Metadata Filtering uses to narrow a similarity search. The field_validator on topics also runs here: a value like ["HNSW"] was rejected with Value error, topics must be lowercase, 1-3 words: ['HNSW'], so a sloppy extraction never reaches the store. Batch runs are also where the repair loop earns its keep, since one malformed page should retry rather than abort the whole job. See LangChain — Models (Structured output) and Runnable — batch.

The same mechanism shapes DocsAssistant’s answers. Instead of free text, the chat agent returns an answer with citations, and a confident flag the web client can use to show a "no good match" message:

A grounded answer as response_format

class Citation(BaseModel):
    page: str = Field(description="Repository path of the page, exactly as search_docs returned it")
    section: str


class DocsAnswer(BaseModel):
    """A grounded answer for the reader."""
    answer: str
    citations: list[Citation] = Field(description="At least one citation for every factual claim")
    confident: bool = Field(description="False if the retrieved sections did not answer the question")


call = {"name": "DocsAnswer", "id": "c", "args": {
    "answer": "Raise ef_search to trade latency for recall.",
    "citations": [{"page": "modules/ROOT/pages/database/vector-rag/ann-indexes.adoc", "section": "HNSW"}],
    "confident": True}}
agent = create_agent(scripted(AIMessage(content="", tool_calls=[call])), tools=[], response_format=DocsAnswer)
out = agent.invoke({"messages": [{"role": "user", "content": "How do I improve HNSW recall?"}]})
print(out["structured_response"])

It printed answer='Raise ef_search to trade latency for recall.' citations=[Citation(page='modules/ROOT/pages/database/vector-rag/ann-indexes.adoc', section='HNSW')] confident=True. Note that the schema constrains the shape of the citations, not their truth: whether page names a page that exists is a check for code after the call (or for the evaluation covered on Evaluation), which is the same point made for tool arguments in Tool Design. The search_docs tool this agent calls is defined on Tools. See LangChain — Structured output.

Choosing an approach

Situation Use

Situation	Use
One model call, result must be validated	`model.with_structured_output(PageMetadata)` with a Pydantic schema.
One model call, you want to see and handle failures	Add `include_raw=True` and check `parsing_error`, or wrap it in a repair loop.
Agent that calls tools, then answers in a fixed shape	`create_agent(…, response_format=Schema)`; read `result["structured_response"]`.
Model with native structured output (hosted models, when the profile says so)	A bare schema type or JSON Schema `dict`, or `ProviderStrategy` when you need `strict`.
Model with tool calling only (a local model through Ollama, in the version checked)	`ToolStrategy`, with `handle_errors` set deliberately.
No tool calling and no native structured output	An output parser with format instructions in the prompt, plus `.with_retry`.

One model call, result must be validated

model.with_structured_output(PageMetadata) with a Pydantic schema.

One model call, you want to see and handle failures

Add include_raw=True and check parsing_error, or wrap it in a repair loop.

Agent that calls tools, then answers in a fixed shape

create_agent(…, response_format=Schema); read result["structured_response"].

Model with native structured output (hosted models, when the profile says so)

A bare schema type or JSON Schema dict, or ProviderStrategy when you need strict.

Model with tool calling only (a local model through Ollama, in the version checked)

ToolStrategy, with handle_errors set deliberately.

No tool calling and no native structured output

An output parser with format instructions in the prompt, plus .with_retry.

Structured output is only as trustworthy as the model behind it: small local models produce more schema violations, which is one more reason to keep schemas small, use closed vocabularies, and validate in code. When the workflow outgrows a single call or a single agent, move to LangGraph Fundamentals, where a node can call with_structured_output to decide which branch runs next.