LangChain4j

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

This page was written against LangChain4j 1.20.2 (the stable modules; the -beta modules are at 1.20.2-beta30) on Java 21, with the Quarkiverse quarkus-langchain4j extension at 1.14.0, as verified on 2026-09-29. The running example is DocsAssistant, which answers questions over this site’s documentation with Ollama’s llama3.1:8b chat model and the nomic-embed-text embedding model over the pgvector store from Integrating with LangChain. Every snippet below was compiled with Maven’s Java 21 toolchain and executed, except the ones that build a real Ollama or OpenAI client (they were compiled only, because no Ollama server was available). The outputs shown come from a scripted fake model described in the next section, so they are exactly what the code prints, not what a real model would say.

LangChain4j is the Java library that plays the role LangChain plays in Python: one API over many model providers, a declarative way to build LLM-backed services out of plain interfaces, and building blocks for tools, memory, RAG, guardrails and agents. It is an independent open-source project, not a port of the Python code, so names differ (a ChatModel here is a BaseChatModel there, an AI Service has no exact Python twin). This page covers the core library without Spring; the experimental agentic module and the MCP client are on LangChain4j Agents & MCP. The framework-neutral background is in Tool Design, Memory and Agent Guardrails & Safety.

Modules and the BOM

LangChain4j is split into small artifacts under the group dev.langchain4j. You depend on the abstractions plus one artifact per provider or store you use:

Artifact What it gives you

langchain4j-core

The low-level API: ChatModel, StreamingChatModel, messages, ChatRequest / ChatResponse, EmbeddingModel, EmbeddingStore, ToolSpecification, guardrail and listener interfaces. Pulled in by langchain4j.

langchain4j

The high-level API: AiServices, MessageWindowChatMemory, TokenWindowChatMemory, RAG components.

langchain4j-ollama, langchain4j-open-ai, langchain4j-anthropic, …​

One artifact per model provider, with the chat, streaming and embedding model classes for it.

langchain4j-pgvector, …​

One artifact per embedding store (see the RAG section below).

langchain4j-agentic, langchain4j-mcp, langchain4j-reactor, langchain4j-test

Beta-versioned modules: agents and workflows, the MCP client, Reactor adapters and test helpers.

langchain4j-spring-boot4-starter, langchain4j-spring-boot-starter, langchain4j-ollama-spring-boot4-starter, …​

Spring Boot starters for Boot 4 and Boot 3 (beta-versioned).

Not every module shares the core’s version number: the core line is at 1.20.2 while modules that are still settling carry a -beta suffix (1.20.2-beta30). The Bill of Materials, langchain4j-bom, manages both kinds, so importing it once lets you omit the version on every LangChain4j dependency, beta modules included. We resolved this exact pom.xml with Maven and it downloaded langchain4j 1.20.2, langchain4j-ollama 1.20.2 and the -beta30 builds of langchain4j-agentic, langchain4j-mcp, langchain4j-reactor and langchain4j-pgvector:

pom.xml — import the BOM, then declare artifacts without versions
<properties>
    <maven.compiler.release>21</maven.compiler.release>
</properties>

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>dev.langchain4j</groupId>
            <artifactId>langchain4j-bom</artifactId>
            <version>1.20.2</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

<dependencies>
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j</artifactId>
    </dependency>
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j-ollama</artifactId>
    </dependency>
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j-pgvector</artifactId>
    </dependency>
</dependencies>

The library targets Java 17 as its baseline; the examples here use Java 21 language features (text blocks, records, var). Older tutorials, and some books, target the 0.x line (0.35 and 0.36 are common); class names and builders have changed since then, so treat that material as history and check names against the 1.x documentation. Official page: LangChain4j — Get started.

The scripted models used on this page

Ollama was not available while this page was written, so the runnable examples use two small ChatModel and StreamingChatModel implementations that replay a script and record every request. Implementing the interfaces is also how you unit-test LangChain4j code without a model; ChatModel only requires doChat, and everything else (the string overloads, listeners, retries) is inherited:

A scripted ChatModel for tests and for this page
import dev.langchain4j.data.message.AiMessage;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.chat.StreamingChatModel;
import dev.langchain4j.model.chat.listener.ChatModelListener;
import dev.langchain4j.model.chat.request.ChatRequest;
import dev.langchain4j.model.chat.response.ChatResponse;
import dev.langchain4j.model.chat.response.StreamingChatResponseHandler;

import dev.langchain4j.model.chat.response.ChatModelStreamingEvent;
import dev.langchain4j.model.chat.response.CompleteResponse;
import dev.langchain4j.model.chat.response.PartialResponse;
import dev.langchain4j.model.chat.Capability;

import java.util.ArrayDeque;
import java.util.Set;
import java.util.concurrent.Flow;
import java.util.concurrent.SubmissionPublisher;
import java.util.ArrayList;
import java.util.Deque;
import java.util.List;

/** Scripted stand-ins for Ollama: replies come from a queue and every request is recorded. */
final class ScriptedChatModel implements ChatModel {

    final List<ChatRequest> requests = new ArrayList<>();
    private final Deque<AiMessage> script = new ArrayDeque<>();
    private List<ChatModelListener> listeners = List.of();

    ScriptedChatModel(AiMessage... replies) {
        script.addAll(List.of(replies));
    }

    static ScriptedChatModel saying(String... texts) {
        return new ScriptedChatModel(java.util.Arrays.stream(texts).map(AiMessage::from).toArray(AiMessage[]::new));
    }

    private boolean jsonSchema;

    ScriptedChatModel withJsonSchema() {
        this.jsonSchema = true;
        return this;
    }

    @Override
    public Set<Capability> supportedCapabilities() {
        return jsonSchema ? Set.of(Capability.RESPONSE_FORMAT_JSON_SCHEMA) : Set.of();
    }

    ScriptedChatModel withListeners(ChatModelListener... l) {
        this.listeners = List.of(l);
        return this;
    }

    @Override
    public List<ChatModelListener> listeners() {
        return listeners;
    }

    @Override
    public ChatResponse doChat(ChatRequest request) {
        requests.add(request);
        return ChatResponse.builder().aiMessage(script.removeFirst()).build();
    }
}

final class ScriptedStreamingChatModel implements StreamingChatModel {

    private final List<String> tokens;

    ScriptedStreamingChatModel(String... tokens) {
        this.tokens = List.of(tokens);
    }

    @Override
    public void doChat(ChatRequest request, StreamingChatResponseHandler handler) {
        tokens.forEach(handler::onPartialResponse);
        handler.onCompleteResponse(ChatResponse.builder().aiMessage(AiMessage.from(String.join("", tokens))).build());
    }
}

/** Implements the Publisher-based doChat that Flow.Publisher AI Service methods require. */
final class ScriptedPublisherChatModel implements StreamingChatModel {

    private final List<String> tokens;

    ScriptedPublisherChatModel(String... tokens) {
        this.tokens = List.of(tokens);
    }

    @Override
    public Flow.Publisher<ChatModelStreamingEvent> doChat(ChatRequest request) {
        return subscriber -> {                       // cold: each subscription starts one model call
            SubmissionPublisher<ChatModelStreamingEvent> events = new SubmissionPublisher<>();
            events.subscribe(subscriber);
            new Thread(() -> {
                tokens.forEach(t -> events.submit(new PartialResponse(t)));
                events.submit(new CompleteResponse(
                        ChatResponse.builder().aiMessage(AiMessage.from(String.join("", tokens))).build()));
                events.close();
            }).start();
        };
    }
}

Two behaviours of the library show up in the fakes and are worth knowing early. First, doChat is the only method a synchronous provider must implement. Second, the asynchronous entry points (chatAsync, and a Flow.Publisher return type on StreamingChatModel) each have their own doChatAsync / publisher-based doChat hook: a model that has not implemented that hook fails loudly instead of silently blocking, which the streaming section demonstrates.

ChatModel and StreamingChatModel

ChatModel is the synchronous interface: chat(String) for a quick question, chat(ChatRequest) when you need messages, parameters, tools or a response format, and it returns a ChatResponse holding the AiMessage, the finish reason and the token usage. StreamingChatModel is the same contract delivered through a callback (StreamingChatResponseHandler) as tokens are generated. Providers implement both, with the same builder options:

Model factories for DocsAssistant, and both ways of asking a question
import dev.langchain4j.data.message.SystemMessage;
import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.model.chat.Capability;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.chat.StreamingChatModel;
import dev.langchain4j.model.chat.request.ChatRequest;
import dev.langchain4j.model.chat.response.ChatResponse;
import dev.langchain4j.model.embedding.EmbeddingModel;
import dev.langchain4j.model.ollama.OllamaChatModel;
import dev.langchain4j.model.ollama.OllamaEmbeddingModel;
import dev.langchain4j.model.ollama.OllamaStreamingChatModel;
import dev.langchain4j.model.openai.OpenAiChatModel;

import java.time.Duration;

final class S1Models {

    static ChatModel ollamaChat() {
        return OllamaChatModel.builder()
                .baseUrl("http://localhost:11434")
                .modelName("llama3.1:8b")
                .temperature(0.0)
                .timeout(Duration.ofSeconds(120))
                .build();
    }

    static ChatModel ollamaJsonSchema() {
        return OllamaChatModel.builder()
                .baseUrl("http://localhost:11434")
                .modelName("llama3.1:8b")
                .supportedCapabilities(Capability.RESPONSE_FORMAT_JSON_SCHEMA)   // send a JSON schema, not prompt text
                .build();
    }

    static StreamingChatModel ollamaStreaming() {
        return OllamaStreamingChatModel.builder()
                .baseUrl("http://localhost:11434")
                .modelName("llama3.1:8b")
                .build();
    }

    static EmbeddingModel ollamaEmbeddings() {
        return OllamaEmbeddingModel.builder()
                .baseUrl("http://localhost:11434")
                .modelName("nomic-embed-text")
                .build();
    }

    static ChatModel hostedChat() {
        return OpenAiChatModel.builder()
                .apiKey(System.getenv("OPENAI_API_KEY"))   // never a literal key
                .modelName("gpt-6-sol")
                .build();
    }

    /** Works with any ChatModel: the one-liner and the explicit ChatRequest form. */
    static void ask(ChatModel model) {
        String quick = model.chat("Which extension stores embeddings in PostgreSQL?");
        System.out.println("quick   : " + quick);

        ChatRequest request = ChatRequest.builder()
                .messages(SystemMessage.from("Answer from the documentation only."),
                        UserMessage.from("Which extension stores embeddings in PostgreSQL?"))
                .build();
        ChatResponse response = model.chat(request);
        System.out.println("explicit: " + response.aiMessage().text() + " | finish=" + response.finishReason());
    }
}

Running S1Models.ask(…​) against a scripted model, which replies pgvector both times:

Output of ask against the scripted model
quick   : pgvector
explicit: pgvector | finish=null

finish=null is an artifact of the fake, which builds its ChatResponse without metadata; a real provider fills in the finish reason and token usage. Three details in the factories matter in practice. The OpenAI key comes from the environment, never from a literal. The Ollama models talk to http://localhost:11434, the default of the Ollama server described in Ollama, and a local 8B model can be slow to answer, hence the explicit timeout. And supportedCapabilities(Capability.RESPONSE_FORMAT_JSON_SCHEMA) opts an Ollama model into sending a JSON schema with the request, which the structured-output section relies on. Hosted model ids follow the rest of this section: gpt-6-sol for OpenAI. Official pages: LangChain4j — Chat and language models and LangChain4j — Ollama.

AI Services

The low-level API leaves you to assemble messages, keep history, parse replies and run the tool loop. AI Services is LangChain4j’s answer: you write a plain Java interface, AiServices.builder(…​) generates an implementation behind a dynamic proxy, and every method call becomes a model call with the prompt, memory, tools, guardrails and output parsing you configured. The annotations on the interface are the prompt:

  • @SystemMessage sets the system prompt; text blocks keep multi-line prompts readable.

  • @UserMessage sets a template for the user message. Variables are written {{name}}; a single parameter without an annotation is available as {{it}}.

  • @V("name") binds a method parameter to a template variable. It is needed for multi-parameter methods unless the code is compiled with -parameters, which the examples here do.

DocsAssistant as an AI Service
import dev.langchain4j.service.AiServices;
import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.UserMessage;
import dev.langchain4j.service.V;
import dev.langchain4j.model.chat.ChatModel;

interface DocsAssistant {

    @SystemMessage("""
            You are DocsAssistant. Answer only from the documentation of the {{component}} component
            and say so when you do not know.""")
    @UserMessage("Question about {{topic}}: {{question}}")
    String answer(@V("component") String component, @V("topic") String topic, @V("question") String question);

    String chat(String message);
}

final class S2AiServices {

    static DocsAssistant build(ChatModel model) {
        return AiServices.builder(DocsAssistant.class)
                .chatModel(model)
                .build();
    }
}

The builder wires everything in one place, and an interface without a @UserMessage (like chat above) sends the single argument as the user message. Running answer("vector-rag", "pgvector", "Which store class do I use?") against the scripted model prints the reply, and the request the model received shows how the templates rendered:

What the model received
SYSTEM: SystemMessage { text = "You are DocsAssistant. Answer only from the documentation of the vector-rag component
and say so when you do not know." }
USER: UserMessage { name = null, contents = [TextContent { text = "Question about pgvector: Which store class do I use?" }], attributes = {} }

The diagram shows what one call does. Everything in the middle is optional and configured on the builder; the boxes outlined by the input and output guardrails and by the listeners are covered in their own sections:

flowchart TD C["assistant.answer(...)"] --> IG["input guardrails"] IG -->|fatal| X["InputGuardrailException"] IG --> MEM["load chat memory"] MEM --> RAG["RAG: retrieve and augment"] RAG --> LLM["ChatModel.chat(request)"] LLM -->|tool calls| T["execute @Tool methods"] T --> LLM LLM -->|final answer| OG["output guardrails"] OG -->|reprompt| LLM OG --> P["parse into the return type"] P --> R["return value"]

Official page: LangChain4j — AI Services.

Tools

A method annotated with @Tool becomes a tool: LangChain4j reads its name, its description (the annotation value) and its parameters, builds a ToolSpecification with a JSON schema, sends it with every request and executes the method when the model asks for it. @P describes a parameter and required = false makes it optional. Tool classes are passed to tools(…​) on the builder as plain objects, so a tool can hold collaborators such as a database client.

DocsAssistant’s two tools follow the guidance in Tool Design. The destructive one, openIssue, does not act by itself: it asks an injected approver, and a rejection is returned to the model as text it can act on. LangChain4j does not impose an approval mechanism for AI Services, so the gate is ordinary application code; in production the Predicate would call your approval UI, and the credential for the GitHub API would come from an environment variable or secret store and carry only the permission to create issues.

Two tools, one guarded by an approval predicate
import dev.langchain4j.agent.tool.P;
import dev.langchain4j.agent.tool.Tool;

import java.util.List;
import java.util.function.Predicate;

final class DocsTools {

    private final Predicate<String> approver;

    DocsTools(Predicate<String> approver) {
        this.approver = approver;
    }

    @Tool("Search the documentation and return matching page paths with a one-line snippet")
    List<String> searchDocs(@P("keywords to search for") String query,
                            @P(value = "maximum number of results", required = false) Integer limit) {
        int max = limit == null ? 3 : limit;
        return List.of("ai/langchain/tools.adoc -- Tools", "database/vector-rag/agentic-rag.adoc -- Agentic RAG")
                .stream().limit(max).toList();
    }

    @Tool("Open a GitHub issue in the docs repository. Requires human approval and returns the outcome.")
    String openIssue(@P("issue title") String title) {
        if (!approver.test(title)) {
            return "REJECTED: a human declined to open \"" + title + "\"; do not retry.";
        }
        return "OPENED: issue \"" + title + "\" (simulated, no network call)";
    }
}

Wiring them and scripting a model that calls searchDocs and then openIssue, with an approver that always says no:

Registering the tools with an AI Service
var toolAssistant = AiServices.builder(DocsAssistant.class)
        .chatModel(model)
        .tools(new DocsTools(title -> false))   // approver: a human declines
        .build();
toolAssistant.chat("Find pgvector docs and open an issue about HNSW tuning.");

The specifications the model received, and the tool messages the library appended between the two model calls, come straight from the recorded requests (long lines trimmed):

Tool specifications and the tool loop
spec: ToolSpecification { name = "searchDocs", description = "Search the documentation and return matching page paths with a one-line snippet", parameters = JsonObjectSchema { properties = {query=JsonStringSchema {description = "keywords to search for" }, limit=JsonIntegerSchema {description = "maximum number of results" }}, required = [query], ... } }
spec: ToolSpecification { name = "openIssue", description = "Open a GitHub issue in the docs repository. Requires human approval and returns the outcome.", ... required = [title] ... }
AI: AiMessage { text = null, toolExecutionRequests = [ToolExecutionRequest { id = "call-1", name = "searchDocs", arguments = "{"query":"pgvector","limit":1}" }] }
TOOL_EXECUTION_RESULT: ToolExecutionResultMessage{id='call-1', toolName='searchDocs', contents=[TextContent { text = "["ai/langchain/tools.adoc -- Tools"]" }], isError=false}
AI: AiMessage { text = null, toolExecutionRequests = [ToolExecutionRequest { id = "call-2", name = "openIssue", arguments = "{"title":"Document HNSW tuning"}" }] }
TOOL_EXECUTION_RESULT: ToolExecutionResultMessage{id='call-2', toolName='openIssue', contents=[TextContent { text = "REJECTED: a human declined to open "Document HNSW tuning"; do not retry." }], isError=false}

Note the schema: limit is described but absent from required, because of required = false, and query is required. The final answer is the third scripted model reply, The issue was not opened.

Other tool features from the official page, not run here: a @ToolMemoryId parameter receives the memory id of the current conversation (so tools can be per-user), toolArgumentsErrorHandler and toolExecutionErrorHandler on the builder decide whether a bad tool call is sent back to the model as text or aborts the service, hallucinatedToolNameStrategy handles calls to tools that do not exist, and tools can be registered without annotations through AiServiceTool when the set of tools is only known at runtime. Tools that come from an MCP server (see the LangChain4j MCP page) are added with a ToolProvider and are covered in LangChain4j Agents & MCP. Official page: LangChain4j — Tools (function calling).

Chat memory

An LLM is stateless, so an AI Service that should remember the conversation needs a ChatMemory. It is a container that stores the messages exchanged, applies an eviction policy, and hands the retained messages back with every request. LangChain4j ships two: MessageWindowChatMemory keeps the last N messages, and TokenWindowChatMemory keeps the last N tokens (it needs a TokenCountEstimator). Messages are never split, so an evicted message goes whole, and evicting an AiMessage that requested tools also evicts the tool results that followed it. The system message is special: only one is kept, and it is retained when older messages are evicted. The official page also points out that memory is not history: memory is what the model sees, history is what the user saw, and only memory is provided.

Two configurations cover most cases. chatMemory(…​) on the builder gives the service one shared memory, fine for a single-user tool. For a multi-user application, chatMemoryProvider(…​) takes a function from a memory id to a ChatMemory, and a @MemoryId parameter on the AI Service method selects the conversation. The provider below is the same idea as LangGraph’s thread_id (see Short-Term Memory & Checkpointers).

ChatMemoryStore is the persistence hook. MessageWindowChatMemory keeps its messages in a store, in memory by default; implementing the three methods (getMessages, updateMessages, deleteMessages) moves them anywhere. Note that updateMessages receives the complete list every time, so a message evicted from the window disappears from the store too. ChatMessageSerializer and ChatMessageDeserializer convert messages to and from JSON:

Per-user memory with a JSON-backed ChatMemoryStore
import dev.langchain4j.data.message.ChatMessage;
import dev.langchain4j.data.message.ChatMessageDeserializer;
import dev.langchain4j.data.message.ChatMessageSerializer;
import dev.langchain4j.memory.ChatMemory;
import dev.langchain4j.memory.chat.ChatMemoryProvider;
import dev.langchain4j.memory.chat.MessageWindowChatMemory;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.service.AiServices;
import dev.langchain4j.service.MemoryId;
import dev.langchain4j.service.UserMessage;
import dev.langchain4j.store.memory.chat.ChatMemoryStore;

import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;

interface MemoryAssistant {
    String chat(@MemoryId String userId, @UserMessage String message);
}

/** Keeps one JSON document per conversation; swap the map for a database table in production. */
final class JsonChatMemoryStore implements ChatMemoryStore {

    final Map<Object, String> rows = new ConcurrentHashMap<>();

    @Override
    public List<ChatMessage> getMessages(Object memoryId) {
        String json = rows.get(memoryId);
        return json == null ? List.of() : ChatMessageDeserializer.messagesFromJson(json);
    }

    @Override
    public void updateMessages(Object memoryId, List<ChatMessage> messages) {
        rows.put(memoryId, ChatMessageSerializer.messagesToJson(messages));
    }

    @Override
    public void deleteMessages(Object memoryId) {
        rows.remove(memoryId);
    }
}

final class S4Memory {

    static MemoryAssistant build(ChatModel model, ChatMemoryStore store) {
        ChatMemoryProvider provider = userId -> MessageWindowChatMemory.builder()
                .id(userId)
                .maxMessages(10)
                .chatMemoryStore(store)
                .build();
        return AiServices.builder(MemoryAssistant.class)
                .chatModel(model)
                .chatMemoryProvider(provider)
                .build();
    }

    static ChatMemory singleUser() {
        return MessageWindowChatMemory.withMaxMessages(10);
    }
}

Running it with two users, "ana" and "ben", interleaved, and a scripted model:

Memory isolation between users
Your name is Ana
rows=[ana, ben]
messages in 3rd request=3
stored ana=4

The third request, Ana’s second question, reached the model with three messages (her first question, its reply and the new question), not Ben’s, and the store holds one JSON row per memory id. After Ana’s second exchange the store has her four messages. Replace the ConcurrentHashMap with a table keyed by memory id for a durable version; long-term facts that should outlive the window belong in a store or vector index instead, as discussed in Agent Memory & Semantic Cache. Official page: LangChain4j — Chat memory.

Structured outputs

An AI Service method can return more than String. LangChain4j maps the model’s reply to the declared return type: primitives and wrappers, enum (the model must pick a constant), List / Set, records and POJOs, and Result<T>, which wraps the parsed value together with the token usage, the finish reason, the RAG sources and the tool executions of the call. The mechanism depends on the provider, from most to least reliable: a JSON schema sent as a dedicated request attribute; JSON mode; and, for providers with neither, instructions appended to the prompt.

Three return types for DocsAssistant’s metadata extraction
import com.fasterxml.jackson.annotation.JsonProperty;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.service.AiServices;
import dev.langchain4j.service.Result;
import dev.langchain4j.service.UserMessage;

import java.util.List;

enum Audience { BEGINNER, PRACTITIONER, EXPERT }

record PageMetadata(@JsonProperty(required = true) String title, Audience audience, List<String> keywords) { }

interface MetadataExtractor {

    @UserMessage("Extract metadata from this documentation page:\n{{it}}")
    PageMetadata extract(String pageText);

    @UserMessage("Classify the audience of this page: {{it}}")
    Audience audienceOf(String pageText);

    @UserMessage("Extract metadata from this documentation page:\n{{it}}")
    Result<PageMetadata> extractWithUsage(String pageText);
}

final class S5Structured {

    static MetadataExtractor build(ChatModel model) {
        return AiServices.create(MetadataExtractor.class, model);
    }
}

With a model that advertises RESPONSE_FORMAT_JSON_SCHEMA (the scripted one in this run), the request carries the schema, and the prompt is left alone. The enum became JsonEnumSchema with its constants:

Request sent to a model that supports JSON schema
response format sent=ResponseFormat { type = JSON, jsonSchema = JsonSchema { name = "PageMetadata", rootElement = JsonObjectSchema {description = null, properties = {title=JsonStringSchema {description = null }, audience=JsonEnumSchema {description = null, enumValues = [BEGINNER, PRACTITIONER, EXPERT] }, keywords=JsonArraySchema {description = null, items = JsonStringSchema {description = null } }}, required = [title], additionalProperties = null, definitions = {} } } }

With a model that does not advertise it, response format stays null and the library appends the format instructions to the user message instead. Same interface, same call, different wire behaviour:

The user message sent to a model without JSON-schema support
UserMessage { contents = [TextContent { text = "Extract metadata from this documentation page:
...page text...
You must answer strictly in the following JSON format: {
"title": (type: string),
"audience": (type: enum, must be one of [BEGINNER, PRACTITIONER, EXPERT]),
"keywords": (type: array of string)
}" }] }

The three calls returned PageMetadata[title=Tool Design, audience=PRACTITIONER, keywords=[tools, schemas]], EXPERT and, through Result<PageMetadata>, the keyword list [tools, schemas] plus the call’s metadata. By default the generated schema marks no field as required; @JsonProperty(required = true), as on title above, adds it to the schema’s required list (see required = [title] in the output). Local 8B models follow schemas less reliably than hosted ones, which is one reason Result<T> and post-parse validation are worth having. The wire formats behind all this are in Structured Output & Tool Calling; the Python equivalents are in Structured Output. Official page: LangChain4j — Structured outputs.

Streaming

Returning a TokenStream from an AI Service method turns the call into a stream. Nothing happens until start(); callbacks deliver each token (onPartialResponse), retrieved RAG content (onRetrieved), tool activity (beforeToolExecution, onToolExecuted), the final ChatResponse (onCompleteResponse) and failures (onError). The service must be built with a StreamingChatModel. Blocking on a CompletableFuture at the end is the simplest way to wait, and is what a main method or a test does; a web handler would forward tokens instead. The same interface also supports cancelling a stream from a callback through the StreamingHandle in onPartialResponseWithContext.

Since LangChain4j 1.20, AI Service methods can also be non-blocking and reactive by changing only the return type: CompletableFuture<T> or CompletionStage<T> for one answer, Flow.Publisher<String> for the answer streamed in chunks, and Flow.Publisher<AiServiceStreamingEvent> for every event of the interaction (partial responses, tool calls, retrieved content, the final response). The publishers are cold: nothing runs until you subscribe, and each subscription starts a new interaction. The official page marks these asynchronous and reactive APIs experimental, while TokenStream is unaffected. The reactive types are plain JDK types, so no library is imposed; the langchain4j-reactor module adds Mono and Flux adapters, and Mutiny’s Uni and Multi are not supported yet.

The same streaming question as a TokenStream and as a Flow.Publisher
import dev.langchain4j.model.chat.StreamingChatModel;
import dev.langchain4j.model.chat.response.ChatResponse;
import dev.langchain4j.service.AiServices;
import dev.langchain4j.service.TokenStream;

import java.util.concurrent.CompletableFuture;
import java.util.concurrent.Flow;

interface StreamingAssistant {
    TokenStream chat(String message);
}

interface ReactiveAssistant {
    Flow.Publisher<String> chatStreaming(String message);
}

final class S6Streaming {

    static ChatResponse tokenStream(StreamingChatModel model, String question) {
        StreamingAssistant assistant = AiServices.create(StreamingAssistant.class, model);
        CompletableFuture<ChatResponse> done = new CompletableFuture<>();
        assistant.chat(question)
                .onPartialResponse(token -> System.out.print(token + "|"))
                .onCompleteResponse(done::complete)
                .onError(done::completeExceptionally)
                .start();
        return done.join();
    }

    static void publisher(StreamingChatModel model, String question) throws InterruptedException {
        ReactiveAssistant assistant = AiServices.create(ReactiveAssistant.class, model);
        CompletableFuture<Void> done = new CompletableFuture<>();
        assistant.chatStreaming(question).subscribe(new Flow.Subscriber<String>() {
            @Override public void onSubscribe(Flow.Subscription s) { s.request(Long.MAX_VALUE); }
            @Override public void onNext(String chunk) { System.out.print(chunk + "|"); }
            @Override public void onError(Throwable t) { done.completeExceptionally(t); }
            @Override public void onComplete() { done.complete(null); }
        });
        done.join();
    }
}

Running both against scripted models that emit Use `, `Pg, Vector and . as four chunks:

TokenStream and Flow.Publisher, each chunk followed by a bar
Use |Pg|Vector|.|
final=Use PgVector.
Use |Pg|Vector|.|

The last line of the streaming run shows the failure mode. The publisher path needs a StreamingChatModel that implements the publisher-based doChat; the callback-only fake does not, so subscribing fails immediately with a clear message instead of blocking a thread:

Publisher method on a callback-only model
callback-only model: AsyncNotSupportedException -- doChat() is not implemented by docs.ScriptedStreamingChatModel

That is not only a testing detail. Inspecting the compiled 1.20.2 classes, OpenAiStreamingChatModel implements the publisher-based doChat while OllamaStreamingChatModel implements only the callback one, so for DocsAssistant on Ollama use TokenStream (which works with every provider), and reach for Flow.Publisher when the provider you use supports it. The same holds for chatAsync and CompletableFuture methods, which need doChatAsync: the same experiment with CompletableFuture<String> failed with doChatAsync() is not implemented. The official page lists, layer by layer, which components (chat memory, stores, guardrails, tools, RAG, MCP, the HTTP client) have asynchronous counterparts, and notes that tools run concurrently in the asynchronous and reactive modes and that streamingBufferSize bounds the event buffer. The Python side of this topic is in Streaming. Official pages: LangChain4j — Response streaming and LangChain4j — Non-blocking and reactive.

Guardrails

Guardrails validate what goes into and comes out of an AI Service. They exist only at the AI Service level (a bare ChatModel has none), and each one should check exactly one thing. An InputGuardrail runs after RAG and before the model is called; an OutputGuardrail runs on the model’s answer. The helper methods on the interfaces express the outcome:

Outcome Effect

success()

Validation passed; the next guardrail runs.

successWith(…​)

Passed, and the message or answer is rewritten for the next step.

failure(…​)

Failed, but the remaining guardrails still run so all problems are collected.

fatal(…​)

Failed and stopped: an InputGuardrailException or OutputGuardrailException reaches the caller.

reprompt(…​) / retry(…​) (output only)

Ask the model again with a corrective message, or simply ask again.

Guardrails can be set on the builder (inputGuardrails, outputGuardrails, or the …​Classes variants), or with @InputGuardrails / @OutputGuardrails annotations on a method or the interface; the builder wins over a method annotation, which wins over the interface annotation. Order matters: put cheap checks first and any guardrail that calls another model last.

DocsAssistant uses one of each. The input guardrail blocks an obvious prompt-injection phrase without spending a model call. The output guardrail insists that answers cite a documentation page and, when they do not, re-prompts once:

One input and one output guardrail
import dev.langchain4j.data.message.AiMessage;
import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.guardrail.InputGuardrail;
import dev.langchain4j.guardrail.InputGuardrailResult;
import dev.langchain4j.guardrail.OutputGuardrail;
import dev.langchain4j.guardrail.OutputGuardrailResult;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.service.AiServices;

import java.util.Locale;

/** Blocks obvious prompt-injection phrases before the model is called. */
final class NoInstructionOverride implements InputGuardrail {

    @Override
    public InputGuardrailResult validate(UserMessage userMessage) {
        String text = userMessage.singleText().toLowerCase(Locale.ROOT);
        return text.contains("ignore previous instructions")
                ? fatal("The message tries to override the system prompt")
                : success();
    }
}

/** Requires every answer to cite a documentation page; if not, asks the model once more. */
final class MustCiteAPage implements OutputGuardrail {

    @Override
    public OutputGuardrailResult validate(AiMessage responseFromLLM) {
        return responseFromLLM.text().contains(".adoc")
                ? success()
                : reprompt("The answer cites no page", "Repeat the answer and cite the .adoc page it comes from.");
    }
}

final class S7Guardrails {

    static DocsAssistant build(ChatModel model) {
        return AiServices.builder(DocsAssistant.class)
                .chatModel(model)
                .inputGuardrails(new NoInstructionOverride())
                .outputGuardrails(new MustCiteAPage())
                .build();
    }
}

The scripted model first answers without a page, then with one. The second call is the reprompt, so the model was called twice for one question; the injection attempt never reached the model, so the counter stays at two:

Guardrail behaviour
Use pgvector; see database/vector-rag/postgresql-pgvector.adoc.
model calls=2
blocked: The guardrail docs.NoInstructionOverride failed with this message: The message tries to override the system prompt
model calls after blocked message=2

A string match is a demonstration, not a defence: real injection protection combines least-privilege tools, approval gates and, for untrusted content, isolation. That larger picture is in Agent Guardrails & Safety and Guardrails & Security. Unit-testing helpers for guardrails are in the langchain4j-test module. Official page: LangChain4j — Guardrails.

Observability

LangChain4j exposes two complementary hooks.

AI Service listeners observe an invocation of an AI Service from start to finish. Each event carries an InvocationContext (invocation id, interface and method name, method arguments, memory id, timestamp), which lets you correlate everything that belongs to one call. The events are AiServiceStartedEvent, AiServiceRequestIssuedEvent, AiServiceResponseReceivedEvent, AiServiceCompletedEvent, AiServiceErrorEvent, ToolExecutedEvent and the input and output guardrail events. Each has a listener interface, and registerListeners(…​) on the builder attaches them. Chat model listeners (ChatModelListener) sit on the model itself: they see every raw request, response and error, including the extra model calls made by the tool loop and by guardrail reprompts, with the OpenTelemetry generative-AI attributes (model, parameters, token usage, finish reason). The provider’s builder takes them through listeners(…​).

Two AI Service listeners and one model listener
import dev.langchain4j.model.chat.listener.ChatModelErrorContext;
import dev.langchain4j.model.chat.listener.ChatModelListener;
import dev.langchain4j.model.chat.listener.ChatModelRequestContext;
import dev.langchain4j.model.chat.listener.ChatModelResponseContext;
import dev.langchain4j.observability.api.event.AiServiceCompletedEvent;
import dev.langchain4j.observability.api.event.ToolExecutedEvent;
import dev.langchain4j.observability.api.listener.AiServiceCompletedListener;
import dev.langchain4j.observability.api.listener.ToolExecutedEventListener;

import java.util.List;

final class LoggingCompletedListener implements AiServiceCompletedListener {

    final List<String> log = new java.util.concurrent.CopyOnWriteArrayList<>();

    @Override
    public void onEvent(AiServiceCompletedEvent event) {
        var ctx = event.invocationContext();
        log.add("completed " + ctx.interfaceName() + "." + ctx.methodName() + " memoryId=" + ctx.chatMemoryId());
    }
}

final class LoggingToolListener implements ToolExecutedEventListener {

    final List<String> log = new java.util.concurrent.CopyOnWriteArrayList<>();

    @Override
    public void onEvent(ToolExecutedEvent event) {
        log.add("tool " + event.request().name() + " -> " + event.resultText());
    }
}

/** Sees every raw model call, including the ones made by tool loops and guardrail reprompts. */
final class CountingModelListener implements ChatModelListener {

    int requests;
    int responses;

    @Override public void onRequest(ChatModelRequestContext ctx) { requests++; }
    @Override public void onResponse(ChatModelResponseContext ctx) { responses++; }
    @Override public void onError(ChatModelErrorContext ctx) { }
}

Registered on the tool-calling service from before and run with the scripted model:

Registering the listeners
var completed = new LoggingCompletedListener();
var toolLog = new LoggingToolListener();
var counting = new CountingModelListener();
var observed = AiServices.builder(DocsAssistant.class)
        .chatModel(model)                       // built with .listeners(List.of(counting))
        .tools(new DocsTools(title -> false))
        .registerListeners(completed, toolLog)
        .build();
observed.chat("Find pgvector docs");
What the listeners recorded
completed docs.DocsAssistant.chat memoryId=default
tool searchDocs -> ["ai/langchain/tools.adoc -- Tools"]
raw model calls: requests=2 responses=2

One service call, one completed event, one tool event and two raw model calls (the tool request and the final answer): the two levels answer different questions. Forward the events to your tracing or metrics backend rather than collecting them in a list; the Python counterpart of that backend is described in Observability with LangSmith. The vendor-neutral view is in Observability for Agents. Official page: LangChain4j — Observability.

Spring Boot and Quarkus

Everything above works in a plain main method. Frameworks add configuration by properties and dependency injection.

Spring Boot. The starters follow a naming convention: langchain4j-{integration}-spring-boot4-starter for Spring Boot 4 and langchain4j-{integration}-spring-boot-starter for Spring Boot 3, all at 1.20.2-beta30. An integration starter (for example langchain4j-ollama-spring-boot4-starter) creates the model from properties such as langchain4j.ollama.chat-model.base-url and langchain4j.ollama.chat-model.model-name, and the general langchain4j-spring-boot4-starter adds declarative AI Services: annotate an interface with @AiService and the starter scans for it, builds it from the ChatModel, StreamingChatModel, ChatMemory, ChatMemoryProvider, ContentRetriever, RetrievalAugmentor and ToolProvider beans it finds, adds every @Tool method of your @Component and @Service beans, and registers the result as a bean. With several candidate beans of the same type, switch to wiringMode = EXPLICIT and name the beans. We resolved both starters with Maven and compiled this against them:

A declarative AI Service for Spring Boot 4
import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.spring.AiService;

@AiService
interface SbAssistant {

    @SystemMessage("You are DocsAssistant. Answer only from this site's documentation.")
    String chat(String userMessage);
}

The property values are ordinary Spring configuration, so the API key of a hosted model belongs in ${OPENAI_API_KEY} style placeholders resolved from the environment, as the official page shows. This page does not start a Spring application; the Spring Boot side of tools and MCP servers is in Spring Boot MCP Servers. Spring AI, the other Java option, has its own Spring AI sub-section in this site’s AI section.

Quarkus. Quarkus uses the Quarkiverse extension io.quarkiverse.langchain4j:quarkus-langchain4j-*, maintained separately (1.14.0 at the time of writing; for example quarkus-langchain4j-ollama for Ollama). Its declarative form is @RegisterAiService on an interface, injected with CDI, and it is what the official LangChain4j Quarkus page points to. The extension and the Quarkus platform’s version pinning are described in Extensions & the Platform. Official pages: LangChain4j — Spring Boot integration, LangChain4j — Quarkus integration and Quarkus LangChain4j — AI Services.

RAG in LangChain4j

Embedding models, EmbeddingStore, EmbeddingStoreIngestor, EmbeddingStoreContentRetriever, PgVectorEmbeddingStore and the RetrievalAugmentor that plugs retrieval into an AI Service are already documented, with a PostgreSQL example for DocsAssistant, in Integrating with LangChain: LangChain4j. This page does not repeat them. What this page adds to that one is the surrounding machinery: the onRetrieved callback and Result.sources() expose the retrieved content, an input guardrail runs after retrieval, and retrieval can be exposed to the model as a tool instead of being injected on every call. The Python side is bridged in Retrieval & the RAG Bridge.

From LangChain to LangChain4j

The concepts line up more than the names do:

LangChain (Python) LangChain4j Difference to watch

init_chat_model, ChatOllama

OllamaChatModel, ChatModel

Explicit provider class and builder; no string-based model factory in the core.

ChatPromptTemplate

@SystemMessage, @UserMessage, {{var}}, @V

Prompts live on the interface; double-brace variables.

create_agent, @tool

AiServices.builder, @Tool, @P

The tool loop is part of every AI Service; there is no separate agent object until the agentic module.

with_structured_output, response_format

Return type of the AI Service method, Result<T>

The schema is derived from the Java type.

Checkpointer and thread_id

ChatMemoryProvider, @MemoryId, ChatMemoryStore

Memory is a message window, not graph state.

Middleware, HumanInTheLoopMiddleware

Guardrails, listeners, application code

No built-in approval step; write it into the tool.

astream, stream_mode

TokenStream, Flow.Publisher

Reactive AI Services are experimental in 1.20.

LangSmith tracing

ChatModelListener, AI Service listeners

Wire them to your own telemetry backend.

The LangGraph-style workflows, supervisors and MCP client have their own page: LangChain4j Agents & MCP. For the JVM-side choice against Spring AI and other frameworks, see Frameworks Landscape.

References