Spring AI

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

This sub-section was written against Spring AI 2.0.1, Spring Boot 4.1.x (4.1.1) and Java 21, with the full list in the version baseline below, all read from Maven Central, the Spring AI reference and its upgrade notes on 2026-09-29. Where an individual page and that baseline ever disagree after a later update, the baseline is the one to trust. Every Java snippet on the pages was compiled against the real 2.0.1 jars in a scratch Maven project, and most were run against a scripted ChatModel, a WireMock stand-in for the provider API or a local Ollama container; each page says in its lead paragraph what was run and what was compiled only.

Spring AI is the Spring project for building applications on large language models. It gives Java developers the same things Spring gives them for databases and messaging: portable interfaces for each kind of model, Spring Boot starters that wire a provider from a few properties, and the usual Spring idioms — dependency injection, Micrometer observations, Spring Security, Testcontainers — applied to prompts, tools, memory and retrieval. On top of the interfaces sits ChatClient, a fluent API with a chain of advisors that add conversational memory, retrieval, the tool-calling loop and guardrails around each call. The project also ships a Model Context Protocol (MCP) client and server, so a Spring application can use remote tools or publish its own.

This page is the map of the sub-section: the layers of the framework, the version table, the provider families, a suggested reading order, the running example and the bibliography that all the pages draw on.

The abstraction layers

The Spring AI abstraction layers, from top to bottom: your application (DocsAssistant controllers, @Tool beans and record types); ChatClient with its advisor chain of memory, RAG, tool-calling, guardrail and logging advisors; the portable interfaces ChatModel, EmbeddingModel, ImageModel, ModerationModel, TranscriptionModel, TextToSpeechModel, VectorStore and ChatMemory; the provider modules and auto-configuration selected by starters and spring.ai properties; and the model runtimes and stores. A column on the right lists the cross-cutting concerns: observability, resilience, MCP, security and quality.

Application code talks to the upper layers only. A starter brings a provider module and its auto-configuration, the auto-configuration reads spring.ai.* properties and registers beans for the portable interfaces, and the code injects those interfaces or a ChatClient.Builder. Changing provider is therefore mostly a matter of changing a dependency and a few properties. The portable interfaces in the 2.0.1 jars are:

Interface What it does Page

ChatModel (with StreamingChatModel)

Sends a Prompt of messages to a chat model and returns a ChatResponse, whole or as a Flux.

ChatModel & ChatClient

EmbeddingModel

Turns text into vectors for semantic search.

Integrating with Spring AI: embeddings

ImageModel

Generates images from an ImagePrompt.

Image Generation

TranscriptionModel

Speech to text; in 2.0 it extends StreamingTranscriptionModel.

Audio Transcription & Speech

TextToSpeechModel (with StreamingTextToSpeechModel)

Text to speech from a TextToSpeechPrompt. Older documentation and books call this interface SpeechModel; the 2.0.1 jars have no type of that name.

Audio Transcription & Speech

ModerationModel

Classifies text against content-safety categories.

Security & Guardrails

VectorStore

Stores documents with their embeddings and runs similarity searches with metadata filters.

Integrating with Spring AI: the VectorStore API

ChatModel, EmbeddingModel, ImageModel, TranscriptionModel, TextToSpeechModel and ModerationModel all extend the generic Model<TReq, TRes> interface in org.springframework.ai.model. ChatMemory sits beside them as the store of conversation history (Chat Memory). The class below uses two of the interfaces and never names a provider; the starter on the classpath decides which models answer.

A provider-neutral DocsAssistant service
package com.example.docsassistant;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.embedding.EmbeddingModel;
import org.springframework.stereotype.Service;

@Service
public class PortableDocsAssistant {

    private final ChatClient chatClient;
    private final EmbeddingModel embeddingModel;

    // The builder and the embedding model come from whichever starter is on the
    // classpath (Ollama here); this class never names a provider.
    public PortableDocsAssistant(ChatClient.Builder builder, EmbeddingModel embeddingModel) {
        this.chatClient = builder
                .defaultSystem("You answer questions about the Irurueta documentation.")
                .build();
        this.embeddingModel = embeddingModel;
    }

    public String ask(String question) {
        return chatClient.prompt()
                .user(question)
                .call()
                .content();
    }

    public int embeddingDimensions() {
        return embeddingModel.dimensions();
    }
}

Source: Spring AI — ChatClient API and Spring AI — Chat Model API.

Version baseline

Every version below was read from Maven Central and the official documentation on 2026-09-29. Each page repeats the subset it uses in its own lead paragraph.

Area Versions

Spring AI

2.0.1, the current stable release, managed by org.springframework.ai:spring-ai-bom:2.0.1. 2.0.0 went GA on 2026-06-12 (release announcement). A 2.1 milestone exists; nothing in this sub-section depends on it.

Spring Boot and Java

Spring AI 2.0.x supports Spring Boot 4.0.x and 4.1.x; the examples use Spring Boot 4.1.1, Spring Framework 7, Jackson 3 (tools.jackson.*) and JSpecify null-safety annotations. Java 17 is the minimum; every example uses Java 21 with Maven.

MCP

The MCP Java SDK (io.modelcontextprotocol.sdk) 2.0.0 as resolved by Spring AI 2.0.1 (2.0.1 is the latest on Maven Central); the community org.springaicommunity:mcp-server-security and mcp-client-security 0.1.14 for OAuth and API keys.

Agents

The community org.springaicommunity:spring-ai-agent-utils 0.12.0 and Embabel 1.5.2 (com.embabel.agent:embabel-agent-starter).

Models and stores

Ollama with llama3.1:8b for chat and nomic-embed-text (768 dimensions) for embeddings, PostgreSQL 17 with pgvector 0.8.6, and ollama/ollama:0.34.4 for Testcontainers — the same as the rest of the AI section. The hosted models used elsewhere (claude-sonnet-5, claude-opus-5-5, claude-haiku-4-5, claude-fable-5-1, gpt-6-sol) appear only in provider matrices and option examples.

The MCP versions come with a skew that matters when you mix Spring AI with the rest of this site. The protocol constants of the MCP Java SDK 2.0.x stop at the 2025-11-25 revision of the specification, while the site’s Model Context Protocol reference is written against the 2026-07-28 revision. A Spring AI 2.0.x server or client therefore speaks 2025-11-25 (the initialize handshake, Streamable HTTP with sessions), and the features that exist only in 2026-07-28 — the stateless request core, server/discover, multi-round-trip requests — are not available yet. MCP Server and MCP Client spell out the consequences.

Material written for Spring AI 1.x, including every book in the bibliography, describes an older API. The differences that matter most, each flagged where it appears on the pages and collected on Migrating from 1.x to 2.x:

  • options classes are immutable builders changed with mutate(), and the .options segment was dropped from property paths (spring.ai.openai.chat.options.model became spring.ai.openai.chat.model);

  • ChatClient always registers the ToolCallingAdvisor, which runs the tool loop, and ChatModel.call() no longer executes tools by itself;

  • chat-memory advisors need a conversation ID, PromptChatMemoryAdvisor is gone, and tool messages are not stored;

  • the FunctionCallback API became ToolCallback, and SpringBeanToolCallbackResolver and toolNames() were removed;

  • the MCP annotations moved to org.springframework.ai.mcp.annotation, and the MCP Java SDK moved to 2.0;

  • the tool span is named execute_tool <tool-name> instead of tool_call <tool-name>.

Provider families

Spring AI groups providers by how they are reached. The table is a summary; the full starter list, and what each provider supports, is on Setup & Starters and the reference’s Chat Models Comparison.

Family Starters

Local runtimes

spring-ai-starter-model-ollama (the DocsAssistant default), local ONNX embeddings with spring-ai-starter-model-transformers, and OpenAI-compatible local servers (Docker Model Runner, vLLM, llama.cpp, LM Studio, Foundry Local) through the OpenAI starter — see Local Models

OpenAI and OpenAI-compatible APIs

spring-ai-starter-model-openai, with spring.ai.openai.base-url pointed at a compatible endpoint; chat, embeddings, images, transcription, speech and moderation

Anthropic, Google, Mistral AI, DeepSeek

spring-ai-starter-model-anthropic, spring-ai-starter-model-google-genai, spring-ai-starter-model-mistral-ai, spring-ai-starter-model-deepseek

Cloud platforms

spring-ai-starter-model-bedrock-converse and spring-ai-starter-model-bedrock for Amazon Bedrock

Specialised

spring-ai-starter-model-elevenlabs (speech), spring-ai-starter-model-stability-ai (images)

With more than one starter of the same kind on the classpath, choose the active one with spring.ai.model.chat, spring.ai.model.embedding, spring.ai.model.image, spring.ai.model.audio.speech, spring.ai.model.audio.transcription or spring.ai.model.moderation.

The running example: DocsAssistant

Every page extends DocsAssistant, a Spring Boot 4.1 service that answers questions about this site’s documentation. It uses the same corpus, Ollama models and pgvector store as Integrating with Spring AI, so the two sections describe one application, and it is the Java counterpart of the DocsAssistant built in LangChain, AI Agents and Model Context Protocol. Each page adds one capability.

Capability What DocsAssistant does Page

Project skeleton

Starts with the BOM, the Ollama and pgvector starters, and Docker Compose or Testcontainers services.

Setup & Starters

Chat endpoint

POST /api/chat answers a question through ChatClient.

ChatModel & ChatClient, Chat Options

Prompt templates

Keeps its system and user prompts in .st resources.

Prompts & Templates

Advisors

Wraps each call with logging and a canary-word check.

Advisors

Structured metadata

POST /api/extract returns a DocMetadata record.

Structured Output

Tools

Calls searchDocs, and openIssue guarded by @PreAuthorize and an approval step.

Tool Calling, Tool Calling Advisor & Tool Search

Per-user memory

Remembers each user’s conversation in JDBC, keyed by the authenticated user and session.

Chat Memory, Chat Memory Repositories

Streaming

GET /api/chat/stream streams the answer as server-sent events.

Streaming & Reactive

MCP server and client

Exposes its tools at /mcp, and consumes GitHub’s MCP server with a token from the environment.

MCP Server, MCP Client

Voice and images

POST /api/voice/ask takes a spoken question and answers with speech; POST /api/images/describe describes a screenshot; illustrations can be generated.

Audio Transcription & Speech, Multimodality, Image Generation

Metrics and tracing

Publishes gen_ai.* metrics to Prometheus and traces with execute_tool spans.

Observability

Evaluator tests and security

Tests its answers with evaluators and stubbed models, and guards its endpoints, memory and tools.

Evaluation, Testing, Security & Guardrails

Agents, local models and deployment

Adds agent workflows, runs on local runtimes, and ships to Kubernetes with retries, limits and secrets.

Agentic Patterns, Local Models, Production & Deployment

Reading path

Read the pages in the order of the navigation on a first pass; the shorter paths below are for readers who already know part of the material.

New to Spring AI

  1. Setup & Starters to create the project.

  2. ChatModel & ChatClient, Prompts & Templates and Chat Options.

  3. Advisors and Structured Output.

  4. Tool Calling and Chat Memory.

Upgrading from Spring AI 1.x, or coming from the books

  1. Migrating from 1.x to 2.x for the old-to-new tables.

  2. Tool Calling Advisor & Tool Search for the new tool loop.

  3. Chat Memory Repositories for the JDBC schema change.

Building RAG and agents

  1. Retrieval & RAG Bridge, then the vector-store reference in Integrating with Spring AI.

  2. Agentic Patterns.

  3. MCP Client and MCP Server.

Taking a service to production

  1. Streaming & Reactive.

  2. Security & Guardrails.

  3. Observability, Evaluation and Testing.

  4. Production & Deployment and Local Models.

Voice and vision

  1. Multimodality, Audio Transcription & Speech and Image Generation.

What’s covered

The pages are grouped the same way as the navigation.

Getting started

  • Setup & Starters — the BOM, starter naming, flattened properties, provider selection, Docker Compose and Testcontainers services, and Spring Initializr.

Core API

  • ChatModel & ChatClient — the two levels of the chat API, builder defaults, call() and stream(), content(), entity() and chatResponse(), usage metadata and several clients.

  • Prompts & Templates — Prompt, message roles, PromptTemplate, .st resources and custom `TemplateRenderer`s.

  • Chat Options — portable and provider options, immutable builders and mutate(), per-request options, and reasoning settings.

  • Advisors — the advisor chain and its ordering constants, CallAdvisor and StreamAdvisor, recursive advisors and custom advisors.

  • Structured Output — entity(), the output converters, native structured output and validation with self-correction.

Tools and memory

  • Tool Calling — @Tool and @ToolParam, the tool-callback types, ToolContext, returnDirect and exception handling.

  • Tool Calling Advisor & Tool Search — the automatic tool loop, a manual loop, ToolSearchToolCallingAdvisor and spring.ai.tools.limits.*.

  • Chat Memory — MessageWindowChatMemory, the mandatory conversation ID, MessageChatMemoryAdvisor and summarising long histories.

  • Chat Memory Repositories — JDBC, Cassandra, MongoDB, Neo4j and Redis, a custom repository, retention and GDPR erasure.

Streaming and MCP

  • Streaming & Reactive — Flux responses, server-sent-event endpoints, streaming with tools and memory, cancellation and virtual threads.

  • MCP Client — connections, mcp-servers.json, SyncMcpToolCallbackProvider, McpClientCustomizer, client annotations and the GitHub MCP server.

  • MCP Server — protocols, @McpTool, @McpResource, @McpPrompt and @McpComplete, exposing @Tool beans, OAuth and the Inspector.

Multimodal and voice

  • Multimodality — Media input, vision questions and a provider matrix.

  • Audio Transcription & Speech — TranscriptionModel, TextToSpeechModel with OpenAI and ElevenLabs, and a voice round trip.

  • Image Generation — ImageModel, ImagePrompt, OpenAI and Stability AI options, and storing the results.

Operations and quality

  • Observability — Micrometer observations, gen_ai.* metrics, Prometheus and Grafana, OpenTelemetry traces and the logging flags.

  • Evaluation — RelevancyEvaluator, FactCheckingEvaluator, LLM-as-a-judge, custom evaluators and golden-set tests.

  • Testing — stubbed models, WireMock, Testcontainers Ollama and slice tests.

  • Security & Guardrails — Spring Security, memory isolation, @PreAuthorize on tools, SafeGuardAdvisor, ModerationModel and document-level access.

Agents, local models and production

  • Agentic Patterns — the effective-agents workflows, spring-ai-agent-utils and Embabel.

  • Local Models — Ollama, Docker Model Runner, OpenAI-compatible local servers and ONNX embeddings.

  • Production & Deployment — retries, rate limits, caching, timeouts, virtual threads, Kubernetes, secrets and cost.

  • Migrating from 1.x to 2.x — the upgrade notes as old-to-new tables and the OpenRewrite recipes.

RAG bridge

  • Retrieval & RAG Bridge — the RAG advisors around a ChatClient, with deep links into the vector-store reference.

Cheat sheet

Related material elsewhere on this site

The vector-store side of Spring AI is not repeated here. The canonical page is Integrating with Spring AI, whose sections on embeddings, the VectorStore API, filter expressions, the ETL pipeline, the RAG advisors, modular RAG, vector-backed chat memory, retrieval tools and testing cover it, and Spring Boot: Redis as a Vector Database covers Redis. Spring Boot itself is documented in Spring Boot. Framework-neutral background lives in the sibling sub-sections: LLM Foundations & Prompting for models and prompting, AI Agents for agent theory, Model Context Protocol for the protocol, Running LLMs Locally for local runtimes and LangChain for the LangChain4j alternative on the JVM, and RAG Systems in Production for running DocsAssistant as a production RAG service. Conversational Channels covers exposing it through chat channels; Hugging Face, Voice Agents, LLMOps & Evaluation and AI Security & Responsible AI are separate sub-sections of this AI section.

Bibliography

The following sources were consulted while preparing this sub-section’s pages. Every class, annotation, method, starter, property and metric name was checked against the Spring AI 2.0.1 jars (with javap and the configuration metadata in the starters), the official documentation or the sources listed here on 2026-09-29 — never copied from a book. Every documentation URL cited on the pages of this sub-section is listed here once.

Requester-provided books, consulted for concepts only and never the source of an example. All three target an older Spring AI line (1.0.x, 0.8.x or a 1.0 milestone), so where they differ from the Spring AI documentation, the documentation is authoritative:

  • Walls, Craig. Spring AI in Action. Foreword by Rod Johnson. Manning Publications, 2026. ISBN 978-1-63343-611-4. Publisher page: https://www.manning.com/books/spring-ai-in-action. Official code repository: https://github.com/habuma/spring-ai-in-action-samples. Consulted as the backbone for the concepts of ChatClient prompts, evaluation, prompt templates and options, retrieval, chat memory, tools, MCP clients and servers, voice and images, observability, safeguarding and agents. It targets Spring Boot 3.5 and Spring AI 1.0.3; the options, memory, tool and MCP changes of 2.0 are flagged on the pages that cover them.

  • Parasuraman, Banu. Mastering Spring AI: The Java Developer’s Guide for Large Language Models and Generative AI. Apress (Springer Nature), 2024. ISBN 979-8-8688-1000-8 (print), 979-8-8688-1001-5 (ebook), DOI 10.1007/979-8-8688-1001-5. Publisher page: https://link.springer.com/book/10.1007/979-8-8688-1001-5. There is no Apress code repository; the author’s repositories are listed in the book’s appendix, for example https://github.com/banup-kubeforce/health-rag. Consulted for the Spring ecosystem around generative AI, prompting types, governance, LLMOps, and productionising concerns such as caching, retries and security. It targets the pre-1.0 0.8.x line, so its ChatClient interface, ImageClient, BeanOutputParser and FunctionCallbackWrapper are described on the pages only as history. The local copy consulted names "Edizioni Ensemble" and ISBN 9788868810009 in its file name; the publisher of record verified online is Apress, which is the record cited here.

  • Anthapu, Ravindranatha, and Siddhant Agarwal. Building Neo4j-Powered Applications with LLMs. Packt Publishing, Jun 2025. ISBN 978-1-83620-623-1. Publisher page: https://www.packtpub.com/en-us/product/building-neo4j-powered-applications-with-llms-9781836206231. Official code repository: https://github.com/PacktPublishing/Building-Neo4j-Powered-Applications-with-LLMs. Chapters 7 and 9 were consulted for a Spring AI ChatClient over Neo4j, LLM summaries stored as node embeddings, and a REST endpoint that triggers the augmentation.

packtpub.com answers automated fetches with an HTTP 403 response, but the publisher page above is a live, reachable page when opened in a browser.

Official documentation — Spring AI reference (https://docs.spring.io/spring-ai/reference/):

Official documentation — Spring and related projects:

Specifications and standards:

Samples, papers and engineering articles:

These books are consulted references, not the primary source of any example on this sub-section’s pages. On any discrepancy between a book and the Spring AI or Spring Boot official documentation, that documentation is authoritative.

References