Retrieval & RAG Bridge

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

Written against Spring AI 2.0.1, Spring Boot 4.1.x and Java 21 (verified 2026-09-29). Retrieval-augmented generation (RAG) means finding passages that are relevant to a question and giving them to the model along with it, so that the answer comes from your documents and not from the model’s memory. Spring AI’s vector-store, embedding, ETL and RAG-advisor surface is already documented in full on Integrating with Spring AI, which stays the reference for it. This page does not repeat that material. It is the bridge between the two: a map of the pieces, links to the sections of that page you will need, and the few facts that only show up when retrieval is combined with the rest of this sub-section, that is, memory, tools and follow-up questions. For DocsAssistant, it is where the chat endpoint stops answering from the model alone and starts answering from the documentation corpus in the pgvector store.

The landscape in one diagram

There are four ways to bring retrieved text into a call, and they can be combined. The diagram shows where each attaches to the ChatClient and what it needs.

flowchart TD U["User question + conversation id"] --> MEM["MessageChatMemoryAdvisor: recent turns (order MIN+200)"] MEM --> R{"Retrieval, choose one or more"} R --> QA["QuestionAnswerAdvisor: naive RAG, one search, one template"] R --> RA["RetrievalAugmentationAdvisor: modular RAG"] R --> VM["VectorStoreChatMemoryAdvisor: long-term memory from past turns"] R --> TL["searchDocs @Tool: the model decides when to search"] RA --> T1["Query transformers: compress, rewrite, translate"] T1 --> T2["Query expander: several queries"] T2 --> T3["DocumentRetriever: VectorStoreDocumentRetriever with filter"] T3 --> T4["Joiner, post-processors, augmenter"] QA --> VS[("VectorStore + EmbeddingModel")] T3 --> VS VM --> VS TL --> VS T4 --> LLM["ChatModel"] QA --> LLM VM --> LLM TL --> LLM ING["ETL: reader, splitter, embed, write"] -. "fills" .-> VS

The rows below say which piece to reach for, what it needs beyond the base starter, and where its reference is.

Table 1. RAG pieces, artifacts and reference sections
Piece	Artifact and package in 2.0.1	Reference
Embeddings	`EmbeddingModel` from a model starter, for example `spring-ai-starter-model-ollama`	Embeddings
`VectorStore`	a vector-store starter, for example `spring-ai-starter-vector-store-pgvector`	The `VectorStore` API
Filter expressions	`org.springframework.ai.vectorstore.filter` in `spring-ai-vector-store`	Filter expressions
ETL	`DocumentReader`, `DocumentTransformer`, `DocumentWriter`	ETL
`QuestionAnswerAdvisor`	`spring-ai-vector-store-advisor`, package `org.springframework.ai.chat.client.advisor.vectorstore`	RAG advisors
`RetrievalAugmentationAdvisor`, retrievers, transformers, augmenter	`spring-ai-rag`, packages under `org.springframework.ai.rag`	Modular RAG
`VectorStoreChatMemoryAdvisor`	`spring-ai-vector-store-advisor`, same package as `QuestionAnswerAdvisor`	Chat memory
Retrieval as a tool	`@Tool` on a method, no extra artifact	Tool calling
Testing RAG	Testcontainers and the evaluators	Testing

Two details in that table differ from older text, so check them when you copy code. First, in the 2.0.1 jars QuestionAnswerAdvisor and VectorStoreChatMemoryAdvisor are in org.springframework.ai.chat.client.advisor.vectorstore, in the artifact that 2.0 renamed from spring-ai-advisors-vector-store to spring-ai-vector-store-advisor. An import from org.springframework.ai.chat.client.advisor does not resolve. Second, RetrievalAugmentationAdvisor and its parts are in spring-ai-rag, an artifact that is managed by the BOM but not necessarily pulled in by a vector-store starter, so add it when you use the modular advisor. Both facts were read from the Spring AI 2.0.1 jars (with javap and the BOM) and confirmed by compiling the examples below.

Choosing between them

QuestionAnswerAdvisor is the shortest path: one similarity search with the user’s text, the passages appended to the prompt through a template. Use it when questions are self-contained.
RetrievalAugmentationAdvisor is the same idea with every stage replaceable: rewriting or compressing the query, expanding it into several, a retriever with a filter and a threshold, joining and post-processing the passages, and an augmenter that decides what to do when nothing was found. Use it for conversations, for access control through the filter and for anything you want to tune stage by stage.
VectorStoreChatMemoryAdvisor is not document retrieval. It searches past turns of a conversation for relevant ones, as long-term memory, which is the subject of Chat Memory.
A retrieval tool leaves the decision to the model: it searches when it thinks it needs to, possibly several times with different queries. It suits open-ended questions and agents, and costs an extra model round trip per search; the pattern is called agentic RAG and is described on Agentic RAG. The searchDocs tool of DocsAssistant is one.

Retrieval quality, meaning chunk size, hybrid search, reranking and metadata filters, is a question about the data and the store more than about Spring AI; it is covered on Retrieval Strategies, Hybrid Search and Reranking, Metadata Filtering and measured with the methods on Evaluating RAG Systems.

Retrieval inside the advisor chain

Retrieval advisors are advisors, so they take part in the ordering described on Advisors. That has a consequence that is easy to miss. The built-in memory advisor runs first (order HIGHEST_PRECEDENCE + 200), then the tool-calling advisor (+ 300), which calls the rest of the chain again on every pass of the tool loop. QuestionAnswerAdvisor and RetrievalAugmentationAdvisor both default to order 0, which places them inside that loop.

A test made the effect visible. A ChatClient with a searchDocs tool and a QuestionAnswerAdvisor over a mocked VectorStore answered one question in which the model called the tool once, so the loop ran twice:

Table 2. Vector-store searches for one question with one tool call
Retrieval advisor order	Searches
default (`0`)	2
`HIGHEST_PRECEDENCE + 250`	1

With the default order retrieval repeats on each pass, paying for another embedding and another search, and re-sending the passages. Give the retrieval advisor an order between memory and the tool loop so that it runs once per question:

DocsRagChat.java: memory, retrieval before the tool loop, and a tool

package com.example.docsassistant;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.client.advisor.MessageChatMemoryAdvisor;
import org.springframework.ai.chat.client.advisor.vectorstore.QuestionAnswerAdvisor;
import org.springframework.ai.chat.memory.ChatMemory;
import org.springframework.ai.vectorstore.SearchRequest;
import org.springframework.ai.vectorstore.VectorStore;
import org.springframework.core.Ordered;
import org.springframework.stereotype.Service;

/**
 * Memory, retrieval and tools on one client. The two orders on the retrieval advisor decide whether the vector
 * store is queried once per question or once per pass of the tool loop.
 */
@Service
public class DocsRagChat {

    private final ChatClient chatClient;

    public DocsRagChat(ChatClient.Builder builder, ChatMemory chatMemory, VectorStore vectorStore, DocsTools docsTools) {
        this.chatClient = builder
                .defaultTools(docsTools)
                .defaultAdvisors(
                        MessageChatMemoryAdvisor.builder(chatMemory).build(),          // MIN + 200
                        QuestionAnswerAdvisor.builder(vectorStore)
                                .searchRequest(SearchRequest.builder().topK(5).build())
                                .order(Ordered.HIGHEST_PRECEDENCE + 250)                // after memory, before the tool loop (MIN + 300)
                                .build())
                .build();
    }

    public String answer(String conversationId, String question) {
        return this.chatClient.prompt()
                .user(question)
                .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, conversationId))
                .call()
                .content();
    }
}

Source: Spring AI: Retrieval Augmented Generation and Spring AI: Advisors API for the advisor order.

The conversation id passed to answer must be derived from the authenticated principal (authentication.getName() + ":" + sessionId), never taken from client input alone, so one user cannot read another’s history. The class compiled and the ordering was tested with a scripted model and a mocked store (two tests, both passing). Whether you want retrieval once or on every pass is a design choice: once is cheaper, and per pass would let the model see fresh passages after a tool changes what is relevant. Choose it on purpose.

The order test that produced the table above

package com.example.docsassistant;

import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;

import java.util.List;

import org.junit.jupiter.api.Test;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.client.advisor.vectorstore.QuestionAnswerAdvisor;
import org.springframework.ai.document.Document;
import org.springframework.ai.vectorstore.SearchRequest;
import org.springframework.ai.vectorstore.VectorStore;
import org.springframework.core.Ordered;

class RagOrderTest {

    private int searchesWithOrder(Integer order) {
        VectorStore store = mock(VectorStore.class);
        when(store.similaritySearch(any(SearchRequest.class))).thenReturn(List.of(new Document("HNSW uses ef_search")));
        QuestionAnswerAdvisor.Builder builder = QuestionAnswerAdvisor.builder(store);
        if (order != null) {
            builder.order(order);
        }
        ScriptedChatModel model = new ScriptedChatModel(true)
                .then(ScriptedChatModel.toolCall("c1", "searchDocs", "{\"query\":\"ef_search\",\"component\":\"\"}"))
                .then(ScriptedChatModel.text("done"));
        String answer = ChatClient.builder(model)
                .defaultTools(new DocsTools(mock(VectorStore.class)))   // the tool's own store is not counted
                .defaultAdvisors(builder.build())
                .build().prompt().user("what is ef_search?").call().content();
        assertThat(answer).isEqualTo("done");
        assertThat(model.prompts).hasSize(2);
        int searches = org.mockito.Mockito.mockingDetails(store).getInvocations().size();
        return searches;
    }

    @Test
    void defaultOrderRetrievesOnEveryPassOfTheToolLoop() {
        assertThat(searchesWithOrder(null)).isEqualTo(2);
    }

    @Test
    void orderBeforeTheToolLoopRetrievesOnce() {
        assertThat(searchesWithOrder(Ordered.HIGHEST_PRECEDENCE + 250)).isEqualTo(1);
    }
}

Source: Spring AI: Advisors API.

A conversation over documents

A follow-up such as "and how do I raise it?" is a bad search query on its own, since "it" means nothing to the vector store. The modular advisor has a CompressionQueryTransformer for this: it asks a model to combine the history and the follow-up into a standalone query, which then goes to the retriever. DocsModularRag uses it, filters the search to public documents, refuses to answer when nothing is found, and returns the sources with the answer.

DocsModularRag.java: compress the follow-up, filter, refuse on empty, return sources

package com.example.docsassistant;

import java.util.List;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.client.ChatClientResponse;
import org.springframework.ai.chat.client.advisor.MessageChatMemoryAdvisor;
import org.springframework.ai.chat.client.advisor.vectorstore.QuestionAnswerAdvisor;
import org.springframework.ai.chat.memory.ChatMemory;
import org.springframework.ai.document.Document;
import org.springframework.ai.rag.advisor.RetrievalAugmentationAdvisor;
import org.springframework.ai.rag.generation.augmentation.ContextualQueryAugmenter;
import org.springframework.ai.rag.preretrieval.query.transformation.CompressionQueryTransformer;
import org.springframework.ai.rag.retrieval.search.VectorStoreDocumentRetriever;
import org.springframework.ai.vectorstore.VectorStore;
import org.springframework.ai.vectorstore.filter.FilterExpressionBuilder;
import org.springframework.core.Ordered;
import org.springframework.stereotype.Service;

/**
 * The modular advisor in a conversation: follow-up questions are condensed into standalone queries, the store is
 * searched with a filter, an empty result is refused instead of answered from memory, and the sources come back
 * with the answer.
 */
@Service
public class DocsModularRag {

    public record AnswerWithSources(String answer, List<String> sources) { }

    private final ChatClient chatClient;

    public DocsModularRag(ChatClient.Builder builder, ChatMemory chatMemory, VectorStore vectorStore) {
        var retrieval = RetrievalAugmentationAdvisor.builder()
                .queryTransformers(CompressionQueryTransformer.builder()
                        .chatClientBuilder(builder.clone())
                        .build())
                .documentRetriever(VectorStoreDocumentRetriever.builder()
                        .vectorStore(vectorStore)
                        .topK(5)
                        .similarityThreshold(0.35)
                        .filterExpression(() -> new FilterExpressionBuilder().eq("visibility", "public").build())
                        .build())
                .queryAugmenter(ContextualQueryAugmenter.builder().allowEmptyContext(false).build())
                .order(Ordered.HIGHEST_PRECEDENCE + 250)   // once per question, before the tool loop
                .build();
        this.chatClient = builder
                .defaultAdvisors(MessageChatMemoryAdvisor.builder(chatMemory).build(), retrieval)
                .build();
    }

    public AnswerWithSources ask(String conversationId, String question) {
        ChatClientResponse response = this.chatClient.prompt()
                .user(question)
                .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, conversationId))
                .call()
                .chatClientResponse();
        String answer = response.chatResponse().getResult().getOutput().getText();
        @SuppressWarnings("unchecked")
        List<Document> documents = (List<Document>) response.context()
                .getOrDefault(RetrievalAugmentationAdvisor.DOCUMENT_CONTEXT, List.of());
        return new AnswerWithSources(answer, documents.stream()
                .map(d -> String.valueOf(d.getMetadata().get("source")))
                .distinct()
                .toList());
    }

    /** The same idea with the naive advisor: its context key differs. */
    static List<Document> retrievedBy(ChatClientResponse response) {
        @SuppressWarnings("unchecked")
        List<Document> documents = (List<Document>) response.context()
                .getOrDefault(QuestionAnswerAdvisor.RETRIEVED_DOCUMENTS, List.of());
        return documents;
    }
}

Source: Spring AI: Retrieval Augmented Generation.

Running it with a scripted model and a mocked store showed:

Two model calls per question. The compression step is a model call of its own, and it ran on the first question too, where there was no history. Budget for it, or use a plain RewriteQueryTransformer or no transformer for the first turn.
The follow-up was rewritten from the history. The prompt sent for compression contained the earlier question, the earlier answer and the follow-up, and asked for a "standalone query".
Sources came back. RetrievalAugmentationAdvisor.DOCUMENT_CONTEXT in the response context held the retrieved Document`s, so their `source metadata could be returned. QuestionAnswerAdvisor stores its documents under RETRIEVED_DOCUMENTS instead, which is why the class has the small second method.
allowEmptyContext(false) still calls the model. With no passages found, the prompt was replaced by an instruction to say politely that the question is outside the knowledge base, and the model wrote that reply. It does not skip the call.

The test is three questions, the third with an empty store:

DocsModularRagTest.java: sources, follow-up and empty context

package com.example.docsassistant;

import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;

import java.util.List;
import java.util.Map;

import org.junit.jupiter.api.Test;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.memory.InMemoryChatMemoryRepository;
import org.springframework.ai.chat.memory.MessageWindowChatMemory;
import org.springframework.ai.document.Document;
import org.springframework.ai.vectorstore.SearchRequest;
import org.springframework.ai.vectorstore.VectorStore;

class DocsModularRagTest {

    @Test
    void sourcesFollowUpAndEmptyContext() {
        VectorStore store = mock(VectorStore.class);
        when(store.similaritySearch(any(SearchRequest.class)))
                .thenReturn(List.of(new Document("ef_search controls recall", Map.of("source", "hnsw.adoc"))));
        // every question costs two model calls: the compression of the query, then the answer
        ScriptedChatModel model = new ScriptedChatModel(false)
                .then(ScriptedChatModel.text("what does ef_search do"))
                .then(ScriptedChatModel.text("It controls recall."))
                .then(ScriptedChatModel.text("how to raise ef_search"))
                .then(ScriptedChatModel.text("Raise it for more recall."))
                .then(ScriptedChatModel.text("something the docs never mention"))
                .then(ScriptedChatModel.text("I do not know."));
        var memory = MessageWindowChatMemory.builder().chatMemoryRepository(new InMemoryChatMemoryRepository()).build();
        DocsModularRag rag = new DocsModularRag(ChatClient.builder(model), memory, store);

        var first = rag.ask("alice:1", "What does ef_search do?");
        assertThat(first.sources()).containsExactly("hnsw.adoc");
        assertThat(model.prompts).hasSize(2);

        var second = rag.ask("alice:1", "And how do I raise it?");
        assertThat(second.answer()).isEqualTo("Raise it for more recall.");
        // the compression prompt carries the history and asks for a standalone query
        assertThat(model.prompts.get(2).getContents())
                .contains("Conversation history", "What does ef_search do?", "It controls recall.", "And how do I raise it?");

        // nothing found: the model is still called, with an instruction to decline politely
        when(store.similaritySearch(any(SearchRequest.class))).thenReturn(List.of());
        rag.ask("alice:2", "Something the docs never mention");
        assertThat(model.prompts.get(model.prompts.size() - 1).getContents()).contains("outside your knowledge base");
    }
}

Source: Spring AI: Retrieval Augmented Generation.

The filter is where per-user document access lives: build the Filter.Expression from the caller’s identity on the request thread, as on Security & Guardrails. The example uses a fixed visibility value to keep the class small; a supplier can read the security context on each call. Never let the model or the client choose the filter.

Citations, APIs and production RAG

Returning sources, as DocsModularRag does, is the start of citation support. Turning sources into checkable citations, exposing retrieval as an API, handling multi-turn conversations at scale, caching, freshness and the operational side of a RAG system are the subject of RAG Systems in Production: see Citations & Grounding, Serving RAG as an API, Conversational RAG & Memory, Caching, Latency & Cost and Ingestion Pipelines & Freshness. Production Considerations covers the vector-store side.

Evaluating whether the retrieved passages were relevant and the answer faithful is done with the evaluators on Evaluation, and a retrieval-backed MCP tool can serve the same store to other applications, see MCP Server.

Where older material differs

Table 3. Older RAG material and the 2.0.1 line
Older text	Spring AI 2.0.1
`import org.springframework.ai.chat.client.advisor.QuestionAnswerAdvisor;`	`org.springframework.ai.chat.client.advisor.vectorstore.QuestionAnswerAdvisor`
`spring-ai-advisors-vector-store`	`spring-ai-vector-store-advisor`; add `spring-ai-rag` for `RetrievalAugmentationAdvisor`
Advisors added "before the model" without an order	Set the order: the defaults of `0` put retrieval inside the tool loop
`PromptChatMemoryAdvisor` next to a RAG advisor	Removed; `MessageChatMemoryAdvisor` with a conversation ID on every call
Retrieval configured per store with old property prefixes	Store properties are unchanged in shape; check `.options` segments in model properties, see Migrating from 1.x to 2.x