Integrating with LangChain

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

Examples on this page target LangChain / langchain-core / LangGraph 1.4 / 1.6 / 1.2, langchain-text-splitters 1.1, langchain-ollama, langchain-postgres 0.0.18, and LangChain4j 1.20.1 against Ollama 0.34 (nomic-embed-text, 768 dimensions, and llama3.1:8b). The full, standalone LangChain reference — chat models, prompts, LCEL, structured output, tools, agents, LangGraph, LangSmith and LangChain4j — is the LangChain sub-section of the AI section; this page collects, under one set of stable anchors, only the document-loading, vector-store, retrieval, RAG-chain and agent-memory surface that the rest of this Vector Databases & RAG section already relies on and links to by anchor. Every concept below was already shown, working against the DocsAssistant schema, on an earlier page in this section — this page is the reference those pages point back to, not a new tutorial.

Embeddings

LangChain’s Embeddings interface abstracts over an embedding provider behind two methods: embed_documents for a batch of texts and embed_query for a single query string, so retrieval code never depends on which model produced the vectors.

from langchain_ollama import OllamaEmbeddings

embeddings = OllamaEmbeddings(model="nomic-embed-text")

doc_vectors = embeddings.embed_documents([
    "search_document: HNSW builds a multi-layer proximity graph over the vectors.",
    "search_document: pgvector adds a vector column type and distance operators to PostgreSQL.",
])
query_vector = embeddings.embed_query("search_query: what does ef_search control?")

See LangChain: Ollama embeddings. A hosted or self-hosted HuggingFace model plugs into the same interface through langchain-huggingface’s `HuggingFaceEmbeddings (local sentence-transformers models) or HuggingFaceEndpointEmbeddings (a remote Inference Endpoint):

from langchain_huggingface import HuggingFaceEmbeddings

embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")

See LangChain: HuggingFace embeddings. generating-embeddings-in-practice.adoc walks through OllamaEmbeddings end to end, including task prefixes and caching; see Generating Embeddings in Practice.

Vector stores

Every LangChain vector store implements the same VectorStore base class, so swapping the underlying engine is a one-line change to the constructor. The methods used throughout this section are:

Method Purpose

add_documents(documents, ids=None)

Embeds and writes a batch of Document objects (page content plus a metadata dict).

similarity_search(query, k=5, filter=None)

Returns the k nearest Document objects to the query’s embedding.

similarity_search_with_score(query, k=5, filter=None)

Same as above, but also returns each result’s raw distance or similarity score.

max_marginal_relevance_search(query, k=5, fetch_k=20, lambda_mult=0.5)

Diversity-aware retrieval: over-fetches fetch_k candidates, then greedily selects k that balance relevance to the query against dissimilarity to what has already been picked (see Hybrid Search & Re-ranking's MMR section).

filter

A store-specific keyword argument (a dict, a SQL-like expression string, or a store’s own filter object) restricting the search to matching metadata, as shown on Metadata Filtering.

delete(ids=None, filter=None)

Removes documents by id or by a metadata filter.

Against the DocsAssistant scenario, langchain-postgres’s `PGVector store is the one used throughout this section. It owns its own schema under the collection name docsassistant — separate tables from the hand-written documents / chunks / chunk_embeddings schema on PostgreSQL & pgvector, because each framework manages its own storage layout rather than reusing another framework’s tables:

from langchain_postgres import PGVector
from langchain_ollama import OllamaEmbeddings

vector_store = PGVector(
    embeddings=OllamaEmbeddings(model="nomic-embed-text"),
    collection_name="docsassistant",
    connection="postgresql+psycopg://docs:docs@localhost:5432/docsassistant",
    use_jsonb=True,
)

vector_store.add_documents(chunks, ids=[c.metadata["chunk_id"] for c in chunks])
hits = vector_store.similarity_search_with_score(
    "search_query: what does ef_search control?", k=5, filter={"component": {"$eq": "vector-rag"}},
)

See LangChain: PGVector. Three more stores are used elsewhere in this section as lighter-weight or local alternatives:

  • SQLiteVec, wrapping the sqlite-vec vec0 table pattern from Vector Search in SQLite; see LangChain: SQLiteVec.

  • FAISS, a disposable, single-process store backed by the library covered on FAISS; see LangChain: FAISS vector store.

  • InMemoryVectorStore, an ephemeral, dependency-free store useful for tests and small demos that keeps every vector in a Python dict and scores it with brute-force cosine similarity — no server, no persistence:

    from langchain_core.vectorstores import InMemoryVectorStore
from langchain_ollama import OllamaEmbeddings

vector_store = InMemoryVectorStore(OllamaEmbeddings(model="nomic-embed-text"))
vector_store.add_documents(chunks)

    See LangChain: InMemoryVectorStore.

For every other engine this site documents, LangChain ships an equivalent store: Qdrant (langchain-qdrant’s `QdrantVectorStore, see Qdrant: RAG Integration Patterns), Redis (langchain-redis’s `RedisVectorStore, see Redis: Vector Search), Elasticsearch (langchain-elasticsearch’s `ElasticsearchStore, see Elasticsearch: Vector & Semantic Search), MongoDB (langchain-mongodb’s `MongoDBAtlasVectorSearch, see MongoDB: Special Indexes & Search), and Neo4j (langchain-neo4j’s `Neo4jVector, see Neo4j: Vector Search & GenAI). See LangChain: vector store integrations for the full list and each store’s own configuration.

Splitters

A document loader reads source files into LangChain Document objects (page content plus a metadata dict). For the DocsAssistant sample corpus — docs/vector-rag/hnsw.adoc, docs/vector-rag/pgvector.adoc and docs/spring-ai/rag.adoc — langchain-community’s `DirectoryLoader globs the AsciiDoc files and reads each one with TextLoader, which records the file path under the source metadata key; the loop then maps it to the section’s source_path and component keys:

from langchain_community.document_loaders import DirectoryLoader, TextLoader

loader = DirectoryLoader(
    "docs", glob="**/*.adoc", loader_cls=TextLoader, loader_kwargs={"encoding": "utf-8"},
)
documents = loader.load()
for doc in documents:
    doc.metadata["source_path"] = doc.metadata["source"]
    doc.metadata["component"] = "spring-ai" if "/spring-ai/" in doc.metadata["source"] else "vector-rag"

See LangChain reference: DirectoryLoader, LangChain reference: TextLoader and LangChain: Document loaders for the PDF, HTML and other format-specific loaders shown on Document Ingestion & Chunking.

langchain-text-splitters then turns the loaded documents into the chunks a vector store indexes. The package used throughout Document Ingestion & Chunking covers RecursiveCharacterTextSplitter (recursive character/token windows with overlap) and MarkdownHeaderTextSplitter (structure-aware splitting on heading levels), both shown there against the DocsAssistant sample corpus:

from langchain_text_splitters import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
chunks = splitter.split_documents(documents)

See LangChain: Recursive character text splitter and LangChain: Markdown header text splitter. Every chunking strategy this section covers — fixed windows, sentence-aware, recursive, structure-aware, semantic, parent-child, late chunking and contextual retrieval — is explained in depth, with the trade-offs between them, on Document Ingestion & Chunking: Chunking strategies.

Retrievers

Every VectorStore exposes a Runnable retriever through as_retriever, so it composes with the rest of a LangChain pipeline (prompts, chains, agent tools) without the caller touching the store’s own API:

retriever = vector_store.as_retriever(
    search_type="mmr",
    search_kwargs={"k": 5, "fetch_k": 20, "lambda_mult": 0.5, "filter": {"component": {"$eq": "vector-rag"}}},
)
docs = retriever.invoke("what does ef_search control?")

search_type selects "similarity" (default), "similarity_score_threshold" (drop results below a minimum score), or "mmr" (the diversity-aware search from Vector stores); search_kwargs forwards the matching keyword arguments — k, score_threshold, filter, fetch_k and lambda_mult — to the underlying store method. See LangChain: Retrieval. Beyond this basic retriever, langchain_classic.retrievers packages the retrieval techniques covered on Hybrid Search & Re-ranking (EnsembleRetriever, ContextualCompressionRetriever) and Retrieval Strategies (MultiQueryRetriever, ParentDocumentRetriever, SelfQueryRetriever); see LangChain: langchain_classic.retrievers.

RAG chain

RAG Fundamentals shows the full pipeline this anchor refers to: a retriever built with as_retriever, a ChatPromptTemplate that formats the retrieved documents and the question, and a chat model, composed with LangChain Expression Language’s | operator into one Runnable:

from langchain_ollama import ChatOllama
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser

prompt = ChatPromptTemplate.from_template(
    "Answer using only the context below.\n\nContext:\n{context}\n\nQuestion: {question}"
)
model = ChatOllama(model="llama3.1:8b", temperature=0.1, num_ctx=8192)


def format_docs(docs):
    return "\n\n".join(f"[{d.metadata['title']} § {d.metadata['section']}]\n{d.page_content}" for d in docs)


rag_chain = (
    {"context": retriever | format_docs, "question": RunnablePassthrough()}
    | prompt
    | model
    | StrOutputParser()
)
answer = rag_chain.invoke("what does ef_search control?")

See LangChain: Retrieval for this composition pattern, and Prompt Augmentation & Generation for the context-formatting and prompt-template choices behind format_docs and prompt.

Agentic RAG

Agentic RAG covers the shift from a fixed chain to an LLM that decides whether and what to retrieve. LangChain 1.x’s create_agent builds a tool-calling agent from a chat model and a list of tools — including a retriever exposed as a tool — in one call:

from langchain.agents import create_agent
from langchain_ollama import ChatOllama
from langchain_core.tools import tool


@tool
def search_docs(query: str, component: str | None = None, k: int = 5) -> list[dict]:
    """Search the DocsAssistant documentation for passages relevant to the query."""
    filter_ = {"component": {"$eq": component}} if component else None
    hits = vector_store.similarity_search_with_score(query, k=k, filter=filter_)
    return [{"source_path": d.metadata["source_path"], "section": d.metadata["section"],
              "content": d.page_content, "score": score} for d, score in hits]


agent = create_agent(model=ChatOllama(model="llama3.1:8b", temperature=0.1), tools=[search_docs])
result = agent.invoke({"messages": [{"role": "user", "content": "What does ef_search control?"}]})

See LangChain: Agents for `create_agent’s full configuration (system prompts, structured output, middleware). For multi-step agentic RAG — retrieve, grade the retrieved documents, then rewrite the query or generate the answer — LangChain 1.x agents run on a LangGraph graph underneath; building that graph directly gives explicit control over the retrieve/grade/ rewrite/generate nodes that `create_agent’s default loop does not expose. See LangChain: Agentic RAG (LangGraph).

Memory for such an agent comes from LangGraph: a checkpointer persists each thread’s conversation state (short-term memory, keyed by thread_id), and a store holds long-term memories that are searched by embedding similarity across threads. langgraph-checkpoint-postgres provides both against the same DocsAssistant PostgreSQL database, with the store’s semantic index using the same nomic-embed-text embeddings:

from langchain.agents import create_agent
from langchain_ollama import ChatOllama, OllamaEmbeddings
from langgraph.checkpoint.postgres import PostgresSaver
from langgraph.store.postgres import PostgresStore

DB_URI = "postgresql://docs:docs@localhost:5432/docsassistant"
embeddings = OllamaEmbeddings(model="nomic-embed-text")

with (
    PostgresSaver.from_conn_string(DB_URI) as checkpointer,
    PostgresStore.from_conn_string(
        DB_URI, index={"dims": 768, "embed": embeddings, "fields": ["text"]},
    ) as store,
):
    checkpointer.setup()  # creates the checkpoint tables on first run
    store.setup()         # creates the store tables and its pgvector index

    store.put(("docsassistant", "user-42"), "pref-1", {"text": "Prefers answers with SQL examples."})
    memories = store.search(("docsassistant", "user-42"), query="how should answers be formatted?", limit=3)

    agent = create_agent(
        model=ChatOllama(model="llama3.1:8b", temperature=0.1),
        tools=[search_docs],
        checkpointer=checkpointer,
        store=store,
    )
    result = agent.invoke(
        {"messages": [{"role": "user", "content": "And which operator computes cosine distance?"}]},
        {"configurable": {"thread_id": "conversation-1"}},
    )

Install with pip install langgraph-checkpoint-postgres "psycopg[binary]". See LangChain: Short-term memory for the Postgres checkpointer, LangChain: Long-term memory for the store and its semantic index, and Agent Memory & Semantic Cache: LangGraph checkpointer and store for how this maps onto the section’s hand-written message_embeddings memory.

LangChain4j

LangChain4j is the JVM counterpart used throughout this section wherever a page shows a Java client alongside its Python and Spring AI ones (for example Generating Embeddings in Practice's OllamaEmbeddingModel and Document Ingestion & Chunking's EmbeddingStoreIngestor). Its central interfaces mirror LangChain’s Python ones under different names:

Interface Role

EmbeddingStore<TextSegment>

The vector-store abstraction, equivalent to LangChain’s VectorStore — add, search, remove.

EmbeddingStoreIngestor

Wires a document loader, a splitter, an EmbeddingModel and an EmbeddingStore into one ingestion call, equivalent to a LangChain load → split → embed → add_documents sequence.

EmbeddingStoreContentRetriever

Wraps an EmbeddingStore as a ContentRetriever, LangChain4j’s retriever abstraction, equivalent to as_retriever.

AiServices with a RetrievalAugmentor

Declares an interface-backed chat service and augments every call with retrieved content before it reaches the model, equivalent to composing a retriever into a LangChain RAG chain.

PgVectorEmbeddingStore is the EmbeddingStore implementation used against the DocsAssistant database; like the Python PGVector store, it owns its own table (docs_lc4j_embeddings), separate from the hand-written schema and from Spring AI’s docs_vector_store:

import dev.langchain4j.data.segment.TextSegment;
import dev.langchain4j.model.embedding.EmbeddingModel;
import dev.langchain4j.model.ollama.OllamaChatModel;
import dev.langchain4j.model.ollama.OllamaEmbeddingModel;
import dev.langchain4j.store.embedding.EmbeddingStore;
import dev.langchain4j.store.embedding.pgvector.PgVectorEmbeddingStore;
import dev.langchain4j.store.embedding.EmbeddingStoreIngestor;
import dev.langchain4j.rag.content.retriever.EmbeddingStoreContentRetriever;
import dev.langchain4j.rag.DefaultRetrievalAugmentor;
import dev.langchain4j.service.AiServices;

EmbeddingModel embeddingModel = OllamaEmbeddingModel.builder()
        .baseUrl("http://localhost:11434")
        .modelName("nomic-embed-text")
        .build();

EmbeddingStore<TextSegment> embeddingStore = PgVectorEmbeddingStore.builder()
        .host("localhost").port(5432).database("docsassistant")
        .user("docs").password("docs")
        .table("docs_lc4j_embeddings")
        .dimension(768)
        .useIndex(true)        // creates an IVFFlat index (vector_cosine_ops) ...
        .indexListSize(100)    // ... whose lists count is required whenever useIndex is true
        .build();

EmbeddingStoreIngestor.builder()
        .embeddingModel(embeddingModel)
        .embeddingStore(embeddingStore)
        .build()
        .ingest(documents);

EmbeddingStoreContentRetriever contentRetriever = EmbeddingStoreContentRetriever.builder()
        .embeddingStore(embeddingStore)
        .embeddingModel(embeddingModel)
        .maxResults(5)
        .build();

interface DocsAssistant {
    String answer(String question);
}

DocsAssistant assistant = AiServices.builder(DocsAssistant.class)
        .chatModel(OllamaChatModel.builder().baseUrl("http://localhost:11434").modelName("llama3.1:8b").build())
        .retrievalAugmentor(DefaultRetrievalAugmentor.builder().contentRetriever(contentRetriever).build())
        .build();

useIndex(true) makes the store create an IVFFlat index on its table, and indexListSize (the IVFFlat lists parameter) must then be set to a value greater than zero — otherwise table initialization throws an IllegalArgumentException. See LangChain4j: PGVector embedding store.

The Maven coordinates for the pieces above (the pgvector module is published on LangChain4j’s -beta track, so its version carries the -beta30 suffix):

<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j</artifactId>
    <version>1.20.1</version>
</dependency>
<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j-ollama</artifactId>
    <version>1.20.1</version>
</dependency>
<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j-pgvector</artifactId>
    <version>1.20.1-beta30</version>
</dependency>

LangChain4j groups RAG implementations into three levels of the same taxonomy used on RAG Fundamentals: Easy RAG (EmbeddingStoreIngestor with default settings, for a first working pipeline), Naive RAG (an explicit EmbeddingStoreContentRetriever composed by hand, as above), and Advanced/Modular RAG (a RetrievalAugmentor with a configurable QueryTransformer, QueryRouter and ContentAggregator, covered in depth on Retrieval Strategies: LangChain4j). See LangChain4j: Retrieval-Augmented Generation and LangChain4j: embedding (vector) stores.

Spring AI equivalents

Every concept above has a Spring AI counterpart under its own fixed anchor on Integrating with Spring AI: embeddings (EmbeddingModel, #spring-ai-embeddings), vector stores (VectorStore and PgVectorStore, #spring-ai-vector-store-api), metadata filtering (#spring-ai-filter-expressions), ingestion (#spring-ai-etl), the RAG chain (QuestionAnswerAdvisor, #spring-ai-rag-advisors), modular RAG (#spring-ai-modular-rag), and agent memory (ChatMemory, #spring-ai-chat-memory) alongside tool-calling (@Tool, #spring-ai-tools). Where LangChain composes a Runnable pipeline or wraps a service in AiServices, Spring AI composes an advisor chain around a ChatClient; the two frameworks solve the same retrieval and augmentation problems with different composition styles, and every concept page in this section shows both side by side.

Related pages

References