RAG Integration Patterns

This section documents the current Qdrant 1.x line (latest 1.19.1), written and verified against the official Qdrant documentation. No specific patch version is pinned. Some capabilities (Qdrant Cloud-only features and the full breadth of client-library APIs beyond Python and Java) are linked, not documented in depth.

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

Retrieval-Augmented Generation (RAG) grounds an LLM’s answer in passages retrieved from a corpus rather than relying only on what the model memorized during training. Qdrant plays the retrieval half of that pattern: it stores an embedding for every passage alongside metadata, then returns the passages closest to a query embedding so they can be inserted into the LLM’s prompt as context. For when Qdrant specifically — rather than Pinecone, Weaviate, Milvus, pgvector, or an existing search engine’s vector support — is the right retrieval store for this job, see Choosing the Right Database — Vector Databases.

End-to-end shape of the pattern

A RAG system has two pipelines that share the same collection: an ingestion pipeline that runs once (or incrementally) per source document, and a retrieval pipeline that runs on every user query.

flowchart TD subgraph Ingestion A[Source document] --> B[Chunk into passages] B --> C[Embed each chunk] C --> D[Upsert as points
vector + payload] end subgraph Retrieval Q[User query] --> E[Embed query] E --> F[Search / filter / re-rank] D -.-> F F --> G[Assemble context] G --> L[LLM generates answer] end

The rest of this page walks through each stage of both paths, then shows a runnable example of the whole thing using qdrant-client directly.

Embedding pipeline: chunking

An LLM’s context window and an embedding model’s input limit both force a source document to be split into smaller chunks before it is embedded. The chunking strategy trades off retrieval precision against how much surrounding context survives in each piece:

  • Fixed-size chunking splits on a token or character count (often with a small overlap between consecutive chunks). It is the cheapest to implement and the most predictable in size, but it can cut a sentence or an argument in half at an arbitrary boundary.

  • Sentence-based chunking splits on sentence boundaries and groups a target number of sentences per chunk. It keeps individual sentences intact, at the cost of chunks that vary more in length and token count.

  • Semantic chunking groups sentences by embedding similarity, starting a new chunk where the topic actually shifts rather than where a fixed budget runs out. It produces the most topically coherent chunks, but costs an extra embedding pass over the document just to find the split points.

There is no single correct choice — the Question Answering System tutorial walks through a complete pipeline built on one particular chunking and embedding choice, as one worked example among these trade-offs.

Embedding pipeline: choosing a model

The embedding model choice fixes three things a Qdrant collection then has to be configured around:

  • Dimension count — the length of the vector each chunk becomes, which must match the collection’s configured vector size exactly (see Collections & Points for how a collection’s vector parameters are set).

  • Normalization — whether the model already outputs unit-length vectors. A normalized model pairs naturally with cosine distance (equivalent to a dot product on unit vectors); an unnormalized one may call for a different distance metric or an explicit normalization step before upserting.

  • Cost and latency — a larger embedding model produces better retrieval quality but costs more per token and adds latency to both ingestion (once per chunk) and every query (once per request), so the choice is a direct trade-off against how much traffic the system serves.

Storing chunks as points

Each chunk becomes one Qdrant point: its vector is the chunk’s embedding, and its payload carries the metadata needed to trace a retrieved chunk back to its source and to filter candidates before they are ranked — typically the source document id, the chunk’s index within that document, a section or heading title, and any access-control tags that scope which chunks a given caller is allowed to retrieve. Payload design and the indexes that make filtering on these fields fast are covered in Filtering & Payload Indexing, not repeated here.

Retrieval at query time

At query time the user’s question is embedded with the same model used for ingestion, then the retrieval path runs in up to four stages:

  1. Search — a dense vector search over the query embedding, or a hybrid combination of dense and sparse search; see Hybrid & Sparse Search for how Qdrant runs and fuses that search, not repeated here.

  2. Filter — payload filters narrow the candidate set to the document ids, sections, or access-control tags the query is allowed to see, using the filtering machinery in Filtering & Payload Indexing.

  3. Re-rank (optional) — a smaller, more expensive scoring pass (e.g. a cross-encoder) reorders the top candidates the vector search returned, trading extra latency for better ranking quality than distance alone gives on the first pass.

  4. Assemble context — the surviving chunks' payload text (and their source metadata) are concatenated into the prompt sent to the LLM.

Integrating with LangChain and LlamaIndex

Both major RAG frameworks ship a first-class Qdrant vector-store integration so the ingestion and retrieval plumbing above does not have to be hand-written for every project:

  • LangChain — the langchain-qdrant package provides a QdrantVectorStore class that implements LangChain’s standard vector-store interface (add_documents, similarity_search, retriever construction) on top of qdrant-client, including hybrid search and payload-filter pass-through.

  • LlamaIndex — the llama-index-vector-stores-qdrant package provides a QdrantVectorStore class that plugs into LlamaIndex’s index and query-engine abstractions the same way, so a VectorStoreIndex backed by it persists and queries nodes as Qdrant points.

Both integrations still upsert points and run searches through qdrant-client underneath, following the same point-and-payload shape and the same search/filter/re-rank flow described above — they just generate that code from higher-level document and query-engine objects.

Example: ingestion and retrieval with qdrant-client

The example below uses qdrant-client directly, since it is the layer both LangChain’s and LlamaIndex’s integrations sit on top of. It chunks a document, embeds the chunks, and upserts them as points with metadata payloads:

from qdrant_client import QdrantClient
from qdrant_client.models import Distance, VectorParams, PointStruct

client = QdrantClient(url="http://localhost:6333")

EMBEDDING_DIM = 384  # must match the embedding model's output dimension

client.create_collection(
    collection_name="kb_chunks",
    vectors_config=VectorParams(size=EMBEDDING_DIM, distance=Distance.COSINE),
)


def chunk_document(text, chunk_size=500, overlap=50):
    """Fixed-size chunking with a small overlap between consecutive chunks."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks


def embed(texts):
    """Stand-in for a call to an embedding model (e.g. sentence-transformers, an API)."""
    ...  # returns a list of EMBEDDING_DIM-length float vectors, one per input text


document_id = "handbook-v2"
section_title = "Onboarding"
source_text = open("handbook.txt").read()

chunks = chunk_document(source_text)
vectors = embed(chunks)

points = [
    PointStruct(
        id=f"{document_id}-{i}",
        vector=vector,
        payload={
            "document_id": document_id,
            "chunk_index": i,
            "section_title": section_title,
            "access_tags": ["employees"],
            "text": chunk_text,
        },
    )
    for i, (chunk_text, vector) in enumerate(zip(chunks, vectors))
]

client.upsert(collection_name="kb_chunks", points=points)

The query-time path embeds the user’s question, searches with a payload filter, and assembles the retrieved chunks into the LLM’s prompt context:

from qdrant_client.models import Filter, FieldCondition, MatchAny

query = "How do I request time off?"
query_vector = embed([query])[0]

results = client.query_points(
    collection_name="kb_chunks",
    query=query_vector,
    query_filter=Filter(
        must=[FieldCondition(key="access_tags", match=MatchAny(any=["employees"]))]
    ),
    limit=5,
).points

# Optional re-ranking pass would reorder `results` here before assembling context.

context = "\n\n".join(
    f"[{hit.payload['section_title']} #{hit.payload['chunk_index']}]\n{hit.payload['text']}"
    for hit in results
)

prompt = f"""Answer the question using only the context below.

Context:
{context}

Question: {query}
"""

Related pages