Vector Databases & RAG

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

Welcome to the Vector Databases & RAG reference. This section covers vector databases — data stores purpose- built to index and search high-dimensional embeddings — and Retrieval-Augmented Generation (RAG), the pattern of grounding a large language model’s answers in text retrieved from such a store, through to the agentic patterns and framework integrations built on top of it. Code examples target Python 3.12 with sentence-transformers 6.1, FAISS 1.15.1, sqlite-vec 0.1.9 and pgvector 0.8.6, plus Ollama 0.34 for local embedding and chat models; the RAG and agent pages layer in langchain / langchain-core / langgraph 1.4 / 1.6 / 1.2, langchain-text-splitters 1.1 and langchain-postgres 0.0.18 on the Python side, LangChain4j 1.20.1 and Spring Boot / Spring AI 4.1 / 2.0.1 on the Java side, RAGAS 0.4.3 for evaluation, and the Model Context Protocol spec dated 2026-07-28 for agent tool-calling. This baseline was checked and re-verified as current on 2026-09-26.

Most pages in this section share one running example, DocsAssistant: an AI agent that answers questions about a corpus of technical documentation — specifically, this site’s own AsciiDoc pages. A single embedding model and a single chat model are used across every stack (plain Python, LangChain, Spring AI) so the examples stay interoperable, with a smaller model reserved for standalone embedding demos.

Aspect Choice used throughout DocsAssistant

Sample corpus

Three small AsciiDoc files, written out inline on Document Ingestion & Chunking and used in every example: docs/vector-rag/hnsw.adoc ("HNSW indexes"), docs/vector-rag/pgvector.adoc ("pgvector basics") and docs/spring-ai/rag.adoc ("RAG with Spring AI"), with metadata component = vector-rag or spring-ai

Embedding model

Ollama nomic-embed-text (768 dimensions, cosine distance); sentence-transformers all-MiniLM-L6-v2 (384 dimensions) is used separately for standalone concept demos on the embedding pages

Chat model

Ollama llama3.1:8b

Primary database

PostgreSQL + pgvector, database docsassistant, with a parallel SQLite/sqlite-vec version

Indexes

HNSW (pgvector’s chunk_embeddings_hnsw, FAISS IndexHNSWFlat), IVF (FAISS IndexIVFFlat), exact brute-force KNN in sqlite-vec’s vec0 table (no ANN index), plus Qdrant for cross-engine benchmarking

Frameworks used

LangChain / LangGraph (Python) and Spring AI (Java), each wired to the same embedding and chat models

Agent tool

A retrieval tool exposed to the chat model, called directly in-process and, in the agent pages, via MCP

Installing and tuning the local model server behind these models — Ollama’s context length, its OpenAI- and Anthropic-compatible APIs, and alternatives such as llama.cpp, vLLM and LM Studio — is covered on Ollama in the Running LLMs Locally sub-section.

How to read this section

The 23 pages in this section build up in six stages: concepts first (what embeddings are and how similarity search works), then engines (the ANN algorithms and the concrete libraries/databases that implement them), then RAG itself (chunking, retrieval and prompt construction), then agents (agentic RAG loops and persistent memory), then the two framework integrations, and finally operations (evaluation and running this in production). Start with What Is a Vector Database? and Embeddings, then Generating Embeddings in Practice to get a model actually producing vectors. From there, move into similarity search and indexing — Similarity Metrics & Semantic Search, ANN Indexes, Quantization & Compression and FAISS — followed by filtering and hybrid retrieval (Metadata Filtering, Hybrid Search & Re-ranking), then vector search inside SQL databases and how to choose an engine (SQLite, PostgreSQL & pgvector, The Landscape and Comparing Vector Databases). The RAG cluster (RAG Fundamentals through Prompt Augmentation & Generation) and the agent cluster (Agentic RAG, Agent Memory & Semantic Cache) come next, then the two framework pages, and the section closes with Evaluating RAG Systems and Production Considerations.

Next: RAG Systems in Production. Once the foundations here are in place, continue with RAG Systems in Production in the AI guide, which covers the day-2 engineering of the same DocsAssistant service: parsing real documents, ingestion freshness, routing, conversation persistence, citations, hallucination control, access control, serving through an API and MCP, GraphRAG, evaluation at scale and observability.

This section covers the ANN algorithms and general-purpose libraries (FAISS) and SQL extensions (sqlite-vec, pgvector) behind vector search, but several full database engines that also do vector search already have their own dedicated reference sections elsewhere on this site, and are not duplicated here. Engine-specific material lives in those sections instead:

Engine Where it is covered

Qdrant

Qdrant Reference

Redis

Redis Reference

Elasticsearch

Elasticsearch Reference

Solr

Solr Reference

Lucene

Lucene Reference

Neo4j

Neo4j Reference

MongoDB

MongoDB Reference

Couchbase

Couchbase Reference

The Vector Database Landscape and Comparing Vector Databases place all of these engines, plus the SQL-extension and FAISS options covered here, side by side.

This section’s two framework pages — Integrating with LangChain and Integrating with Spring AI — only cover the vector-store, RAG and agent-memory surface of each framework. Each framework is covered end to end in its own sub-section of the AI section: LangChain and Spring AI.

What’s covered

Foundations

  • What Is a Vector Database? — why unstructured data needs semantic search, the vector data type and its three operations, and the hybrid architecture every vector database implements.

  • Embeddings — what an embedding is, how embedding models are trained, and the properties (dimensionality, distance, clustering) that make them useful for search.

  • Generating Embeddings in Practice — producing embeddings with sentence-transformers and with Ollama, batching, and normalization.

Similarity search & indexing

Filtering, hybrid search & relevance

  • Metadata Filtering — combining vector similarity with structured filters, and pre- versus post-filtering trade-offs.

  • Hybrid Search & Re-ranking — combining dense and sparse (keyword) retrieval with Reciprocal Rank Fusion, and cross-encoder re-ranking.

Vector search in SQL databases & choosing an engine

  • Vector Search in SQLite — the sqlite-vec extension and its vec0 virtual tables.

  • PostgreSQL & pgvector — the vector column type, HNSW and IVFFlat indexes, and using PostgreSQL as both the primary and the vector store for DocsAssistant.

  • The Vector Database Landscape — a survey of dedicated vector databases and multi-model engines that support vector search.

  • Comparing Vector Databases — a feature, deployment and pricing comparison across the landscape, including managed-service offerings.

RAG

  • RAG Fundamentals — the retrieve-then-generate loop and why grounding an LLM’s answers in retrieved text reduces hallucination.

  • Document Ingestion & Chunking — splitting source documents into chunks, chunk size/overlap trade-offs, and late chunking.

  • Retrieval Strategies — top-k retrieval, maximal marginal relevance, and multi-query retrieval.

  • Prompt Augmentation & Generation — assembling retrieved context into a prompt, context-window budgeting, and citation.

Agents

  • Agentic RAG — letting an agent decide when and what to retrieve, instead of a fixed retrieve-then-generate pipeline.

  • Agent Memory & Semantic Cache — persisting conversation memory as embeddings, and caching semantically similar prior queries.

Frameworks

Quality & operations

  • Evaluating RAG Systems — RAGAS metrics (faithfulness, answer relevance, context precision/recall) and ANN-Benchmarks-style recall/latency evaluation.

  • Production Considerations — re-indexing, monitoring, cost, and scaling a vector search + RAG deployment.

  • Cheat Sheet (PDF) — a single-page, printable summary of everything in this section, with a downloadable PDF.

mindmap root((Vector Databases & RAG)) Foundations What Is a Vector Database? Embeddings Generating Embeddings in Practice Similarity search & indexing Similarity Metrics ANN Indexes Quantization & Compression FAISS Filtering, hybrid search & relevance Metadata Filtering Hybrid Search & Re-ranking SQL vector search & choosing an engine SQLite pgvector Landscape Comparison RAG RAG Fundamentals Ingestion & Chunking Retrieval Strategies Prompt Augmentation Agents Agentic RAG Agent Memory & Semantic Cache Frameworks LangChain Spring AI Quality & operations Evaluating RAG Systems Production Considerations

Bibliography

Requester-provided book, consulted as a reference for concepts and the practitioner’s angle only — never the primary source for any code example on this section’s pages:

Embeddings and models:

Vector search libraries and SQL extensions:

Vector engines (landscape and comparison pages):

Managed services, cloud availability, pricing and benchmarks (the same sources used as references on Comparing Vector Databases):

LangChain, LangGraph and LangChain4j:

Spring AI (the Spring AI reference documentation):

Agents and protocols:

Evaluation:

Papers and articles:

The Borwankar book above is a consulted bibliographic reference only, never the primary or main reference for this section’s pages or code examples. Where it and a tool’s own official documentation disagree, that tool’s official documentation is authoritative, and every code example was verified against the versions stated at the top of this page when the pages were written.