FAISS

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

Examples on this page target FAISS 1.15.1 (the faiss-cpu / faiss-gpu PyPI packages) with Python 3.12 and NumPy. FAISS (Facebook AI Similarity Search) is not a database: it is a C++ library with Python bindings that implements similarity-search and clustering algorithms over dense vectors, with no server process, no network protocol and no persistence format beyond a single file. The DocsAssistant scenario uses it as a local, disposable index alongside the pgvector-backed system of record: an IndexFlatIP ground truth, an IndexHNSWFlat(768, 16) and an IndexIVFFlat with nlist = 64, persisted to docsassistant.faiss.

Architecture: Index, index_factory and the core operations

Every FAISS index is a subclass of the abstract Index type and shares the same small surface: it is constructed either directly (faiss.IndexFlatIP(d)) or through index_factory, which parses a short string describing a pipeline of transforms and a coarse/fine quantizer (for example "IVF64,Flat" or "PCA256,IVF64,PQ16"). Indexes that need statistics about the data distribution (IVF*, PQ*) must be train-ed on a representative sample before vectors can be added; flat indexes need no training.

import faiss
import numpy as np

d = 768  # matches nomic-embed-text
n = 10_000
rng = np.random.default_rng(seed=42)
vectors = rng.normal(size=(n, d)).astype("float32")
faiss.normalize_L2(vectors)  # cosine via inner product on unit vectors

index = faiss.IndexFlatIP(d)
print(index.is_trained)  # True -- Flat needs no training
index.add(vectors)
print(index.ntotal)  # 10000

query = vectors[:3]
k = 5
distances, ids = index.search(query, k)  # ids: row position, not a stable external id

By default add assigns sequential row ids 0..ntotal-1; IndexIDMap (or IndexIDMap2, which also supports removal by id) wraps any index so add_with_ids can attach the caller’s own ids — in the DocsAssistant scenario, the chunks.id primary key — instead of relying on insertion order:

inner = faiss.IndexFlatIP(d)
index = faiss.IndexIDMap(inner)

chunk_ids = np.arange(1000, 1000 + n, dtype="int64")  # chunks.id values
index.add_with_ids(vectors, chunk_ids)

distances, ids = index.search(query, k)  # ids are now real chunk_ids

search always returns exactly k neighbours (padded with -1 ids if ntotal < k). When the useful notion is "every neighbour within a distance" rather than a fixed count, range_search takes a radius instead of a k and returns a ragged result via lims/distances/labels arrays:

radius = 0.75  # inner-product threshold on unit-normalised vectors
lims, range_distances, range_ids = index.range_search(query, radius)
for i in range(len(query)):
    hits = range_ids[lims[i]:lims[i + 1]]
    print(f"query {i}: {len(hits)} neighbours within radius")

Persistence

An index is serialised to (and read back from) a single file with write_index / read_index — there is no separate schema or metadata file, so anything beyond the raw vectors and ids must be stored elsewhere (see What FAISS does not give you):

faiss.write_index(index, "docsassistant.faiss")
loaded = faiss.read_index("docsassistant.faiss")
assert loaded.ntotal == index.ntotal

CPU vs. GPU

The same index classes exist in CPU and GPU variants. faiss-cpu is the default install; faiss-gpu (or faiss-gpu-cuvs for the cuVS-accelerated build) requires a CUDA-capable GPU and moves an index onto it with index_cpu_to_gpu and back with index_gpu_to_cpu (a GPU index cannot be passed to write_index directly). This is worthwhile for training and bulk search over large IVF/PQ indexes but rarely needed for a corpus the size of DocsAssistant’s documentation chunks. The sketch below moves an empty IndexFlatIP to the GPU, fills and queries it there, and copies it back to the CPU to persist it:

res = faiss.StandardGpuResources()
cpu_flat = faiss.IndexFlatIP(d)
gpu_index = faiss.index_cpu_to_gpu(res, 0, cpu_flat)  # device 0
gpu_index.add(vectors)  # a plain Flat index, so add() assigns ids 0..n-1
distances, ids = gpu_index.search(query, k)

cpu_index = faiss.index_gpu_to_cpu(gpu_index)  # bring back for write_index
faiss.write_index(cpu_index, "docsassistant_flat.faiss")

See the official Getting started, Faiss indexes and Faiss on the GPU wiki pages.

Composite indexes

FAISS indexes compose: a factory string chains a vector transform, a coarse quantizer and (optionally) a codec into one object that trains and searches as a unit.

  • IndexPreTransform — applies a transform (typically PCAMatrix, via "PCA256,…​" in a factory string) to every vector before it reaches the wrapped index, both on add and on search. Useful to cut dimensionality before an IVF/PQ stage; see Quantization & Compression for the recall trade-off.

  • IndexRefineFlat — wraps a fast, approximate (and often quantized) index with an exact re-ranking pass: the approximate index over-fetches k * refine_factor candidates, then IndexFlatL2/IndexFlatIP computes exact distances on just those candidates to reorder the final top-k. This recovers most of the recall lost to quantization at a fraction of exact search’s cost.

  • IndexShards — splits a large collection across several sub-indexes (for example one per shard of a corpus too big to fit — or train efficiently — in one process) and fans a query out to all of them, merging results. It is a scale-out mechanism, not a durability one: there is still no replication, no network protocol and no coordinator.

# PCA to 256 dims, IVF with 64 lists, flat inner-product refinement
index = faiss.index_factory(768, "PCA256,IVF64,Flat,RFlat", faiss.METRIC_INNER_PRODUCT)
index.train(vectors)
index.add(vectors)
index.k_factor = 4  # RFlat: fetch 4x candidates before exact re-ranking
distances, ids = index.search(query, k)

Index-properties table

Approximate memory per vector for a d = 768-dimensional embedding (float32 baseline: 768 * 4 = 3072 bytes), ignoring per-index bookkeeping overhead:

Index Bytes / vector (d = 768) Notes

IndexFlatL2 / IndexFlatIP

3072 B

Exact, no training, linear scan.

IndexIVFFlat

3072 B + small ID/list overhead

Approximate via nprobe lists visited; vectors stored uncompressed.

IndexHNSWFlat(d, M)

d*4 + M*2*4 B (M=16 → ~3200 B)

Same memory formula used throughout ANN Indexes: Flat, IVF, HNSW and Beyond; graph edges dominate the overhead over Flat.

IndexIVFPQ (PQ16, 8-bit codes)

~16 B + list overhead

See Quantization & Compression for the encode/decode arithmetic.

IndexPreTransform (PCA256) + IndexIVFFlat

256*4 = 1024 B + overhead

Reduced dimensionality lowers both storage and distance-computation cost.

Benchmarking Flat vs. IVF vs. HNSW

The snippet below builds the three DocsAssistant index types — the Flat (IndexFlatIP) ground truth, an IVF64,Flat (IndexIVFFlat with nlist = 64) and an HNSW16 (IndexHNSWFlat(768, 16)) — over the same synthetic 768-dimensional dataset, and prints build time, query time and recall@10 against the ground truth. No output is reproduced here on purpose: the numbers depend entirely on the machine, the FAISS build (CPU/GPU, SIMD level) and the data distribution, so run it on your own hardware and embeddings.

import time
import numpy as np
import faiss

d, n, n_queries, k = 768, 50_000, 200, 10
rng = np.random.default_rng(seed=7)
data = rng.normal(size=(n, d)).astype("float32")
faiss.normalize_L2(data)
queries = rng.normal(size=(n_queries, d)).astype("float32")
faiss.normalize_L2(queries)

def build_and_time(factory: str):
    index = faiss.index_factory(d, factory, faiss.METRIC_INNER_PRODUCT)
    t0 = time.perf_counter()
    if not index.is_trained:
        index.train(data)
    index.add(data)
    build_s = time.perf_counter() - t0

    if factory.startswith("IVF"):
        index.nprobe = 8  # visit 8 of the 64 lists (default is 1)

    t0 = time.perf_counter()
    _, ids = index.search(queries, k)
    query_s = time.perf_counter() - t0
    return index, ids, build_s, query_s

ground_truth, gt_ids, _, _ = build_and_time("Flat")

def recall_at_k(candidate_ids: np.ndarray) -> float:
    hits = sum(
        len(set(candidate_ids[i]) & set(gt_ids[i]))
        for i in range(n_queries)
    )
    return hits / (n_queries * k)

for factory in ("Flat", "IVF64,Flat", "HNSW16"):
    _, ids, build_s, query_s = build_and_time(factory)
    recall = 1.0 if factory == "Flat" else recall_at_k(ids)
    print(f"{factory:12s} build={build_s:6.3f}s query={query_s:6.4f}s recall@{k}={recall:.3f}")

See the official index factory and Faster search wiki pages.

The shape to expect — the same trade-offs described in ANN Indexes: Flat, IVF, HNSW and Beyond — is:

  • Flat has the cheapest build (no training, no graph) and the slowest queries, and its recall is 1.0 by construction because it is the ground truth.

  • IVF64,Flat spends extra build time on k-means training, then answers queries faster than Flat because it scans only nprobe of the 64 lists; its recall rises towards 1.0 as nprobe grows and falls when a true neighbour sits in an unvisited list.

  • HNSW16 has the slowest build (it inserts every vector into a graph) and usually the fastest queries at high recall, tuned at query time with index.hnsw.efSearch, at the cost of extra memory for the graph edges.

Synthetic Gaussian vectors are a pessimistic case for approximate indexes, because they have none of the cluster structure real embeddings have, so expect recall on real DocsAssistant embeddings to be higher at the same settings. Always re-measure on your own hardware and your own embedding distribution before trusting any number.

What FAISS does not give you

FAISS solves nearest-neighbour search and nothing else. It has no concept of:

  • Metadata / filtering — an index stores vectors and integer ids, not JSON payloads or columns; filtered search means maintaining a separate id → metadata mapping (a dict, a SQLite table, a pandas DataFrame) and joining results yourself, or pre/post-filtering candidate ids.

  • Transactions — add/remove_ids/write_index are not atomic together; a crash between updating the index and updating your metadata store leaves them inconsistent, with no rollback.

  • Updates — most indexes (notably IVF*) support removal only through remove_ids, and in-place vector updates are commonly implemented as remove-then-re-add; there is no UPDATE primitive.

  • Access control, networking, replication, durability guarantees — there is no server, no auth, no client/server protocol; every process that needs the index loads the whole file into its own memory.

This gap is exactly why real systems that need vector search and metadata, transactions or concurrent access either pair FAISS with an external store (as DocsAssistant keeps pgvector as the system of record and uses FAISS only as a disposable side index, with IndexFlatIP as the exact ground truth to measure the ANN indexes against) or embed vector search directly inside a database engine. sqlite-vss, an early SQLite extension that embedded FAISS itself as its search backend, is the historical example of this path; it has since been superseded by sqlite-vec, which does not vendor FAISS at all: its vec0 virtual tables run exact, brute-force KNN inside SQLite (no ANN index as of 0.1.x), trading search speed on large collections for transactional, single-file SQL semantics that FAISS itself does not provide (see Vector Search in SQLite).

For a disposable, single-process store that does not need any of the above, LangChain’s FAISS vector store wraps exactly this library plus a lightweight in-memory docstore for metadata:

from langchain_community.vectorstores import FAISS
from langchain_ollama import OllamaEmbeddings

embeddings = OllamaEmbeddings(model="nomic-embed-text")
store = FAISS.from_texts(
    texts=["HNSW indexes...", "pgvector basics...", "RAG with Spring AI..."],
    embedding=embeddings,
    metadatas=[
        {"source_path": "docs/vector-rag/hnsw.adoc", "component": "vector-rag"},
        {"source_path": "docs/vector-rag/pgvector.adoc", "component": "vector-rag"},
        {"source_path": "docs/spring-ai/rag.adoc", "component": "spring-ai"},
    ],
)
store.save_local("docsassistant_faiss_index")
reloaded = FAISS.load_local(
    "docsassistant_faiss_index", embeddings, allow_dangerous_deserialization=True
)
hits = reloaded.similarity_search("how do I speed up similarity search", k=5)

See the official LangChain FAISS vector store page.

Workflow

flowchart LR Data[Vectors] --> Train{train\nneeded?} Train -- "IVF / PQ" --> TrainStep[index.train] Train -- "Flat / HNSW" --> AddStep TrainStep --> AddStep[index.add /\nadd_with_ids] AddStep --> SearchStep[index.search /\nrange_search] AddStep --> Persist[write_index] Persist --> Reload[read_index] Reload --> SearchStep

In LangChain and Spring AI

LangChain

The FAISS vector store shown above is one of the concrete implementations behind LangChain’s generic vector-store abstraction — construction, similarity search, persistence and retriever conversion all go through the same interface documented in Vector stores; swapping FAISS for PGVector later is a one-line change to the constructor, not a rewrite of the surrounding agent or retrieval code.

Spring AI

There is no dedicated Spring AI FAISS starter: FAISS ships no server and no client/server wire protocol, so it does not fit Spring AI’s VectorStore auto-configuration model the way pgvector, Redis or Qdrant do. A Spring application that wants FAISS specifically would need to shell out to Python or call it through a JNI binding; in practice, projects on the JVM either use LangChain4j against a database-backed store or pick one of Spring AI’s supported VectorStore implementations instead. See VectorStore API for the general abstraction and the list of stores it does support out of the box.

Related pages

References