Indexing with the Index service (and legacy Views)
|
This section documents the current Couchbase Server 7.6.x line as published at the Couchbase Server documentation, which is the reference these pages are written and verified against. No specific patch version is pinned. Some capabilities (Enterprise-Edition-only Analytics, auditing, encryption at rest, the Backup service and rack-zone awareness, and Capella-only App Services and Columnar) are linked, not documented in depth. This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production, as Couchbase iterates quickly. This section’s bibliography lists the reference material consulted while preparing these pages. |
Couchbase separates the storage of indexes from the storage of documents: the Index service maintains
secondary indexes on its own nodes, and the Query service uses them to turn a SQL++ WHERE clause into an
index seek plus a targeted document fetch. This page covers the Global Secondary Index (GSI) types and
options, how to read the plan the optimizer produces, and the legacy MapReduce Views feature that GSI
replaced.
The Index service and Global Secondary Indexes
A Global Secondary Index is a copy of one or more document fields (and any literal or computed key expressions), kept sorted in a B-tree-style structure on Index-service nodes and mapping each index key to a document ID. It is "global" because one logical index is maintained independently of the Data-service partitioning — unlike the old per-node Views — so a scan does not have to fan out to every data node. The Query service seeks or range-walks the index, then asks the Data service for the matching documents by ID.
See Global Secondary Indexes for the architecture and CREATE INDEX for the full statement grammar.
The primary index
-- Every document ID in the keyspace; no predicate needed to build it. CREATE PRIMARY INDEX ON `travel-sample`.inventory.airline USING GSI; -- It lets an unplanned, ad-hoc query run at all: SELECT * FROM `travel-sample`.inventory.airline WHERE callsign = "UNITED"; -- With no primary index and no other usable index this fails with -- "No index available on keyspace ... that matches your query"
A primary index indexes every key in the keyspace. It makes exploratory queries "just work" in development, but for any predicate it cannot satisfy directly it degrades to a full scan of every key. A production keyspace should carry named secondary indexes for its real query patterns and no primary index; keep the primary index for development and the interactive shell only. See the CREATE INDEX reference.
Secondary and composite indexes
-- Single-field secondary index. CREATE INDEX idx_city ON `travel-sample`.inventory.hotel(city); -- Composite (compound) index: the leading key must be usable by the predicate. CREATE INDEX idx_city_name ON `travel-sample`.inventory.hotel(city, name); -- Serves: WHERE city = ? and WHERE city = ? AND name = ? -- Does not serve: WHERE name = ? (name is not a leading key)
Composite key order is load-bearing in the same way as a relational compound index: put fields matched by
equality first, then a field used for ordering, then a field matched by a range. The
CREATE INDEX page
documents the index-key expression syntax, which may include functions (CREATE INDEX ix ON b(LOWER(name)))
and a trailing INCLUDE MISSING to keep documents that omit the leading key.
Partial (WHERE-clause) indexes
-- Index only the documents that match the index WHERE clause. CREATE INDEX idx_open_orders ON orders(customerId) WHERE status = "OPEN"; SELECT * FROM orders WHERE customerId = 42 AND status = "OPEN"; -- uses idx_open_orders SELECT * FROM orders WHERE customerId = 42; -- cannot: filter not implied
A partial index stores keys for a subset of the keyspace, so it is smaller and cheaper to maintain. The
optimizer uses it only when the query predicate implies the index filter. See the WHERE clause in
CREATE INDEX.
Array indexing
-- Index each element of an array with DISTINCT ARRAY ... FOR ... IN ... END CREATE INDEX idx_sched_day ON `travel-sample`.inventory.route (DISTINCT ARRAY s.day FOR s IN schedule END); -- The ANY ... SATISFIES predicate is now an index scan, not a keyspace scan: SELECT META().id FROM `travel-sample`.inventory.route WHERE ANY s IN schedule SATISFIES s.day = 2 END;
DISTINCT ARRAY collapses duplicate values within one document; ALL ARRAY keeps them. A composite array
index can index several fields per element with FLATTEN_KEYS, and mix array keys with scalar keys:
CREATE INDEX idx_sched ON `travel-sample`.inventory.route (ALL ARRAY FLATTEN_KEYS(s.day, s.utc) FOR s IN schedule END, sourceairport);
See Indexing Arrays for the full form, including nested arrays and array indexes on the document root.
Adaptive indexes
-- One index that covers equality predicates on ANY field...
CREATE INDEX idx_adaptive ON `travel-sample`.inventory.hotel(DISTINCT PAIRS(SELF));
-- ...or a named subset of fields:
CREATE INDEX idx_adaptive_sub ON `travel-sample`.inventory.hotel
(DISTINCT PAIRS({city, country, state, type}));
An adaptive index turns each field/value pair into an index entry, so one index answers equality predicates on many fields without a separate composite index per combination. It suits sparse, unpredictable ad-hoc filtering; it costs more to write and store than a targeted index, so it is a convenience, not a default. See Global Secondary Indexes.
Covering indexes
-- If every field the query reads is in the index key, the Data-service fetch is skipped. CREATE INDEX idx_cover ON `travel-sample`.inventory.hotel(city, name, country); EXPLAIN SELECT name, country FROM `travel-sample`.inventory.hotel WHERE city = "Paris"; -- plan shows an IndexScan3 with a "covers" array and NO Fetch operator
A covered query is the pipeline in the figure above with step 3 removed: the index alone has the answer. Add the projected fields to the tail of the index key to make an otherwise hot query covering. See CREATE INDEX.
Partitioned indexes and index replicas
-- Hash-partition one index across Index nodes; keep one extra copy of each partition.
CREATE INDEX idx_part ON orders(customerId, orderDate)
PARTITION BY HASH(customerId)
WITH { "num_partition": 8, "num_replica": 1 };
PARTITION BY HASH spreads a single large index’s storage and scan load across Index nodes so no one node
holds the whole index. num_replica keeps N additional copies of every partition on other nodes for high
availability and read scale-out; a lost Index node is covered by the replica while the index is rebuilt. See
Index
Partitioning.
Deferred build
-- Create several indexes cheaply, deferring the scan that populates them.
CREATE INDEX idx_a ON orders(a) WITH { "defer_build": true };
CREATE INDEX idx_b ON orders(b) WITH { "defer_build": true };
-- Then build them together in a single pass over the keyspace:
BUILD INDEX ON orders(idx_a, idx_b) USING GSI;
Deferring lets you define many indexes and then populate them in one keyspace scan instead of one scan per
index, which matters when bootstrapping a keyspace or restoring after import. Check readiness with
SELECT * FROM system:indexes WHERE state != "online". See
CREATE INDEX for
defer_build and the BUILD INDEX companion statement.
Reading query plans: EXPLAIN, ADVISE and the optimizer
EXPLAIN and ADVISE
-- EXPLAIN: print the plan without running the statement. EXPLAIN SELECT name FROM `travel-sample`.inventory.hotel WHERE city = "Paris"; -- ADVISE: ask the Index Advisor which indexes would help this statement. ADVISE SELECT name FROM `travel-sample`.inventory.hotel WHERE city = "Paris" AND country = "France"; -- => recommended: CREATE INDEX adv_city_country ON ...(`city`, `country`)
EXPLAIN returns the operator tree the Query service will execute; ADVISE (and the equivalent Index
Advisor in the Query Workbench) analyses the predicate and suggests covering or composite indexes. Both are
documented under the
Cost-Based
Optimizer and the SQL++ language reference.
The cost-based optimizer and UPDATE STATISTICS
-- Collect distribution statistics so the CBO can cost candidate plans. UPDATE STATISTICS FOR `travel-sample`.inventory.hotel(city, country, type); -- ANALYZE is an accepted synonym: ANALYZE KEYSPACE `travel-sample`.inventory.hotel(city, country);
From Couchbase Server 7.0 the Query service has a cost-based optimizer (CBO): given statistics on the
number of documents, distinct values, and value distribution per index key, it estimates the rows each plan
would touch and picks the cheapest, rather than following fixed rules. With no statistics it falls back to
the older rule-based heuristics. Refresh statistics after a large data change, and force a specific index
with USE INDEX (idx_name USING GSI) when a plan is still wrong. See
Cost-Based
Optimizer.
IndexScan vs. PrimaryScan, covering vs. fetch
EXPLAIN SELECT META().id FROM orders WHERE status = "OPEN";
Read the operator tree top-down:
-
PrimaryScan3— the plan is walking the primary index, i.e. every key in the keyspace. This is almost always a missing secondary index and is the SQL++ equivalent of a MongoDBCOLLSCAN. -
IndexScan3— a secondary GSI drove the query. Itsspansentry shows the seek and range bounds the optimizer derived from the predicate; a narrow span over few keys means the index is selective. -
A
Fetchoperator after the scan means the Data service loaded whole documents (steps 3-4 in the figure). Acoversarray on theIndexScan3and noFetchis a covered query.
For the same ideas on the document-database side — IXSCAN vs. COLLSCAN, the Equality-Sort-Range key
order, covered queries and explain() — see MongoDB Indexing. The
statements these plans come from are covered in
SQL++ querying.
Legacy MapReduce Views
Before the Query and Index services (Couchbase Server 4.0), secondary access was provided by MapReduce
Views: JavaScript map and optional reduce functions stored in design documents per bucket, producing
a sorted view index that clients queried by key or key range over HTTP.
// Design document "_design/orders" holding one view, "by_status"
{
"views": {
"by_status": {
"map": "function (doc, meta) { if (doc.type == 'order') emit(doc.status, doc.total); }",
"reduce": "_sum"
}
}
}
// Query: GET /orders/_design/orders/_view/by_status?key="OPEN"&stale=false
-
mapcallsemit(key, value)once per matching document; the view index is those pairs sorted by key.reduce(_count,_sum,_stats, or a custom function) aggregates over a key range. -
stale=falseforces the view up to date before answering;okandupdate_aftertrade freshness for latency. This is the Views analogue of query scan consistency. -
Spatial views indexed GeoJSON bounding boxes for geo lookups.
Views are *deprecated in favour of GSI + SQL* and are retained only so that pre-4.x applications remain readable and can be migrated. They should not be used for new development: GSI indexes are global rather than a per-node scatter-gather, are maintained incrementally by a dedicated service, and are queried with the same SQL used for everything else, including the geo and vector queries that replace spatial views. The current documentation at the Views introduction supersedes any older material that presents Views as a primary indexing mechanism; on any discrepancy, the current server documentation wins.
See also
Continue with Full-Text & Vector Search, Analytics & Eventing.