Indexing with the Index service (and legacy Views)

This section documents the current Couchbase Server 7.6.x line as published at the Couchbase Server documentation, which is the reference these pages are written and verified against. No specific patch version is pinned. Some capabilities (Enterprise-Edition-only Analytics, auditing, encryption at rest, the Backup service and rack-zone awareness, and Capella-only App Services and Columnar) are linked, not documented in depth.

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production, as Couchbase iterates quickly.

This section’s bibliography lists the reference material consulted while preparing these pages.

Couchbase separates the storage of indexes from the storage of documents: the Index service maintains secondary indexes on its own nodes, and the Query service uses them to turn a SQL++ WHERE clause into an index seek plus a targeted document fetch. This page covers the Global Secondary Index (GSI) types and options, how to read the plan the optimizer produces, and the legacy MapReduce Views feature that GSI replaced.

The Index service and Global Secondary Indexes

A Global Secondary Index is a copy of one or more document fields (and any literal or computed key expressions), kept sorted in a B-tree-style structure on Index-service nodes and mapping each index key to a document ID. It is "global" because one logical index is maintained independently of the Data-service partitioning — unlike the old per-node Views — so a scan does not have to fan out to every data node. The Query service seeks or range-walks the index, then asks the Data service for the matching documents by ID.

Left-to-right pipeline: a SQL++ predicate feeds an IndexScan on the Index service, which emits index keys and document IDs to a Fetch on the Data service, which returns the projected row to the client

See Global Secondary Indexes for the architecture and CREATE INDEX for the full statement grammar.

The primary index

-- Every document ID in the keyspace; no predicate needed to build it.
CREATE PRIMARY INDEX ON `travel-sample`.inventory.airline USING GSI;

-- It lets an unplanned, ad-hoc query run at all:
SELECT * FROM `travel-sample`.inventory.airline WHERE callsign = "UNITED";
-- With no primary index and no other usable index this fails with
--   "No index available on keyspace ... that matches your query"

A primary index indexes every key in the keyspace. It makes exploratory queries "just work" in development, but for any predicate it cannot satisfy directly it degrades to a full scan of every key. A production keyspace should carry named secondary indexes for its real query patterns and no primary index; keep the primary index for development and the interactive shell only. See the CREATE INDEX reference.

Secondary and composite indexes

-- Single-field secondary index.
CREATE INDEX idx_city ON `travel-sample`.inventory.hotel(city);

-- Composite (compound) index: the leading key must be usable by the predicate.
CREATE INDEX idx_city_name ON `travel-sample`.inventory.hotel(city, name);

-- Serves:  WHERE city = ?          and  WHERE city = ? AND name = ?
-- Does not serve:  WHERE name = ?   (name is not a leading key)

Composite key order is load-bearing in the same way as a relational compound index: put fields matched by equality first, then a field used for ordering, then a field matched by a range. The CREATE INDEX page documents the index-key expression syntax, which may include functions (CREATE INDEX ix ON b(LOWER(name))) and a trailing INCLUDE MISSING to keep documents that omit the leading key.

Partial (WHERE-clause) indexes

-- Index only the documents that match the index WHERE clause.
CREATE INDEX idx_open_orders ON orders(customerId) WHERE status = "OPEN";

SELECT * FROM orders WHERE customerId = 42 AND status = "OPEN";  -- uses idx_open_orders
SELECT * FROM orders WHERE customerId = 42;                      -- cannot: filter not implied

A partial index stores keys for a subset of the keyspace, so it is smaller and cheaper to maintain. The optimizer uses it only when the query predicate implies the index filter. See the WHERE clause in CREATE INDEX.

Array indexing

-- Index each element of an array with DISTINCT ARRAY ... FOR ... IN ... END
CREATE INDEX idx_sched_day ON `travel-sample`.inventory.route
  (DISTINCT ARRAY s.day FOR s IN schedule END);

-- The ANY ... SATISFIES predicate is now an index scan, not a keyspace scan:
SELECT META().id FROM `travel-sample`.inventory.route
WHERE ANY s IN schedule SATISFIES s.day = 2 END;

DISTINCT ARRAY collapses duplicate values within one document; ALL ARRAY keeps them. A composite array index can index several fields per element with FLATTEN_KEYS, and mix array keys with scalar keys:

CREATE INDEX idx_sched ON `travel-sample`.inventory.route
  (ALL ARRAY FLATTEN_KEYS(s.day, s.utc) FOR s IN schedule END, sourceairport);

See Indexing Arrays for the full form, including nested arrays and array indexes on the document root.

Adaptive indexes

-- One index that covers equality predicates on ANY field...
CREATE INDEX idx_adaptive ON `travel-sample`.inventory.hotel(DISTINCT PAIRS(SELF));

-- ...or a named subset of fields:
CREATE INDEX idx_adaptive_sub ON `travel-sample`.inventory.hotel
  (DISTINCT PAIRS({city, country, state, type}));

An adaptive index turns each field/value pair into an index entry, so one index answers equality predicates on many fields without a separate composite index per combination. It suits sparse, unpredictable ad-hoc filtering; it costs more to write and store than a targeted index, so it is a convenience, not a default. See Global Secondary Indexes.

Covering indexes

-- If every field the query reads is in the index key, the Data-service fetch is skipped.
CREATE INDEX idx_cover ON `travel-sample`.inventory.hotel(city, name, country);

EXPLAIN SELECT name, country FROM `travel-sample`.inventory.hotel WHERE city = "Paris";
-- plan shows an IndexScan3 with a "covers" array and NO Fetch operator

A covered query is the pipeline in the figure above with step 3 removed: the index alone has the answer. Add the projected fields to the tail of the index key to make an otherwise hot query covering. See CREATE INDEX.

Partitioned indexes and index replicas

-- Hash-partition one index across Index nodes; keep one extra copy of each partition.
CREATE INDEX idx_part ON orders(customerId, orderDate)
  PARTITION BY HASH(customerId)
  WITH { "num_partition": 8, "num_replica": 1 };

PARTITION BY HASH spreads a single large index’s storage and scan load across Index nodes so no one node holds the whole index. num_replica keeps N additional copies of every partition on other nodes for high availability and read scale-out; a lost Index node is covered by the replica while the index is rebuilt. See Index Partitioning.

Deferred build

-- Create several indexes cheaply, deferring the scan that populates them.
CREATE INDEX idx_a ON orders(a) WITH { "defer_build": true };
CREATE INDEX idx_b ON orders(b) WITH { "defer_build": true };

-- Then build them together in a single pass over the keyspace:
BUILD INDEX ON orders(idx_a, idx_b) USING GSI;

Deferring lets you define many indexes and then populate them in one keyspace scan instead of one scan per index, which matters when bootstrapping a keyspace or restoring after import. Check readiness with SELECT * FROM system:indexes WHERE state != "online". See CREATE INDEX for defer_build and the BUILD INDEX companion statement.

Reading query plans: EXPLAIN, ADVISE and the optimizer

EXPLAIN and ADVISE

-- EXPLAIN: print the plan without running the statement.
EXPLAIN SELECT name FROM `travel-sample`.inventory.hotel WHERE city = "Paris";

-- ADVISE: ask the Index Advisor which indexes would help this statement.
ADVISE SELECT name FROM `travel-sample`.inventory.hotel
WHERE city = "Paris" AND country = "France";
-- => recommended: CREATE INDEX adv_city_country ON ...(`city`, `country`)

EXPLAIN returns the operator tree the Query service will execute; ADVISE (and the equivalent Index Advisor in the Query Workbench) analyses the predicate and suggests covering or composite indexes. Both are documented under the Cost-Based Optimizer and the SQL++ language reference.

The cost-based optimizer and UPDATE STATISTICS

-- Collect distribution statistics so the CBO can cost candidate plans.
UPDATE STATISTICS FOR `travel-sample`.inventory.hotel(city, country, type);

-- ANALYZE is an accepted synonym:
ANALYZE KEYSPACE `travel-sample`.inventory.hotel(city, country);

From Couchbase Server 7.0 the Query service has a cost-based optimizer (CBO): given statistics on the number of documents, distinct values, and value distribution per index key, it estimates the rows each plan would touch and picks the cheapest, rather than following fixed rules. With no statistics it falls back to the older rule-based heuristics. Refresh statistics after a large data change, and force a specific index with USE INDEX (idx_name USING GSI) when a plan is still wrong. See Cost-Based Optimizer.

IndexScan vs. PrimaryScan, covering vs. fetch

EXPLAIN SELECT META().id FROM orders WHERE status = "OPEN";

Read the operator tree top-down:

  • PrimaryScan3 — the plan is walking the primary index, i.e. every key in the keyspace. This is almost always a missing secondary index and is the SQL++ equivalent of a MongoDB COLLSCAN.

  • IndexScan3 — a secondary GSI drove the query. Its spans entry shows the seek and range bounds the optimizer derived from the predicate; a narrow span over few keys means the index is selective.

  • A Fetch operator after the scan means the Data service loaded whole documents (steps 3-4 in the figure). A covers array on the IndexScan3 and no Fetch is a covered query.

For the same ideas on the document-database side — IXSCAN vs. COLLSCAN, the Equality-Sort-Range key order, covered queries and explain() — see MongoDB Indexing. The statements these plans come from are covered in SQL++ querying.

Legacy MapReduce Views

Before the Query and Index services (Couchbase Server 4.0), secondary access was provided by MapReduce Views: JavaScript map and optional reduce functions stored in design documents per bucket, producing a sorted view index that clients queried by key or key range over HTTP.

// Design document "_design/orders" holding one view, "by_status"
{
  "views": {
    "by_status": {
      "map": "function (doc, meta) { if (doc.type == 'order') emit(doc.status, doc.total); }",
      "reduce": "_sum"
    }
  }
}
// Query: GET /orders/_design/orders/_view/by_status?key="OPEN"&stale=false
  • map calls emit(key, value) once per matching document; the view index is those pairs sorted by key. reduce (_count, _sum, _stats, or a custom function) aggregates over a key range.

  • stale=false forces the view up to date before answering; ok and update_after trade freshness for latency. This is the Views analogue of query scan consistency.

  • Spatial views indexed GeoJSON bounding boxes for geo lookups.

Views are *deprecated in favour of GSI + SQL* and are retained only so that pre-4.x applications remain readable and can be migrated. They should not be used for new development: GSI indexes are global rather than a per-node scatter-gather, are maintained incrementally by a dedicated service, and are queried with the same SQL used for everything else, including the geo and vector queries that replace spatial views. The current documentation at the Views introduction supersedes any older material that presents Views as a primary indexing mechanism; on any discrepancy, the current server documentation wins.