Query Engine: Indexing & Data Modeling

This content was generated with the assistance of AI and should be verified against the official Redis documentation before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

Examples on this page target Redis Open Source 8.10 (8.10.2) with the Jedis 8 and Lettuce 7.7 client libraries. The Redis Query Engine (the module formerly branded RediSearch) turns Redis into a secondary-index engine over hash and JSON keys: instead of scanning or maintaining lookup sets by hand, an index declared once with FT.CREATE is kept up to date automatically as matching keys are written, and can then be queried, aggregated, or searched by vector similarity. This page covers building and maintaining that index; the query and aggregation syntax that runs against it is covered in Query Engine: Search & Aggregations, and similarity search over the VECTOR field is covered in Vector Search: Query Engine & Vector Sets.

The Bookshop catalog index

The Bookshop catalog stores each book as a JSON document under bookshop:book:<isbn>, with fields $.isbn, $.title, $.author, $.genres[], $.price, $.year, $.description, $.stock, and a 384-dimension $.embedding vector. A single index, idx:books, indexes that prefix, and every other page in this section that queries, aggregates, or vector-searches the catalog assumes this same index and schema exist.

127.0.0.1:6379> FT.CREATE idx:books ON JSON PREFIX 1 bookshop:book:
    SCHEMA
        $.isbn AS isbn TAG
        $.title AS title TEXT WEIGHT 5 SORTABLE WITHSUFFIXTRIE
        $.author AS author TEXT WEIGHT 2 NOSTEM PHONETIC dm:en
        $.genres[*] AS genres TAG SEPARATOR , CASESENSITIVE
        $.price AS price NUMERIC SORTABLE
        $.year AS year NUMERIC SORTABLE
        $.stock AS stock NUMERIC
        $.description AS description TEXT INDEXMISSING INDEXEMPTY
        $.warehouse_location AS warehouse_location GEO
        $.embedding AS embedding VECTOR HNSW 6 TYPE FLOAT32 DIM 384 DISTANCE_METRIC COSINE
OK

This is the canonical idx:books schema referenced throughout the rest of this section. It derives from FT.CREATE; the field-type options it uses are explained below.

FT.CREATE: declaring an index

FT.CREATE index [ON HASH \| JSON] [PREFIX count prefix [prefix …​]] [FILTER expression] [LANGUAGE language] [SCORE_DECIMALS decimals] SCHEMA field-spec [field-spec …​] declares a new index and starts an asynchronous backfill over every existing key that already matches its prefix (and optional FILTER expression); once the backfill finishes, every subsequent write to a matching key updates the index automatically. ON JSON (the default target for this page) indexes JSON documents by JSONPath expression; ON HASH indexes flat hash field names instead. PREFIX count prefix [prefix …​] restricts indexing to keys whose name starts with one of the given prefixes — bookshop:book: above — so unrelated keys in the same keyspace are never touched. See FT.CREATE.

Each field-spec is <json-path AS attribute \| hash-field> <TEXT \| TAG \| NUMERIC \| GEO \| GEOSHAPE \| VECTOR> [options…​], repeated once per indexed field. On a JSON index, AS renames the JSONPath to a short attribute name used in queries — $.warehouse_location AS warehouse_location above, for instance, so a query refers to warehouse_location rather than repeating the path. On a hash index, the hash field name is used directly and AS is only needed to alias it to something shorter.

Field types

TEXT fields

A TEXT field is tokenized, stemmed, and scored for full-text search. title and author above are TEXT:

  • WEIGHT weight — multiplies this field’s contribution to a document’s relevance score (default 1); title is weighted 5 above so a title match outranks an author or description match on the same query.

  • NOSTEM — disables stemming for this field, so run no longer also matches running; useful for a field, such as author, where stemming a proper name would garble it rather than help matching.

  • PHONETIC matcher — indexes a phonetic encoding of the field alongside its literal tokens, so a query can match names that sound alike but are spelled differently; dm:en above is the English Double Metaphone matcher.

  • SORTABLE — keeps an extra, unindexed copy of the field’s value solely so queries can SORTBY it without a slower post-hoc sort; title is SORTABLE above.

  • WITHSUFFIXTRIE — builds a suffix trie alongside the normal inverted index, so a wildcard query anchored at the end of a term (a suffix or infix match) resolves in the trie instead of scanning every indexed term; title uses it above to support "contains" style title search.

See FT.CREATE and Redis Query Engine scoring.

TAG fields

A TAG field holds one or more exact-match labels, indexed as whole values rather than tokenized — the right choice for an ISBN, a genre, a status code, or anything queried by equality rather than relevance. isbn and genres above are TAG:

  • SEPARATOR char — the character that splits a single string value into multiple tags (default ,); genres sets it explicitly above, matching the default, to document the intent. A JSON array field such as $.genres[*] is naturally multi-valued and needs no separator, since each array element already becomes its own tag.

  • CASESENSITIVE — by default tag values are lower-cased before indexing and matching, so Fiction and fiction are the same tag; CASESENSITIVE preserves case, which genres uses above since a bookshop frontend may want to display and filter on exact genre casing.

See FT.CREATE and Redis tag fields.

NUMERIC fields

A NUMERIC field indexes a number for range queries and, optionally, SORTBY. price, year, and stock above are all NUMERIC, with price and year also marked SORTABLE for catalog pages that sort by either. See FT.CREATE.

GEO and GEOSHAPE fields

A GEO field indexes a single longitude,latitude point for radius queries — warehouse_location above records which fulfilment warehouse a stock line ships from, letting a query filter to warehouses within a given radius of a delivery address. GEOSHAPE is the polygon counterpart, indexing an arbitrary shape (given as WKT) rather than a point, for queries such as "is this delivery address inside this shipping zone" — declared as GEOSHAPE [FLAT \| SPHERICAL] to choose a flat-earth or spherical coordinate system. The Bookshop schema above only needs point warehouse locations, so it uses GEO; a delivery-zone field would instead be declared, for example, as $.delivery_zone AS delivery_zone GEOSHAPE SPHERICAL. See FT.CREATE and Redis geospatial indexing.

VECTOR fields

embedding above is a VECTOR field indexing each book’s 384-dimension embedding for similarity search. Declaring and querying a VECTOR field — algorithm choice, distance metric, and KNN/range queries — is covered in full in Vector Search: Query Engine & Vector Sets; this page only establishes the field as part of the canonical idx:books schema.

INDEXMISSING and INDEXEMPTY

By default, a document where a field is entirely absent, or a TEXT/TAG field whose value is an empty string or empty array, is simply left out of that field’s index — there is no way to query for "documents missing this field" or "documents with an empty value". description above adds both modifiers:

  • INDEXMISSING — lets a query match documents where the field is absent entirely, via ismissing(\@description).

  • INDEXEMPTY — lets a query match documents where the field is present but empty.

Both are opt-in per field since they add index overhead; add them only to fields a query genuinely needs to filter on missingness or emptiness for. See FT.CREATE.

JSONPath aliasing and multi-value fields

Every field above is declared as a JSONPath ($.title, $.genres[], …​) followed by AS attribute: the path is what the indexer reads from each JSON document, and the attribute is the short name every query uses instead of repeating the path. $.genres[] AS genres is a multi-value field — the [*] wildcard means each element of the genres array is indexed as its own tag, so a book with ["fiction", "mystery"] matches a query for either genre independently, not as one combined value. The same wildcard pattern applies to any JSON array field on a TAG, TEXT, or NUMERIC index, not just genres.

Indexing a hash-backed dataset

Not every Bookshop key is JSON — Hashes models the shopping cart as a hash, and a catalog could equally be stored as one hash per book instead of a JSON document. Indexing that shape uses ON HASH and references flat hash field names instead of JSONPaths:

127.0.0.1:6379> FT.CREATE idx:books_hash ON HASH PREFIX 1 bookshop:bookhash:
    SCHEMA
        title TEXT WEIGHT 5 SORTABLE
        genres TAG SEPARATOR ,
        price NUMERIC SORTABLE
        year NUMERIC SORTABLE
        embedding VECTOR HNSW 6 TYPE FLOAT32 DIM 384 DISTANCE_METRIC COSINE
OK

The field options (WEIGHT, SORTABLE, SEPARATOR, the vector algorithm parameters, and so on) behave identically on a hash index; the only structural difference from the JSON schema above is that there is no JSONPath or AS to declare, and no wildcard multi-value syntax — a hash field is always a single scalar string, so a "multi-value" tag field on a hash relies entirely on SEPARATOR splitting one delimited string rather than indexing a JSON array. This page’s remaining examples continue to use the JSON-backed idx:books. See FT.CREATE.

Inspecting and evolving an index

FT.INFO index reports an index’s schema, document and record counts, and indexing status — the first stop when checking whether a backfill has finished or why a field isn’t matching as expected:

127.0.0.1:6379> FT.INFO idx:books
 1) index_name
 2) idx:books
 3) index_options
 4) (empty array)
 5) index_definition
 6) 1) key_type
    2) JSON
    3) prefixes
    4) 1) bookshop:book:
 7) attributes
 8) ...
 9) num_docs
10) "3"
11) indexing
12) (integer) 0

See FT.INFO.

FT.ALTER index SCHEMA ADD field-spec [field-spec …​] adds one or more fields to an existing index without rebuilding it from scratch; existing documents are updated lazily as they’re next touched, or immediately for a full-text backfill of the new field depending on server version. It can only add fields, not remove or change the type of an existing one — changing an existing field’s type needs the re-index procedure below.

127.0.0.1:6379> FT.ALTER idx:books SCHEMA ADD $.publisher AS publisher TAG
OK

See FT.ALTER.

Aliases

An alias is an alternate name application code queries by, decoupled from the concrete index name behind it — the mechanism the re-indexing procedure below relies on to cut over without downtime:

  • FT.ALIASADD alias index — points a new alias at an existing index; querying alias behaves identically to querying index directly. See FT.ALIASADD.

  • FT.ALIASUPDATE alias index — repoints an existing alias at a different index atomically, in one call — the step that actually performs a zero-downtime cutover. See FT.ALIASUPDATE.

  • FT.ALIASDEL alias — removes an alias without touching the index it pointed to. See FT.ALIASDEL.

  • FT.ALIASLIST — lists every alias currently defined and the index each one resolves to. See FT.ALIASLIST.

127.0.0.1:6379> FT.ALIASADD books idx:books
OK
127.0.0.1:6379> FT.ALIASLIST
1) 1) "books"
   2) "idx:books"

Application code should query the books alias, never idx:books directly, so a later re-index (below) is invisible to it.

FT.DROPINDEX and zero-downtime re-indexing

FT.DROPINDEX index [DD] removes an index definition; the DD flag also deletes the documents it indexed, which is almost never what’s wanted for a search index over data that lives independently in Redis — omitting DD drops only the index structures and leaves every bookshop:book:* key untouched. See FT.DROPINDEX.

A schema change that FT.ALTER can’t express — changing a field’s type, its SORTABLE/CASESENSITIVE flags, or the vector algorithm’s parameters — needs a full re-index. Doing this against the live idx:books name would leave queries failing for the duration of the backfill; routing traffic through the books alias avoids that entirely: build the new index under a new name, let its backfill complete, repoint the alias in one atomic call, then drop the old index once nothing references it.

127.0.0.1:6379> FT.CREATE idx:books_v2 ON JSON PREFIX 1 bookshop:book:
    SCHEMA
        $.isbn AS isbn TAG
        $.title AS title TEXT WEIGHT 5 SORTABLE WITHSUFFIXTRIE
        $.author AS author TEXT WEIGHT 2 NOSTEM PHONETIC dm:en
        $.genres[*] AS genres TAG SEPARATOR , CASESENSITIVE
        $.price AS price NUMERIC SORTABLE
        $.year AS year NUMERIC SORTABLE
        $.stock AS stock NUMERIC
        $.description AS description TEXT INDEXMISSING INDEXEMPTY
        $.warehouse_location AS warehouse_location GEO
        $.embedding AS embedding VECTOR HNSW 6 TYPE FLOAT32 DIM 384 DISTANCE_METRIC COSINE
OK
127.0.0.1:6379> FT.INFO idx:books_v2
...
127.0.0.1:6379> FT.ALIASUPDATE books idx:books_v2
OK
127.0.0.1:6379> FT.DROPINDEX idx:books
OK

The FT.INFO call between creating idx:books_v2 and cutting over is a manual stand-in for polling indexing until it reports 0 — an application would wait on that condition (or a known-fixed delay for a small catalog) before calling FT.ALIASUPDATE, so the alias only ever points at a fully backfilled index.

flowchart TD A["FT.CREATE idx:books_v2 (new schema)"] --> B["backfill completes\n(FT.INFO indexing = 0)"] B --> C["FT.ALIASUPDATE books idx:books_v2"] C --> D["queries via alias 'books'\nnow serve from idx:books_v2"] D --> E["FT.DROPINDEX idx:books\n(old index, no DD)"] style A fill:#eef,stroke:#88a style C fill:#efe,stroke:#8a8 style E fill:#fee,stroke:#a88

In Lettuce, the same cutover, waiting for the backfill before repointing the alias:

package com.example.bookshop.search;

RedisCommands<String, String> commands = connection.sync();

commands.ftCreate("idx:books_v2", CreateOptions.<String, String>builder()
                .on(CreateOptions.DataType.JSON)
                .withPrefix("bookshop:book:")
                .build(),
        buildBooksSchema());

while (commands.ftInfo("idx:books_v2").getIndexing()) {
    Thread.sleep(500);
}

commands.ftAliasupdate("books", "idx:books_v2");
commands.ftDropindex("idx:books");

This derives from FT.ALIASUPDATE and FT.DROPINDEX.

GC, memory, and index-management best practices

An index’s inverted lists, tag sets, and (for VECTOR fields) HNSW graph are kept in memory alongside the keys they index, roughly doubling the memory footprint of an indexed dataset — MAXMEMORY-constrained deployments should size for the data and every index over it, not the data alone. Garbage collection of stale index entries (left behind when a document is deleted or a value it indexed changes) runs incrementally in the background rather than synchronously with the write, so a burst of deletes shows a transient bump in index size that a subsequent GC pass reclaims rather than an immediate one.

A few practices keep index management predictable:

  • Prefer few, broad indexes with a narrow PREFIX over many narrow ones — each index carries its own fixed overhead (schema metadata, GC bookkeeping), so ten small indexes cost more than one covering the same keyspace.

  • Always query through an alias (books above), never a concrete index name, so a future schema change is a re-index behind the alias rather than an application deploy.

  • Keep FT.ALTER for additive changes only, and reserve the alias-based re-index procedure above for anything FT.ALTER can’t express — retyping a field, changing vector algorithm parameters, or fixing a TAG field’s CASESENSITIVE/SEPARATOR settings.

  • Watch FT.INFO’s `hash_indexing_failures (or indexing_failures, depending on server version) after a bulk load — documents that fail to index (a malformed JSON path, a vector of the wrong dimension) are silently skipped rather than rejected at write time.

Related pages

References