Indexing, CRUD, bulk & concurrency control

This section documents the current Elasticsearch 9.x line (with 8.19 as the final 8.x release) as published at the Elasticsearch documentation, which is the reference these pages are written and verified against. No specific patch version is pinned. Some capabilities (Kibana-only UIs, the ML/NLP model-management workflow, cross-cluster replication, and parts of the paid / serverless-only surface) are linked, not documented in depth.

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production, as Elasticsearch iterates quickly.

This section’s bibliography lists the reference material consulted while preparing these pages.

Elasticsearch writes are document-oriented: you address one JSON document by index and id, and the shard applies the change, bumps its bookkeeping, and replicates it. This page covers the single-document APIs, the batch APIs (_bulk, _mget, _update_by_query, _delete_by_query, _reindex), how to make a read-modify-write safe with if_seq_no / if_primary_term, and what the refresh parameter costs. The family overview is at Document APIs.

Single-document APIs

Index a document: create or replace

PUT /<index>/_doc/<id> creates the document or replaces it wholesale if the id already exists. POST /<index>/_doc (no id) lets Elasticsearch assign a URL-safe id. PUT /<index>/_create/<id> is create-only — it returns 409 if the id exists, which is how you make an insert idempotent.

# Create or fully replace, explicit id.
PUT /products/_doc/p-1
{
  "name": "Trail shoe",
  "price": 89.0,
  "in_stock": 42
}

# Auto-generated id.
POST /products/_doc
{
  "name": "Wool sock",
  "price": 12.0
}

# Create-only: 409 if p-1 already exists.
PUT /products/_create/p-1
{ "name": "Trail shoe" }
# https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html

Every write response carries _version, _seq_no and _primary_term (used for concurrency control, below) and result (created or updated). There is one mapping type per index, always _doc; the 1.x custom type name in the path is gone.

Read: GET and HEAD

GET /products/_doc/p-1          # full document + metadata (_seq_no, _primary_term, _version)
GET /products/_source/p-1       # the _source only
HEAD /products/_doc/p-1         # 200 if it exists, 404 if not, no body
GET /products/_doc/p-1?_source_includes=name,price
# https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-get.html

A GET by id is real-time: it reads the translog if the document is not yet in a refreshed segment, so it sees a write that a search would not see yet.

Delete

DELETE /products/_doc/p-1
# https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete.html

A delete writes a tombstone; the space is reclaimed on segment merge, not immediately.

Partial and scripted updates

POST /<index>/_update/<id> reads the current _source, applies your change on the data node, and re-indexes the whole document — no client round trip for the merge. Pass a doc to merge fields, or a script (Painless) to compute the new state. retry_on_conflict=<n> makes the update itself re-read and re-apply on a version conflict, which is the common way to run a scripted counter under contention.

# Partial update: shallow-merge these fields into the stored _source.
POST /products/_update/p-1
{
  "doc": {
    "price": 79.0,
    "on_sale": true
  }
}

# Scripted update: decrement a counter in place, retrying on conflict.
POST /products/_update/p-1?retry_on_conflict=3
{
  "script": {
    "source": "ctx._source.in_stock -= params.sold",
    "params": { "sold": 3 }
  }
}
# https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update.html

The Painless language, ctx, and params are covered on Query languages & scripting.

Upserts

An upsert updates the document if it exists and inserts a starting document if it does not.

# doc + upsert: merge "doc" if p-1 exists, otherwise index the "upsert" body.
POST /products/_update/p-1
{
  "doc": { "price": 79.0 },
  "upsert": { "name": "Trail shoe", "price": 79.0, "in_stock": 0 }
}

# doc_as_upsert: shorthand for "use doc as the insert body too".
POST /products/_update/p-1
{
  "doc": { "price": 79.0 },
  "doc_as_upsert": true
}

# scripted_upsert: run the script even on the insert path (ctx.op == "create").
POST /products/_update/p-1
{
  "script": {
    "source": "ctx._source.in_stock = (ctx._source.in_stock ?: 0) + params.delta",
    "params": { "delta": 5 }
  },
  "upsert": {},
  "scripted_upsert": true
}
# https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update.html#upserts

Multi-document APIs

The _bulk NDJSON format

POST /_bulk takes newline-delimited JSON (NDJSON), not a JSON array: one compact action line, then a source line for index / create / update (none for delete). Every line ends with \n, including the last, and the request is sent as application/x-ndjson. One _bulk call can mix operations and indices.

POST /_bulk
{ "index":  { "_index": "products", "_id": "p-1" } }
{ "name": "Trail shoe", "price": 89.0 }
{ "create": { "_index": "products", "_id": "p-2" } }
{ "name": "Wool sock", "price": 12.0 }
{ "update": { "_index": "products", "_id": "p-1" } }
{ "doc": { "price": 79.0 } }
{ "delete": { "_index": "products", "_id": "p-3" } }
curl -s -H 'Content-Type: application/x-ndjson' \
  -XPOST 'localhost:9200/_bulk' --data-binary @requests.ndjson
# --data-binary (not -d) so curl keeps the newlines
# https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html

Bulk error handling is per item: one bad item does not fail the batch. Check the top-level errors flag, then walk items for any status >= 400.

{
  "took": 7,
  "errors": true,
  "items": [
    { "index":  { "_id": "p-1", "status": 200, "_seq_no": 12, "_primary_term": 3 } },
    { "create": { "_id": "p-2", "status": 409,
                  "error": { "type": "version_conflict_engine_exception" } } },
    { "update": { "_id": "p-1", "status": 200, "result": "updated" } },
    { "delete": { "_id": "p-3", "status": 404, "result": "not_found" } }
  ]
}

A 429 on an item means the write queue is full — retry just those items with backoff. Most clients ship a bulk helper that batches, retries 429, and reports failures for you.

_mget

_mget fetches many documents by id in one request.

GET /_mget
{
  "docs": [
    { "_index": "products", "_id": "p-1" },
    { "_index": "products", "_id": "p-2", "_source": ["name"] }
  ]
}

# Single index in the path -> just ids.
GET /products/_mget
{ "ids": ["p-1", "p-2"] }
# https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-multi-get.html

_update_by_query and _delete_by_query

These run a query, then update (with an optional script) or delete every match. Elasticsearch takes a snapshot of the index at the start and applies each write with the document’s own if_seq_no / if_primary_term, so a document changed after the snapshot causes a version conflict. By default the first conflict aborts the job; conflicts=proceed counts them in version_conflicts and keeps going. slices=auto parallelises the scan by shard.

# Re-price in-stock products by 10%, continuing past conflicts, sliced by shard.
POST /products/_update_by_query?conflicts=proceed&slices=auto
{
  "query": { "term": { "on_sale": true } },
  "script": {
    "source": "ctx._source.price = Math.round(ctx._source.price * 0.9)"
  }
}

# Delete everything under a threshold.
POST /products/_delete_by_query?conflicts=proceed
{
  "query": { "range": { "price": { "lt": 1 } } }
}
# https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update-by-query.html
# https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete-by-query.html

Add wait_for_completion=false to get a task id back immediately and poll GET /_tasks/<task-id>; long jobs should always run this way.

_reindex

_reindex copies documents from one index to another — the standard way to apply a mapping change, change the shard count, or split/merge indices. The source can carry a query to copy a subset; dest.op_type: create skips ids that already exist. A script transforms each document in flight, and dest.pipeline runs it through an ingest pipeline.

# Local copy of a subset into a fresh index.
POST /_reindex
{
  "source": { "index": "products", "query": { "range": { "price": { "gte": 10 } } } },
  "dest":   { "index": "products-v2", "op_type": "create" }
}

# Rename a field while copying.
POST /_reindex
{
  "source": { "index": "products" },
  "dest":   { "index": "products-v2" },
  "script": { "source": "ctx._source.stock = ctx._source.remove('in_stock')" }
}

# Transform every document through an ingest pipeline.
POST /_reindex
{
  "source": { "index": "products" },
  "dest":   { "index": "products-v2", "pipeline": "enrich-products" }
}

# Remote: pull from another cluster (its host must be in reindex.remote.whitelist).
POST /_reindex
{
  "source": {
    "remote": { "host": "https://old-cluster:9200", "username": "u", "password": "p" },
    "index": "products"
  },
  "dest": { "index": "products" }
}
# https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html

Optimistic concurrency control

Elasticsearch has no row locks. Instead every write to a shard advances a per-shard sequence number and every document records the _seq_no and _primary_term of the write that last touched it. To make a read-modify-write safe, send the pair you read back as if_seq_no and if_primary_term on the write: the shard applies it only if the document is still at that version, otherwise it returns HTTP 409 version_conflict_engine_exception. The 1.x version parameter for internal concurrency is removed — use the _seq_no / _primary_term pair.

GET /products/_doc/p-1
# response includes: "_seq_no": 12, "_primary_term": 3, "_source": { "in_stock": 42, ... }

# Apply only if nobody else has written p-1 since seq_no 12.
PUT /products/_doc/p-1?if_seq_no=12&if_primary_term=3
{
  "name": "Trail shoe",
  "in_stock": 41
}
# Lost the race -> 409 -> re-GET and retry.
# https://www.elastic.co/guide/en/elasticsearch/reference/current/optimistic-concurrency-control.html
{
  "error": {
    "type": "version_conflict_engine_exception",
    "reason": "[p-1]: version conflict, required seqNo [12], primary term [3], current document has seqNo [15] and primary term [3]"
  },
  "status": 409
}

The client owns the retry loop: re-read the document, re-apply the change to the fresh version, re-send with the new _seq_no / _primary_term, and give up after a bounded number of attempts.

flowchart TD R[GET /idx/_doc/id] --> M[Read _seq_no + _primary_term and _source] M --> A[Apply the change in the client] A --> W[PUT with if_seq_no and if_primary_term] W --> C{HTTP status} C -- 200 / 201 --> D[Done] C -- 409 version conflict --> B{Retry budget left?} B -- yes --> R B -- no --> F[Fail: surface the conflict]

This is the same read-modify-retry contract that Couchbase CAS enforces with a per-document CAS token, and that MongoDB update operators sidestep by mutating fields server-side in one atomic step. A single _update with a script or $inc-style operator avoids the loop entirely for simple field arithmetic.

External versioning: syncing from a system of record

When the source of truth is elsewhere — a SQL row with a monotonic version column or an updated_at timestamp — pass that number as version with version_type=external. Elasticsearch accepts the write only if the supplied number is greater than the stored version, so replays and out-of-order deliveries are dropped as 409 and you never need to read before writing. version_type=external_gte also accepts an equal number.

# n is supplied by the source system (e.g. epoch-millis of the row's last change).
PUT /products/_doc/p-1?version=170459999000&version_type=external
{
  "name": "Trail shoe",
  "price": 79.0
}
# A later out-of-order replay with version=170000000000 is stale -> 409, ignored.
# https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#index-versioning

The refresh parameter

Elasticsearch is near-real-time: a write is durable in the translog immediately but not visible to search until the shard’s next refresh, controlled by index.refresh_interval (default 1s). The refresh parameter on a write chooses how much to pay for immediate visibility:

Value Behaviour

false (default)

Return as soon as the write is safe in the translog. Visible at the next scheduled refresh. Cheapest.

wait_for

Hold the response until a refresh makes this write visible (bounded by refresh_interval). Forces no extra refresh; concurrent wait_for writes share the same refresh.

true

Force an immediate refresh of the affected shards. Visible at once, but creates a tiny segment and invalidates caches every call — use only in tests or one-off fixes.

 
PUT /products/_doc/p-1?refresh=wait_for
{ "name": "Trail shoe", "in_stock": 41 }

POST /products/_bulk?refresh=true
{ "index": { "_index": "products", "_id": "p-2" } }
{ "name": "Wool sock" }
# https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-refresh.html

refresh=true in a hot write path is the classic throughput mistake — each forced refresh adds a segment the background merger then has to reclaim. For a bulk load, set index.refresh_interval to -1 for the duration and restore it afterwards; see Performance tuning for the full indexing-throughput checklist.