Horizontal scaling with sharded clusters
|
This section documents the current MongoDB 8.x server line as published at the MongoDB Server Manual, which is the reference these pages are written and verified against. No specific patch version is pinned. Some capabilities (Atlas Search, Atlas Vector Search, and parts of encryption and backup) are Atlas-only — they are linked, not documented in depth. This content was generated with the assistance of AI and should be verified against the official manual before being relied on in production, since MongoDB iterates quickly. This section’s bibliography lists the reference material consulted while preparing these pages. |
Sharding partitions one collection’s documents across several independent servers so that data volume and write throughput can grow past what a single replica set handles. The application does not address shards directly: it talks to a router that knows, from the shard key, where every document lives. This page covers the cluster’s parts, the kinds of shard key, how to pick one, and the balancer that keeps chunks evenly spread.
Cluster components
A sharded cluster has three kinds of process:
-
Shards — each shard is a full replica set (see Replica sets & high availability) and holds a subset of the sharded data. Non-sharded collections all live on one shard, the database’s primary shard.
-
mongosrouters — stateless query routers. A client connects to amongos, which reads the shard key from each query, consults the routing table, and forwards the operation to just the shard(s) that can hold matching documents, merging their replies. -
Config servers — a dedicated replica set (the CSRS) that stores cluster metadata: the list of shards, the chunk ranges, and zone definitions.
mongoscaches this and refreshes it when it changes.
mongodb://mongos1:27017,mongos2:27017/shop?retryWrites=true&w=majority
A client always uses a mongos connection string, never a shard’s own. See
Sharding for the full architecture.
Enabling sharding on a collection
Sharding is opt-in per collection. Enable it for the database, then shard the collection on a key; the key fields must be backed by an index (see Indexes).
// https://www.mongodb.com/docs/manual/reference/method/sh.shardCollection/
sh.enableSharding("shop")
// ranged key
db.orders.createIndex({ customerId: 1, orderDate: 1 })
sh.shardCollection("shop.orders", { customerId: 1, orderDate: 1 })
// hashed key
db.events.createIndex({ _id: "hashed" })
sh.shardCollection("shop.events", { _id: "hashed" })
Shard keys
The shard key is one or more indexed fields whose value decides which chunk — and therefore which shard — a document belongs to.
-
Ranged (
{ field: 1 }) — documents are ordered by the key and split into contiguous ranges. Range queries on the key hit only the shards covering that range, but a key whose values keep increasing sends every new document to the same range. -
Hashed (
{ field: "hashed" }) — MongoDB hashes the key value and ranges over the hash. Inserts spread evenly even when the underlying value is monotonic, at the cost of range queries, which must now scatter to every shard. -
Compound (
{ a: 1, b: 1 }) — ranged over the fields in order; a high-cardinality second field can rescue a low-cardinality first field.
A chunk is a contiguous span of the key space owned by one shard. The balancer splits a chunk once it grows past the configured chunk size (default 128 MB) and migrates whole chunks between shards. A collection starts as a single chunk covering the entire key range and accumulates more as data grows.
The shard key value of a document may be changed by an update since MongoDB 4.2 (the update must include the
full key and use w: "majority"), but the set of fields in the shard key is fixed once the collection is
sharded. You can only refine it — append more fields with sh.refineCollectionShardKey() — or change it
wholesale by resharding (below).
// https://www.mongodb.com/docs/manual/reference/method/sh.refineCollectionShardKey/
db.orders.createIndex({ customerId: 1, orderDate: 1, _id: 1 })
sh.refineCollectionShardKey("shop.orders", { customerId: 1, orderDate: 1, _id: 1 })
Choosing a shard key
A good key spreads both storage and, especially, writes evenly, while still letting common queries target a single shard. Judge a candidate on:
-
High cardinality — many distinct values, so the key space can be divided into many chunks.
countryCodecaps the cluster at ~200 chunks;userIddoes not. -
Low frequency — no single value dominates. If 40% of documents share one key value, that value’s chunk cannot be split and its shard becomes a hotspot regardless of cardinality.
-
Non-monotonic — a key that always increases (
ObjectId, a timestamp, an auto-increment counter) makes the highest chunk the target of every insert; that one chunk and shard take the entire write load while the others sit idle. This is the ascending shard key problem. Either put a hashed key on the monotonic field, or lead the compound key with a well-distributed field. -
Query isolation — the fields your frequent queries filter on should be in the key, so
mongosroutes those queries to one shard instead of scattering to all of them.
// https://www.mongodb.com/docs/manual/core/sharding-choose-a-shard-key/
// createdAt alone: monotonic -> insert hotspot on the last shard
// { deviceId: 1, createdAt: 1 }: writes spread across devices, and
// time-range queries for one device still hit a single shard
db.readings.createIndex({ deviceId: 1, createdAt: 1 })
sh.shardCollection("iot.readings", { deviceId: 1, createdAt: 1 })
// pure write-scatter with no natural partition field: hash the id
db.audit.createIndex({ _id: "hashed" })
sh.shardCollection("iot.audit", { _id: "hashed" })
The balancer, zones, and resharding
The balancer is a background process on the config server primary. It watches the chunk count per shard and, when the gap between the most- and least-loaded shard exceeds a threshold, migrates one chunk at a time from the fuller shard to the emptier one until the counts even out. Reads and writes continue against both shards during a migration.
// https://www.mongodb.com/docs/manual/core/sharding-balancer-administration/
sh.status() // shards, databases, per-collection chunk distribution
sh.getBalancerState() // is balancing enabled
sh.startBalancer() / sh.stopBalancer()
sh.disableBalancing("shop.orders") // freeze one collection, e.g. during a bulk load
Zone sharding pins ranges of the shard key to named zones, and zones to shards, so documents physically land on chosen shards — used for data locality (keep EU customers on EU-hosted shards) or tiered hardware (recent data on SSD shards). The balancer then honours the zone ranges as well as the even-count goal.
// https://www.mongodb.com/docs/manual/core/zone-sharding/
sh.addShardToZone("shard-eu", "EU")
sh.updateZoneKeyRange(
"shop.orders",
{ countryCode: "AT", customerId: MinKey },
{ countryCode: "DE", customerId: MaxKey },
"EU"
)
Resharding rebuilds a sharded collection under a completely new shard key. reshardCollection copies the
data into a new distribution in the background and does a brief cutover at the end; the collection stays
readable and writable throughout. It is the escape hatch when a shard key turns out to be a bad choice.
// https://www.mongodb.com/docs/manual/core/sharding-reshard-a-collection/
db.adminCommand({
reshardCollection: "shop.orders",
key: { customerId: "hashed" }
})