Storage internals, security & administration

The Data service is memory-first. Every read and write goes through the managed cache, an in-process cache sized by the bucket’s RAM quota. Item metadata for every key is pinned in the cache at all times; values are held while there is room. The set of values currently in cache is the working set, and the share of items whose value is resident is the resident ratio. When the cache fills, the least-recently-used clean items are ejected (value dropped, metadata kept); a later read of an ejected key fetches its value from disk. A collapsing resident ratio is the primary signal that a bucket needs more RAM or a Magma backend.

Writes are acknowledged from the cache and drained to disk asynchronously through the disk write queue. The queue absorbs bursts; a persistently long queue means disk cannot keep up with the write rate. Durable writes and cbbackupmgr care about what has drained — see Concurrency, locking & durable writes.

Two on-disk storage engines back a Couchbase bucket:

Ephemeral buckets skip the storage engine entirely — data lives only in the managed cache and its replicas, so a full-cluster restart loses it. They give predictable latency with no disk I/O and are suited to sessions, caches, and transient state; eviction is configured as noEviction (reject writes when full) or nruEviction (drop not-recently-used items).

Compaction reclaims the space that appended, superseded blocks leave behind. Auto-compaction triggers it per file once fragmentation crosses a threshold; it can also be scheduled for a quiet window.

Deletes and expirations do not erase an item immediately; they write a tombstone (key + metadata, no value) so the removal can replicate to replicas and over XDCR. The metadata purge interval (per bucket, three days by default) is how long tombstones are retained before compaction sweeps them.

DCP — the Database Change Protocol — is the single ordered mutation stream every downstream consumer reads: replicas, GSI, Search, Analytics, Eventing, XDCR, and the Kafka/Spark connectors. It replaced the older TAP protocol and adds restart-from-sequence-number so a consumer that disconnects resumes without a full resync.

After a restart a Data node performs warm-up: it loads all key metadata (and, depending on the engine and quota, values) back into the managed cache before the node reports healthy and starts serving. Warm-up time scales with item count, which is one reason Magma — with less resident metadata — restarts faster on large data sets.

The memory-first cache with an asynchronous drain and periodic compaction is the same shape as MongoDB’s WiredTiger cache plus checkpoints — see MongoDB WiredTiger — with Couchbase making the cache an explicit, quota-sized tier and exposing the resident ratio as a first-class health metric.

Authorisation is role-based access control (RBAC). A user (or a group the user belongs to) is granted roles; each role is a privilege (read data, write data, run a query, manage a bucket, …​) optionally scoped to a bucket, scope, or collection. Couchbase ships a large set of built-in roles — data_reader, data_writer, query_select, bucket_admin, cluster_admin, security_admin, ro_admin, and many more — and custom roles cannot be defined, so least privilege is expressed by combining narrowly scoped built-in roles.

Authentication is either local (credentials stored in the cluster) or external:

Transport security: enable TLS for client connections and, separately, node-to-node encryption so intra-cluster traffic (replication, rebalance, DCP) is also encrypted. --cluster-encryption-level strict refuses all non-TLS ports.

Encryption at rest (Enterprise Edition) protects the data files, typically layered with OS or volume encryption; it is linked from the disclaimer, not detailed here. Auditing records administrative and optionally data events to a rotating JSON log for review.

The _system scope is a reserved per-bucket scope Couchbase uses for internal collections (transactions, Eventing, mobile). Do not create application collections in it; RBAC on it is managed by the server.

Network hardening: bind admin ports to a management network, put client ports behind a firewall that allows only application subnets, and open only the ports a deployment actually uses (8091/18091 console, 11210/11207 KV, 8093/18093 Query, 8094 Search, plus the rest per service).

Security checklist:

For the relational-database parallel, see MongoDB Security; the concepts (authentication sources, RBAC, TLS, auditing) line up, with Couchbase scoping roles down to the collection.

Backup and restore is the Backup service (schedules and a repository, configurable in the console) with cbbackupmgr as its command-line form. cbbackupmgr does incremental, compressed, resumable backups and is the current tool; the earlier cbbackup, cbrestore, and cbtransfer utilities are removed from current releases and their use cases are covered by cbbackupmgr plus cbimport / cbexport.

Bulk import / export moves JSON and CSV between files and a collection:

Monitoring: GET /pools/default returns cluster and per-node health; the stats REST API and the per-service /metrics endpoint expose time series in Prometheus exposition format for scraping; cbstats reads low-level Data-service counters directly from a node; logs land under the install’s logs/ directory and named alerts (auto-failover fired, disk nearly full, metadata overhead high, IP changed) surface in the console and can be emailed.

The Autonomous Operator runs Couchbase on Kubernetes and OpenShift. A CouchbaseCluster custom resource declares size, per-node services, volumes, and backup schedules; the Operator reconciles the running cluster to it — performing adds, removes, rebalances, failovers, rolling upgrades, and scheduled backups without manual couchbase-cli steps.

For the equivalent operational surface on the other database in this section, see MongoDB Administration.

Continue with SDKs, connectors & Couchbase Mobile.