Persistence, Durability & Redis as a Primary Database

This content was generated with the assistance of AI and should be verified against the official Redis documentation before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

Examples on this page target Redis Open Source 8.10 (8.10.2) with the Jedis 8 and Lettuce 7.7 client libraries. It covers the two on-disk persistence mechanisms Redis offers — RDB snapshots and the append-only file (AOF) — how they combine, the WAIT/WAITAOF commands for making a write’s durability observable before replying to a caller, the Redis 8.10 BACKUP command family for taking a consistent backup without stopping writes, and what changes when Redis is asked to be a primary database rather than only a cache in front of one.

RDB: point-in-time snapshots

RDB persistence writes the whole dataset to a single compact binary file (dump.rdb by default) at configured save points — combinations of an elapsed time and a minimum number of changes, any one of which triggers a snapshot. save "" disables automatic snapshotting entirely, leaving only manually triggered snapshots (or none, for a pure cache with no persistence at all).

# redis.conf -- save a snapshot if at least 1 key changed in 900s,
# or at least 10 keys changed in 300s, or at least 10000 keys changed in 60s
save 900 1
save 300 10
save 60 10000

SAVE writes a snapshot synchronously on the command-executing thread, blocking every other client for the whole write — essentially never what you want against a live instance. BGSAVE instead forks the process: the child writes the new RDB file from its own copy-on-write view of memory while the parent keeps serving commands on the original thread, and the new file atomically replaces the old one only once it is complete. Because both save points and an explicit BGSAVE share this fork step, a very large dataset can still make fork() itself take long enough to be felt as a latency spike on the parent, even though the child’s write never blocks it.

127.0.0.1:6379> BGSAVE
Background saving started
127.0.0.1:6379> INFO persistence
# Persistence
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:0

RDB’s data-loss window is bounded by the interval between save points, not by any single command: whatever was written since the last completed snapshot is lost if the process dies before the next one, which is why RDB alone suits data that can tolerate losing the last few minutes far better than it suits a system of record. See BGSAVE and Redis persistence.

AOF: logging every write

The append-only file instead logs every write command as it happens, in the same wire format Redis itself speaks, and rebuilds the dataset by replaying that log on startup. It is off by default and turned on with appendonly yes.

# redis.conf
appendonly yes
appendfsync everysec

appendfsync controls how often the AOF is actually flushed and `fsync`ed to disk, independently of how often commands are appended to it in memory:

Value Behaviour

always

fsync after every write (batched across a pipeline or a burst of concurrent clients into one fsync). The safest setting and the slowest, since every write’s durability now depends on a disk round trip before the reply is sent.

everysec (default)

fsync once per second from a background thread, while the main thread keeps writing without waiting on it. Loses at most about one second of writes if the process dies, at close to RDB-level write throughput.

no

Never calls fsync itself; the kernel decides when buffered writes actually reach disk (typically every 30 seconds on Linux). Fastest, and the least durable of the three.

Since Redis 7.0, the AOF is multi-part: instead of one growing file, a base file (an RDB- or AOF-format snapshot of the dataset at the time of the last rewrite) and one or more incremental files live together in a directory (appenddirname, default appendonlydir), tracked by a manifest file that records which base and increment files currently make up the log. BGREWRITEAOF compacts the log the same way BGSAVE compacts a snapshot: it forks, the child writes a fresh base file containing only the minimal set of commands needed to reach the current dataset, the parent keeps appending new writes to a new increment file in the meantime, and the two are atomically swapped in via the manifest once the child finishes — so a rewrite never blocks writes and never risks losing data written during it.

127.0.0.1:6379> BGREWRITEAOF
Background append only file rewriting started
127.0.0.1:6379> INFO persistence
# Persistence
aof_enabled:1
aof_rewrite_in_progress:0
aof_last_bgrewrite_status:ok
aof_base_size:1231000
aof_current_size:1231512

See BGREWRITEAOF and Redis persistence for the full multi-part AOF format and rewrite mechanics.

RDB and AOF together

The two are not mutually exclusive: save points and appendonly yes can both be configured on the same instance, and Redis restarts from whichever one is more complete. With both enabled, a restart always loads the AOF, never the RDB file, since the AOF is guaranteed to be the more up-to-date of the two — the RDB file is then kept around for what it is genuinely good at: a compact, fast-to-restore point-in-time copy for off-box backups and disaster recovery, rather than the mechanism a restart actually relies on. Redis also avoids running an RDB save and an AOF rewrite at the same time, since both fork and both drive heavy disk I/O; whichever is requested second while the other is in progress is deferred until the first completes.

# redis.conf -- both mechanisms enabled
save 900 1
appendonly yes
appendfsync everysec

This combination is the closest Redis gets to the durability profile of a traditional relational database: snapshotting for cheap, restorable backups, AOF for sub-second recovery-point durability on restart. See Redis persistence ("Interactions between AOF and RDB persistence").

WAIT and WAITAOF: making durability observable

Both appendfsync everysec and asynchronous replication mean a command can reply OK to the client before the write it represents is actually durable anywhere but the primary’s memory. WAIT and WAITAOF let a caller block until a specific durability target is actually met, turning an implicit guarantee into one the application can check before, say, telling a customer their order succeeded.

WAIT numreplicas timeout blocks until at least numreplicas connected replicas have acknowledged having received and applied all write commands issued by the calling connection so far, or until timeout milliseconds pass (0 blocks forever); it says nothing about whether any of those replicas has actually fsynced the data to disk, only that they have it in memory.

127.0.0.1:6379> SET bookshop:cart:42 "..."
OK
127.0.0.1:6379> WAIT 1 2000
(integer) 1

WAITAOF numlocal numreplicas timeout is the disk-durability counterpart: it blocks until numlocal local fsyncs (0 or 1, since a single instance has only one local AOF) and numreplicas replica fsyncs have happened for every write issued by the connection so far, and returns the actual counts reached whether or not the targets were met before timeout. It requires AOF to be enabled wherever a local fsync is being waited on — asking for numlocal 1 against an instance with appendonly no fails outright, since there is no local AOF to fsync.

127.0.0.1:6379> SET bookshop:cart:42 "..."
OK
127.0.0.1:6379> WAITAOF 1 1 2000
1) (integer) 1
2) (integer) 1

Both commands return immediately with whatever counts have already been reached, rather than actually blocking, when issued inside MULTI/EXEC or a script, since blocking there would stall every other client behind the single command-executing thread. See WAIT and WAITAOF.

BACKUP: online backups without stopping writes

New in Redis 8.10, the BACKUP command family produces a self-contained, restorable backup while the instance keeps serving writes, without the operator having to hand-manage AOF rewrites or coordinate around BGSAVE. A backup is itself a multi-part-AOF-shaped artifact: a BASE snapshot, an INCR file of writes accumulated since, and a manifest tying the two together, all hard-linked into a dedicated backup directory (backupdirname, default backupdir, resolved under the server’s working directory dir) rather than copied, so opening the backup window costs no extra disk I/O up front.

Subcommand Effect

BACKUP START

Opens a backup window and produces a fresh BASE snapshot via an AOF rewrite, reusing the existing multi-part AOF if one is already running, or temporarily starting the AOF machinery (without changing the configured appendonly value) if it isn’t. Fails if the backup directory isn’t empty or another backup is already in progress.

BACKUP LIST

Returns the absolute paths of the immutable backup files pinned so far — just the BASE while a backup is still accumulating INCR data, then BASE, INCR and manifest once sealed.

BACKUP SEAL

Freezes the backup: flushes and fsyncs the INCR file, hard-links it alongside the BASE, and writes the standalone manifest. After this point the file set is complete and immutable.

BACKUP STATUS

Reports the current state of the backup state machine (idle, snapshotting, pending, incrementing, or sealed).

BACKUP CLEANUP

Removes the sealed backup’s files (once they have been safely copied elsewhere) and returns the state machine to idle.

BACKUP ABORT

Cancels a backup that has not been sealed yet.

A typical backup, taken and shipped off-box, looks like this against a single node:

127.0.0.1:6379> BACKUP START
OK
127.0.0.1:6379> BACKUP STATUS
1) "state"
2) "incrementing"
127.0.0.1:6379> BACKUP LIST
1) "/data/backupdir/appendonly.aof.1.base.rdb"
127.0.0.1:6379> BACKUP SEAL
OK
127.0.0.1:6379> BACKUP LIST
1) "/data/backupdir/appendonly.aof.1.base.rdb"
2) "/data/backupdir/appendonly.aof.1.incr.aof"
3) "/data/backupdir/appendonly.aof.manifest"

After copying the three files reported by BACKUP LIST to durable off-box storage, BACKUP CLEANUP releases them from the backup directory. Because starting a backup is decoupled from sealing it, a control plane managing several Bookshop shards can stagger BACKUP START across nodes so they don’t all fork at once, let each accumulate INCR data independently, then seal and copy each in turn.

Restoring is a startup-time operation driven by the preload-file configuration setting, given as <type>:<path> where <type> is aof or rdb. Pointed at a sealed backup’s manifest, it loads exactly that BASE+INCR file set — reflecting the dataset as of the BACKUP SEAL call, not merely the earlier BACKUP START snapshot — instead of the server’s normally configured appenddirname/dump.rdb:

# redis.conf, on the instance being restored
preload-file aof:/data/backupdir/appendonly.aof.manifest
$ redis-server /path/to/redis.conf
* Loading RDB base file from preload-file...
* Loading AOF increments from preload-file...
* Done loading

preload-file can equally point at a lone RDB file (preload-file rdb:/path/to/backup.rdb) for a plain snapshot restore. See BACKUP START, BACKUP SEAL, and Redis persistence ("Online backups with the BACKUP command family") for the full state machine and restore procedure.

The data-loss window, profile by profile

No persistence at all loses everything on any unclean stop — appropriate only for data that is purely derived or cache, never for anything Bookshop treats as a system of record. RDB alone bounds the loss window to the gap between the last completed save point and the crash, typically minutes with a realistic save schedule; a snapshot every five minutes means up to five minutes of writes can vanish. appendfsync everysec narrows that to about one second, at write latency close to RDB’s, which is why it’s the default and the right starting point for most workloads that care about durability at all. appendfsync always narrows it further, to essentially nothing beyond the write itself, at the cost of an fsync round trip on (batches of) every write — a real latency tax, not a free upgrade. RDB and AOF together inherit AOF’s tighter restart recovery point while keeping RDB’s cheap, portable snapshots for backup and disaster recovery, so the two address different failure scenarios rather than duplicating the same one. WAIT/WAITAOF don’t change any of these windows on their own; they let a caller confirm, for one specific write, that a chosen durability target was actually reached before treating that write as safe — turning a statistical guarantee into a per-request check where it matters enough to pay the latency for it.

Timeline diagram comparing the data-loss window of four Redis durability profiles — RDB-only with five-minute save points, AOF everysec, AOF always, and RDB plus AOF together — showing how much of the most recent writes each profile can lose if the process crashes at an arbitrary point
flowchart TD Start["Choosing a persistence profile"] --> Q1{"Can this data be regenerated\nor re-fetched from elsewhere?"} Q1 -- yes, pure cache --> NoPersist["No persistence\n(or RDB only, best-effort)"] Q1 -- no --> Q2{"How much data loss\nis tolerable on crash?"} Q2 -- minutes are fine --> RDBOnly["RDB only\nsave points every few minutes"] Q2 -- about a second --> AOFSec["AOF, appendfsync everysec"] Q2 -- effectively none --> Q3{"Is the write-latency cost\nof fsync-per-write acceptable?"} Q3 -- yes --> AOFAlways["AOF, appendfsync always\n(+ WAITAOF for critical writes)"] Q3 -- no, need throughput too --> RDBAOF["RDB + AOF everysec\n+ WAITAOF on critical writes only"] RDBOnly --> Backups["Plus BACKUP command family\nfor off-box, restorable backups"] AOFSec --> Backups AOFAlways --> Backups RDBAOF --> Backups

Redis as a primary database

Redis is increasingly used as a system of record in its own right — not only a cache in front of one — and that shift changes which guarantees actually matter.

BASE, not ACID

A relational database’s ACID transaction gives atomicity, consistency, isolation and durability across an arbitrary set of statements, enforced by the engine itself. Redis instead sits closer to the BASE model (Basically Available, Soft state, Eventual consistency) that distributed key-value and document stores generally favor: high availability and low latency are prioritized, and strict cross-key consistency is something the application layer opts into rather than gets by default.

What Redis does give you: every single command is atomic, because the single-threaded execution model means no other command can interleave with it. MULTI/EXEC queues several commands and executes them back to back with nothing else interleaved — atomic in the sense that nothing runs between them — but it is not ROLLBACK-capable: a command that fails at execution time (a type error, for instance) does not undo the commands before it in the same transaction, unlike a relational ROLLBACK on constraint violation. Lua scripts and Redis Functions extend the same single-threaded atomicity to arbitrary read-modify-write logic spanning several keys, which is the usual way to get transaction-like behavior beyond what MULTI/EXEC alone provides. There is no isolation level to choose and no concept of a dirty read across keys the way a relational engine defines one — reads simply see whatever the last completed atomic operation left behind.

The latency cost of durability knobs

Every durability guarantee above this line is paid for in latency, and running Redis as a primary database means deciding, per write path, how much of that cost is worth paying:

Knob Cost

appendfsync always

An fsync round trip (batched, but still a disk write) before every write replies, typically the single largest latency cost among these knobs.

WAITAOF on a critical write

Blocks the calling connection until the target fsync/replica counts are met, adding however long that takes on top of the write itself — pay it selectively, for writes that actually need the confirmation, not on every write.

min-replicas-to-write / min-replicas-max-lag

Rejects writes outright once fewer than the configured number of replicas are known connected and within the configured lag, trading write availability for a floor on how far a failover can silently lose data — not a latency cost on the happy path, but an availability one when replicas fall behind or drop.

A workable default is to run most writes at appendfsync everysec for throughput, and reach for WAITAOF (or WAIT against replicas, if AOF durability alone isn’t the concern) only on the specific write paths — an order confirmation, a payment record — where the extra latency is worth the tighter durability guarantee.

Consistency under failover

Redis replication is asynchronous by default: a primary acknowledges a write before any replica has applied it, which means a failover triggered right after such a write can promote a replica that never received it, silently losing it. min-replicas-to-write narrows, but does not eliminate, this window by refusing writes once too few replicas are keeping up, and WAIT/WAITAOF narrow it further still for the specific writes a caller chooses to confirm — but neither turns Redis replication into the synchronous, quorum-committed replication a distributed relational or consensus-based store provides by default. Sentinel and Cluster failover both pick a promotion candidate from the replicas available at that moment, not necessarily the one with the most recent data if replicas are lagging unevenly; see Replication, Sentinel & Cluster for how a promotion is actually chosen.

Modelling a primary store with JSON and the Query Engine

Used as a primary store, Bookshop’s bookshop:book:<isbn> documents are the natural fit: each book is a JSON document under a stable key, and the Query Engine’s idx:books index (built with FT.CREATE over bookshop:book:*) gives it the secondary-index and query capability a relational primary store would get from ordinary WHERE/ORDER BY clauses — filtering by genre or price range, faceted counts, full-text search over the title — without a separate database to keep in sync. This works well for the access patterns the index actually covers; it does not give Redis a query planner or ad hoc join capability comparable to a relational engine, so a Bookshop feature that needs arbitrary multi-entity joins is still better served by a relational store, whether alone or alongside Redis.

When to pair Redis with a system of record

Redis earns its place as the sole store for data whose value comes from low-latency, high-throughput access and whose consistency needs are met by per-command and per-transaction atomicity — a cart, a session, a rate limiter, a leaderboard. It is a weaker fit as the only store for data that needs multi-entity transactional integrity beyond what Lua/Functions conveniently express, long-term analytical querying, or regulatory retention and audit guarantees a relational engine’s tooling already covers. The common pattern in that case is not "Redis instead of a relational database" but Redis in front of one — a cache, a read-optimized JSON/Query-Engine projection, or a fast path for the subset of operations that genuinely need Redis’s latency profile — with the relational store remaining the system of record for everything else. See Choosing the Right Database for the general framework behind that decision, including when a second engine is worth its operational cost at all.

  • Architecture, Keyspace & Eviction — the single-threaded execution model that underlies command-level atomicity, and maxmemory-policy (relevant to why eviction and persistence pull in opposite directions on a primary-store instance).

  • Replication, Sentinel & Cluster — how replicas, Sentinel, and Cluster failover actually pick and promote a replica, referenced above.

  • JSON — the JSON document type behind bookshop:book:<isbn>.

  • Query Engine: Indexing & Data Modeling — the idx:books index used for secondary-index-style querying over JSON documents.

  • Choosing the Right Database — the general framework for picking a database category and deciding when to pair a specialized store with a system of record.

References