StatefulSets

A stateless Pod (any Bookshelf Deployment) holds no data a client cares about surviving a restart — delete it, and an identical replacement picks up exactly where the last one left off, because there was nothing "in" it to lose. A stateful Pod holds data that took real work to produce (rows in a database, an index, replicated log positions), and that data has gravity: it is far cheaper to move the compute to where the data already sits than to move the data itself every time a Pod is rescheduled. That is the property a bare Deployment cannot express at all — its Pods are fungible, and a fresh Deployment-managed Pod for a database has no way to reattach to the disk its predecessor was writing to, or to be found by the same name a replication protocol was told to dial. StatefulSets exist to make three specific things stable across rescheduling: identity (name and network address), storage (which disk a given identity always gets back), and order (which Pod comes up, or goes down, first).

A StatefulSet names its Pods <statefulset-name>-<ordinal>, counting up from 0 (or from spec.ordinals.start, below) — postgres-0, postgres-1, postgres-2, never a random suffix. The ordinal is reused: deleting postgres-1 and letting the controller recreate it produces another Pod still named postgres-1, with the same identity a client or a replication protocol was told about. That name only becomes reachable by other Pods, however, once it is paired with a headless Service — an ordinary Service with clusterIP: None — set as the StatefulSet’s spec.serviceName. A normal Service’s DNS name resolves to one virtual IP that load-balances across every matching Pod; a headless Service’s DNS name resolves to the individual Pod IPs directly, and, because it is named in serviceName, the StatefulSet controller also registers one DNS record per Pod, following the pattern <pod-name>.<service-name>.<namespace>.svc.cluster.local. For the Bookshelf PostgreSQL StatefulSet below, that is postgres-0.postgres.bookshelf.svc.cluster.local, postgres-1.postgres.bookshelf.svc.cluster.local, and so on — addresses stable enough for a replication topology (a primary’s connection string, a replica’s primary_conninfo) to hard-code, unlike a Deployment Pod’s IP, which changes on every reschedule.

Source: Kubernetes Docs — Headless Services.

A Pod’s own spec.volumes list is shared by whatever ReplicaSet or StatefulSet manages it — every replica gets an identical volume. A stateful Pod instead needs one PersistentVolumeClaim per ordinal, each bound to its own PersistentVolume, so postgres-1 always reattaches to the same disk it wrote to last, never the disk postgres-0 or postgres-2 wrote to. spec.volumeClaimTemplates is what makes that automatic: the controller creates one PVC per template per ordinal, named <template-name>-<statefulset-name>-<ordinal> (data-postgres-0, data-postgres-1, …), and mounts it into the matching Pod every time that ordinal is (re)created. See Persistent Volumes and Storage Classes for how a StorageClass provisions the PersistentVolume each claim binds to.

By default, deleting the StatefulSet or scaling it down does not delete these PVCs — a deliberate safety choice, since deleting a StatefulSet is not the same decision as deleting its data. spec.persistentVolumeClaimRetentionPolicy makes that behaviour explicit and configurable, with two independent policies: whenDeleted (the StatefulSet object itself is deleted) and whenScaled (the StatefulSet is scaled down, dropping the highest ordinals), each set to either Retain (the default — PVCs survive) or Delete (PVCs for the removed ordinals are deleted along with them).

The postgres-credentials Secret is the one created in Bookshelf End to End. From postgres:18 onwards the image stores its data in a version-specific PGDATA (/var/lib/postgresql/18/docker) and refuses to start if a volume is mounted at the old /var/lib/postgresql/data path, so the claim is mounted one level up.

Source: Kubernetes Docs — StatefulSet PersistentVolumeClaim Retention.

Source: Docker Official Images — postgres, PGDATA.

By default (OrderedReady) the controller creates, scales and deletes Pods one ordinal at a time, strictly in order, waiting for each one to become Running and Ready (see Health Probes and Container Lifecycle) before starting the next — postgres-0 must be ready before postgres-1 is created, and scaling down removes postgres-2 fully before touching postgres-1. That ordering matters for a primary/replica topology where a replica’s start-up script dials the primary by name and expects it already listening. podManagementPolicy: Parallel drops that guarantee entirely: every ordinal is launched, or terminated, at the same time, with no Pod waiting on another’s readiness. Parallel suits workloads that coordinate their own membership after the fact (a Cassandra or Elasticsearch-style ring that gossips to discover peers, rather than depending on postgres-0 existing before postgres-1 starts) and it noticeably shortens a full scale-out or restart, at the cost of every replica having to tolerate its peers not being up yet.

spec.updateStrategy controls what happens when spec.template changes, mirroring the Deployment strategies in ReplicaSets and Deployments but adapted to ordinals instead of an anonymous ReplicaSet swap.

Under RollingUpdate, rollingUpdate.partition (an ordinal number, default 0) restricts the update to Pods whose ordinal is greater than or equal to it — every Pod below the partition is left on the old template untouched. That turns a rolling update into a canary for a stateful workload: set partition: 2 on a 3-replica StatefulSet and only postgres-2 is updated, while postgres-0 and postgres-1 keep serving on the old image until the partition is lowered (or removed) to let the rest follow. rollingUpdate.maxUnavailable (an integer or percentage, defaulting to 1; behind the MaxUnavailableStatefulSet feature gate, alpha in 1.24-1.34, beta from 1.35 and enabled by default in 1.37) bounds how many Pods the rolling update is allowed to take down at once when podManagementPolicy: Parallel is also set — with the default OrderedReady policy it has no visible effect beyond 1, since Pods are already updated strictly one at a time.

Source: Kubernetes Docs — StatefulSet Rolling Updates.

spec.ordinals.start (an integer, default 0) shifts the first ordinal a StatefulSet assigns — replicas: 3 with ordinals.start: 5 produces postgres-5, postgres-6, postgres-7 instead of postgres-0..2. This is mainly a migration tool: it lets a new StatefulSet take over a numeric range that does not collide with an existing one covering the same headless Service and DNS names, so both can briefly coexist (for example while migrating replicas one at a time onto a new storage class) without one controller’s ordinal 2 fighting another’s ordinal 2 for the same PVC name and DNS record.

Putting the pieces above together, the Bookshelf PostgreSQL StatefulSet follows the same steps as the official basic StatefulSet tutorial: create the headless Service and the StatefulSet, watch the Pods come up in order, scale it, update it, then delete it.

Source: Kubernetes Docs — StatefulSet Basics and Kubernetes Docs — Running a Replicated Stateful Application.

A StatefulSet only automates the generic mechanics above — ordinals, one PVC per ordinal, ordered rollout — it has no idea that PostgreSQL replicas need a base backup taken before joining, that a failed primary needs a specific replica promoted and the others repointed at it, or that a minor-version upgrade needs a particular drain sequence. Scaling PostgreSQL beyond a single, manually wired primary/replica pair, or wanting automatic failover, backups and point-in-time recovery, is exactly the point where a purpose-built operator (a controller with a CustomResourceDefinition describing "a PostgreSQL cluster" as one object, covered in Custom Resources and Operators) earns its added complexity over a hand-written StatefulSet: it still creates a StatefulSet underneath, but it also encodes the domain-specific operational runbook — failover, backup scheduling, safe upgrades — that would otherwise be a stack of bespoke scripts around kubectl patch and kubectl exec. Stateful Applications in Practice picks this decision back up with a concrete PostgreSQL operator example.