Why Distributed Transactions Don’t Work Across Microservices
|
This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production; see the section bibliography for the reference material consulted while preparing these pages. |
Strategic Design draws one database per bounded context; Aggregates draws one transaction per aggregate inside that database. Both pages promise this page as the explanation for why — why a single ACID transaction spanning two services is not merely discouraged as a style choice but effectively unavailable as an engineering option once a domain is split this way. This page makes that case directly: what a monolith’s local transaction actually gives up at the service boundary, why the traditional fix (two-phase commit) breaks down for the infrastructure a microservice architecture actually runs on — Java 21, Spring Boot 4.1.x, Spring Data JPA, Spring Modulith 2.x, Spring for Apache Kafka 4.1.x — and what replaces it once the answer is "nothing replaces it, restructure the problem instead."
Database per service, and why a shared database of record does not fix this
Strategic Design already establishes that each bounded context owns its own database, queried and written only by that context’s own code. The tempting shortcut around that rule is to keep one shared database of record underneath several services anyway — one schema, several processes pointed at it — on the theory that a single database at least keeps ACID transactions available across "services" that are really just separately deployed callers of the same tables. Brown, Woolf, and Yoder’s Cloud Application Architecture Patterns names the alternative directly as Self-Managed Data Stores: each service manages its own persistent state, in its own store, never sharing a schema with another service, specifically because a shared database of record reintroduces every coupling the service split was meant to remove:
-
Schema evolution stalls. A column rename, an index change, or a normalization fix now needs every service that touches the shared schema to agree on a migration window, because none of them can be sure another service is not reading or writing the same table in a way the change would break. One service’s release cadence becomes every service’s release cadence.
-
Locking couples unrelated work. Two services holding row or table locks against the same physical database contend for the same connection pool, the same buffer cache, and the same lock manager, even when the business operations they are running have nothing to do with each other — exactly the lock-contention argument Aggregates makes against oversized aggregates, one level up at the database itself.
-
The database becomes a single point of failure and a single scaling bottleneck. A shared database of record cannot fail over, be resized, or be migrated to a different storage technology for one service without every other service on it going down or being blocked, which defeats independent deployability — one of the two reasons the whole split into services was undertaken in the first place.
Database-per-service is not merely a modeling nicety, then; it is the strategic decision that makes everything that follows in this page unavoidable rather than optional. Once each service genuinely owns its own store, an operation spanning two services can no longer commit as one physical database transaction, because there is no longer one physical database underneath both of them to commit against.
What local ACID gives up at the boundary
Inside one service, on top of one database, Spring Data JPA’s @Transactional boundary gives a caller the full
strength of ACID: atomicity (every write commits or none do), consistency (the database’s own constraints
hold before and after), isolation (concurrent transactions cannot observe each other’s half-finished state), and
durability (a commit survives a crash). PlaceOrderService.placeOrder
(Structuring the Code) relies on exactly this: the
Order row and its outbox row either both land or neither does, because both writes happen inside the same
database’s transaction log, coordinated by the same local resource manager.
The traditional answer to "how do I get that same guarantee across two databases, or two services" is the X/Open XA standard and its two-phase commit (2PC) protocol, and it is worth stating precisely rather than hand-wavingly, because every problem in the next section is a problem with this exact mechanism:
-
A coordinator (a transaction manager) opens a distributed transaction and enlists every resource manager involved — here, each service’s own database connection would need to be enlisted as an XA resource.
-
Prepare phase. The coordinator asks every resource manager to prepare: each one does everything short of making the write durable and visible — it writes to its own transaction log, acquires whatever locks the write needs, and replies "yes, I can commit this" (or "no"). A resource manager that votes yes has made a durable promise: it must be able to commit later even if it crashes and restarts in between.
-
Commit phase. Only once every resource manager has voted yes does the coordinator tell them all to commit; if any one voted no, it tells them all to roll back instead. Every resource manager keeps its locks held for the entire window between voting yes and receiving the coordinator’s final decision — it cannot unilaterally decide either way once it has voted, because the coordinator’s decision is final and might still be "commit."
That is a real, well-specified protocol, and it does deliver atomicity across multiple resource managers when everything involved supports it and stays reachable. The next section is about what happens once "everything involved" means REST calls between independently deployed services running on infrastructure that was never built to be an XA resource manager.
Four concrete problems in a microservice architecture
(a) Most of the infrastructure does not support it
XA needs every participant to implement the XAResource interface (or an equivalent two-phase protocol) and to
durably log prepared-but-not-yet-committed state until the coordinator’s final word arrives. Most NoSQL data
stores were never built with that contract in mind — document stores, wide-column stores, and key-value stores
overwhelmingly expose only single-document or single-key atomicity, not a prepare/commit handshake a coordinator
can drive. Modern brokers are no different: neither Apache Kafka nor RabbitMQ implements XA in the way
Spring for Apache Kafka 4.1.x or Spring AMQP expose to application code — Kafka’s own transactions coordinate
producer writes and consumer offsets within Kafka itself, not with an external XA coordinator, and RabbitMQ’s
publisher confirms are a delivery acknowledgment, not a two-phase vote. A REST call to another service is worse
still: HTTP has no notion of "prepare but do not commit yet," so there is no way to enlist a plain REST endpoint
as an XA resource at all. Two-phase commit is only even a candidate where every participant is a classic
XA-capable relational database reachable over a stable connection — which describes the inside of one service,
not the boundary between several.
(b) Availability is the product of the participants' availability
Even setting protocol support aside, a distributed transaction across N services is only as available as all N services being reachable and healthy at the same moment — and probabilities of independent failures multiply, they do not average. Take two services each independently available 99.5% of the time (roughly 43.8 hours of downtime a year each, a perfectly ordinary SLA):
P(both available) = 0.995 x 0.995
= 0.990025
~ 99.0%Two 99.5% services combine into a 99.0% operation — the combined downtime roughly doubles, from ~44 hours/year
each to ~88 hours/year for any operation that needs both up simultaneously. The Place Order Saga’s four
participants (Order, Inventory, Payment, Shipping) make the effect starker still: at the same 99.5% each,
0.995^4 ~ 0.9801, or about 98.0% — a combined operation nearly four times as likely to fail on availability
alone as any single participant. A two-phase-commit transaction spanning all four would need every one of them
reachable for the entire prepare-through-commit window, not just at one instant, which only makes the effective
availability worse than this static multiplication already suggests.
(c) Locks held across the network for the whole prepare/commit window
Two-phase commit’s prepare phase is a promise, not a suggestion: once a resource manager votes yes, it must hold its locks — on rows, on connections, on whatever it touched — until the coordinator’s commit or rollback arrives, because it cannot know which one is coming. Inside one service that window is milliseconds. Across a network, that window now includes however long the slowest remaining participant takes to prepare, plus however long the coordinator itself takes to collect every vote and relay the final decision back out — and every other prepared participant sits there holding its locks meanwhile, unable to serve any other transaction that needs the same rows. A slow or temporarily overloaded Payment service does not just slow down its own work under this protocol; it holds Inventory’s and Order’s locks hostage for as long as it takes to respond, turning one participant’s local slowness into every participant’s throughput problem.
(d) CAP, and a cloud built to restart, relocate, and retry
The CAP theorem states that a distributed system cannot simultaneously guarantee consistency (every read sees the latest write), availability (every request gets a response), and partition tolerance (the system keeps working when a network partition separates nodes) — a network partition is always possible, so in practice a system must choose, during a partition, between staying consistent (refusing requests it cannot guarantee are current) and staying available (answering anyway, possibly with stale or divergent data). Two-phase commit chooses consistency: a prepared participant that cannot reach the coordinator must simply wait, potentially indefinitely, rather than guess. Brown, Woolf, and Yoder’s Cloud Application Architecture Patterns frames the consequence for exactly the environment this section’s Java 21 / Spring Boot 4.1.x / Kafka 4.1.x baseline actually runs on — a cloud platform, not a fixed rack of always-on servers — and its Service Orchestrator pattern’s motivating observation is blunt: cloud infrastructure "is rather inhospitable for performing transactions." Instances restart during routine deployments, get relocated to different nodes by the orchestrator, and calls to them get retried automatically at several layers (load balancer, service mesh, client library) — every one of those completely ordinary cloud behaviors is, from a 2PC coordinator’s point of view, an indistinguishable-from-failure partition, exactly the scenario CAP says forces a choice, and the coordinator’s choice is to block.
The "not even Starbucks uses two-phase commit" argument
Gregor Hohpe’s widely cited essay makes the case with a real-world analogy rather than a formalism: watch a Starbucks barista, and the cashier who takes payment and the barista who makes the drink are not coordinated by anything resembling two-phase commit. The cashier takes the order and the money, writes a name on the cup, and the transaction is done — from the customer’s point of view, paid-for. The drink itself gets made whenever the queue allows, asynchronously, sometimes for several customers' orders out of order relative to payment. If something goes wrong making the drink, nobody unwinds the payment and re-runs the whole interaction from scratch; the barista just fixes the drink and hands it over, a correction applied locally rather than the entire prior exchange rolled back. Hohpe’s summary line for this pattern is that the system optimizes for "decoupling and throughput," not for forcing every step of a multi-party process to succeed or fail as one atomic unit — and a coffee shop that actually tried to hold the register locked until the espresso machine confirmed completion would serve a small fraction of the customers it serves today. The essay’s point transfers directly: a business process that spans several independently-paced participants scales by accepting that each step commits on its own and the whole sequence is reconciled afterward, not by trying to make the whole sequence atomic. See References below for the essay itself, which is worth reading in full rather than summarized further here.
The consequence: one transaction, one aggregate, one service
Put the four problems above together and the traditional distributed-transaction escape hatch is not merely harder to use across microservices, it is not a real option: unsupported by the infrastructure most services actually run on, structurally less available than any one participant, prone to holding cross-network locks for unbounded windows, and fighting the exact operational behavior (restarts, relocation, retries) a cloud platform is built to do routinely. Aggregates' rule three — one transaction creates or updates exactly one aggregate — is this page’s conclusion stated as a design rule rather than an availability argument: if a distributed transaction cannot be relied on, the only transaction boundary left to rely on is the one local database transaction inside one service, wrapping exactly one aggregate. Everything that needs to happen between aggregates or services after that one local commit — notifying Inventory an order was placed, notifying Shipping an order was approved — has to happen through Domain Events instead, published from inside that same local transaction and consumed asynchronously, which is exactly how eventual consistency replaces the atomic consistency a cross-service transaction can no longer provide.
That still leaves the practical question of how a multi-step business operation — like placing an order, which genuinely does need Inventory, Payment, and Shipping to each do something — gets coordinated without a distributed transaction tying them together. Brown, Woolf, and Yoder’s Service Orchestrator pattern names three strategies, in roughly ascending order of how much cross-service coordination they actually require:
-
Read-only or idempotent orchestration. If every step an orchestrator drives is either read-only (querying, never mutating) or safely repeatable, there is nothing to roll back if a later step fails — retry, or discard and start over, is always safe. This covers far fewer real operations than it might seem, but it is the cheapest strategy whenever it applies, because it needs no compensation logic at all.
-
Consolidating into one service. If two operations that seem to need cross-service atomicity actually always change together and belong to the same consistency boundary, the fix is not a more clever distributed transaction — it is recognizing the boundary was drawn wrong and merging the two into one service (or one aggregate) so the operation becomes, once again, one local transaction.
-
A coordinated business process with compensations — a saga. When the steps genuinely belong to different services and genuinely mutate state, each step commits its own local transaction and, if a later step fails, an explicit compensating action undoes the effect of an earlier step instead of relying on a rollback the infrastructure cannot provide. Sagas covers this strategy in full — it is the one the running Place Order Saga (reserve inventory, authorize payment, schedule shipping, approve order) actually uses, precisely because placing an order is not read-only and Order/Inventory/Payment/Shipping are not one service.
One more caveat belongs here for completeness: none of this means XA or two-phase commit is never usable again — it can still make sense entirely within one runtime, coordinating, say, a database and a local message queue that both genuinely support it and never cross a service boundary. Some platforms also offer a lighter-weight compensation-based alternative that is explicitly not 2PC: Quarkus’s Sagas with Narayana LRA subsection covers Long Running Actions (LRA) as exactly this kind of saga-shaped alternative to XA, available to services built on that runtime.
Decision table: when you think you need a distributed transaction
The table below is meant to be consulted at the moment "we need a transaction across two services" first comes up in a design discussion — the honest answer is almost always one of the four rows below, not a literal cross-service ACID transaction.
| What the operation actually needs | Why | What actually applies |
|---|---|---|
Two writes that always change together, inside code you already own as one service |
There was never a real service boundary between them — the boundary was drawn one aggregate too fine, or one service too many |
Single aggregate/service: merge the writes into one aggregate, one local |
Data from another service, but nothing about the operation mutates that other service’s state |
No participant needs to roll back, because none of them wrote anything — a stale read is the only risk, and it is bounded by how fresh the query needs to be |
Read-only/idempotent composition: a synchronous REST/RPC query, safely retried on failure |
A rule or invariant that keeps needing "and also check the other service," over and over, across many operations |
The two services are coupled tightly enough, and this often enough, that the coupling is the real design and splitting it was premature |
Consolidation: merge the two services (or the two aggregates) into one bounded context |
A multi-step business process that must mutate state in several independently-owned services, and must reach a consistent end state even if a step fails partway through |
Each service can only guarantee its own local transaction; a failure partway through has to be actively undone, not merely rolled back by infrastructure that cannot see across services |
Saga: each step’s local transaction plus an explicit compensating action for failures after the fact |
Code: checkout written naively, then restructured
The naive version below is exactly the mistake this page argues against: one @Transactional method that tries
to make a REST call to Inventory, a REST call to Payment, and a local database write all succeed or fail
together, as if @Transactional could somehow reach across the two HTTP calls the way it reaches across two JPA
repository calls inside one service:
@Service
public class NaiveCheckoutService {
private final OrderRepository orderRepository;
private final InventoryClient inventoryClient; // REST client to the Inventory service
private final PaymentClient paymentClient; // REST client to the Payment service
public NaiveCheckoutService(OrderRepository orderRepository, InventoryClient inventoryClient,
PaymentClient paymentClient) {
this.orderRepository = orderRepository;
this.inventoryClient = inventoryClient;
this.paymentClient = paymentClient;
}
// @Transactional here is a lie: Spring's transaction manager only knows about the
// local JPA resource. It has no idea an XA-style prepare/commit would need to also
// cover the two REST calls below, and plain HTTP has no such phase to join anyway.
@Transactional
public OrderCreation checkout(PlaceOrderCommand command) {
OrderCreation creation = Order.place(command);
// (1) A REST call is not an XA resource -- there is no "prepare" step Inventory
// can join, so this either fully happens or it doesn't; there is no in-between
// Spring's transaction manager could roll back if a later step fails.
inventoryClient.reserve(creation.order().id(), creation.order().lines());
// (2) Same problem again for Payment, compounded: if this call fails, the
// reservation made in step (1) is already committed inside Inventory's own
// database and cannot be rolled back by this method throwing an exception --
// @Transactional only ever rolls back the local orderRepository write below.
paymentClient.authorize(creation.order().id(), command.customerId(), creation.order().totalAmount());
// (3) If the process crashes here -- after both remote calls succeeded but
// before this local save -- Inventory and Payment both believe the order is
// real, while Order's own database has no row for it at all: a partial failure
// that leaves three services in mutually inconsistent states, with no
// coordinator, no compensations, and nothing that will ever reconcile it.
orderRepository.save(creation.order());
return creation;
}
}Every numbered comment is a distinct way this method can leave the system inconsistent, and none of them is a
coding mistake in the ordinary sense — @Transactional is doing exactly what it promises, atomicity over the
one JPA resource it manages; the bug is asking it to cover two REST calls it was never able to reach in the first
place. The restructured version applies the consequence from above directly: PlaceOrderService
(Structuring the Code) keeps its @Transactional
boundary around exactly one aggregate and one local database, and everything Inventory and Payment need to know
travels as a published domain event instead of an inline REST call inside the transaction:
@Service
public class PlaceOrderService {
private final OrderRepository orderRepository;
private final DomainEventPublisher domainEventPublisher;
public PlaceOrderService(OrderRepository orderRepository, DomainEventPublisher domainEventPublisher) {
this.orderRepository = orderRepository;
this.domainEventPublisher = domainEventPublisher;
}
// The @Transactional boundary now covers only what one database can actually
// guarantee atomically: the Order row and the outbox row it writes alongside it.
// Inventory and Payment are never called from inside this method at all.
@Transactional
public OrderCreation placeOrder(PlaceOrderCommand command) {
OrderCreation creation = Order.place(command);
Order order = creation.order();
long version = orderRepository.save(order);
// writes one outbox row per event, same local transaction
domainEventPublisher.publish("Order", order.id().value().toString(), version, creation.events());
return creation;
}
}This is the same PlaceOrderService Structuring the
Code already defines, unchanged — the point of restructuring checkout this way is precisely that nothing
about placing the order itself needed to grow more complex to stop depending on a distributed transaction; the
complexity that the naive version tried to force into one method instead moves into
the transactional outbox (how domainEventPublisher
gets the OrderPlaced event from that outbox row onto a broker reliably) and into
Sagas (how a PlaceOrderSaga reacts to OrderPlaced and drives
Inventory’s reservation and Payment’s authorization as their own local transactions, with compensation if either
one fails). Neither mechanism is repeated here.
Mermaid: two-phase commit, and the coordinator-crash window
The sequence below plays out the 2PC protocol described above literally, then highlights the specific window that makes it dangerous in practice: once every resource manager has voted yes and is holding its locks, a coordinator crash leaves every one of them "in doubt" — prepared, locked, and unable to safely decide anything on their own until the coordinator recovers.
Nothing in that diagram is a modeling exaggeration: a resource manager that has voted yes is contractually required to be ready to commit whenever the coordinator eventually says so, however long that takes, which is exactly why problem (c) above calls the prepare-to-commit window a lock-holding liability rather than a brief technicality.
