Cache & Hedge

resilience4j-cache wraps a JCache (JSR-107, package javax.cache) Cache in a Resilience4j Cache<K, V> that does one thing: a read-through lookup. Given a key and a function that produces the value, it returns the cached value if there is one, and otherwise calls the function, stores the result and returns it. It adds three things to the JCache API: hit, miss and error events, a hit/miss counter, and the decorator shape that composes with the other modules.

It is not a cache. It stores nothing itself, has no size limit, no expiry, no eviction and no serialisation of its own: all of that comes from the JCache provider you put on the classpath. The module depends only on resilience4j-core and the JCache API (javax.cache:cache-api; the 2.4.0 build uses 1.1.0 and Spring Boot 4.1.1 resolved 1.1.1 in the sample project). The JCache API (1.1.x) still lives in the javax.cache package, not jakarta.cache, and that is the package every provider in this page uses.

The flow in the diagram is the whole module, CacheImpl in the source. The sections below walk through its consequences.

resilience4j-cache is in the resilience4j-bom and is a dependency of resilience4j-all (the module that provides Decorators), so with resilience4j-all you already have it. The resilience4j-spring-boot4 starter does not bring it. A JCache provider is separate and is your choice; the example below uses the Caffeine one.

Source: Resilience4j: Cache and the resilience4j-cache module at v2.4.0. The Caffeine artifact is documented in the Caffeine JCache wiki page.

With exactly one JCache provider on the classpath, Caching.getCachingProvider() finds it through the ServiceLoader (the test printed CaffeineCachingProvider). With more than one, name the provider explicitly or the call fails.

Cache.of(javax.cache.Cache) wraps an existing JCache instance. The static decorateSupplier, decorateCallable and decorateCheckedSupplier helpers turn a supplier into a function of the cache key, which is the one place the Cache differs from the other modules, whose decorators keep the shape of the decorated function.

Source: Resilience4j: Cache, adapted. The readme.io page shows Vavr’s CheckedFunction1 and Try; since Resilience4j 2.0.0 Vavr is gone from the core and the checked function type is io.github.resilience4j.core.functions.CheckedFunction. Run: the events were [MISS 978-0, HIT 978-0, MISS 978-1], and getName() returned the JCache name isbn-titles.

Decorators (from resilience4j-all) adds withCache(cache) to the Supplier and CheckedSupplier builders. It returns a DecorateFunction (a DecorateCheckedFunction for the checked builder), because it turns the chain into a function of the key, so it is the last decoration of a chain:

Source: Resilience4j: Cache and Decorators at v2.4.0. Run: the breaker recorded exactly one successful call after two lookups.

withCache is defined only for Supplier and CheckedSupplier decorations, not for CompletionStage, Callable, Runnable or reactive types, and the Kotlin and Reactor modules have no cache operator. For a reactive pipeline, wrap the blocking lookup yourself or use a reactive cache library.

Decorators applies calls in the order written, the last one outermost, and withCache produces a function, so it is always the outermost link. That is also the right place: a hit then costs a map lookup and does not consume a retry attempt, a breaker permit, a rate-limiter permit or a bulkhead slot, and does not pollute their metrics with calls that never reached the backend. Within the miss path the order is the usual one from Fallbacks & Composition: Retry outside CircuitBreaker.

The Bookshelf IsbnTitles service puts all of it together, in a Spring Boot application:

Source: Resilience4j: Cache and the Spring Boot caching reference (for the auto-configured javax.cache.CacheManager). Run: three calls for the same ISBN produced one request to the stubbed API, two cache hits, one miss and one successful call on the breaker. The Spring Boot javax.cache.CacheManager bean used above comes from the JCache auto-configuration, which needs spring-boot-starter-cache, @EnableCaching and a provider on the classpath; without them, create the manager yourself with Caching.getCachingProvider().getCacheManager().

The Cache publishes three events through getEventPublisher(), and keeps two counters. Event consumers run on the calling thread, and the event objects are created only when at least one consumer is registered.

getMetrics() returns getNumberOfCacheHits() and getNumberOfCacheMisses(), two LongAdder counters that only grow. There is no Micrometer binder, no Actuator endpoint and no health indicator for the Cache module in 2.4.0, so to chart a hit ratio, register the two counters as gauges yourself, or use the provider’s own statistics (JCache has a statisticsEnabled configuration flag and an MXBean). A cache that fails on every call shows up as growing error events and, importantly, as no growth in either counter.

The module’s central promise, stated on the readme.io page, is that a failing cache does not fail the call: when a lookup from a distributed cache fails, the exception is absorbed and the function is called. The source and a run with a JCache wrapper that throws show exactly what that means for each way a cache can fail.

Run: with a Bookshelf backend that answered 500 to every request and Retry configured for two attempts, one IsbnTitles.title call produced four requests to the backend (two attempts, twice) before the caller saw an exception. That doubling comes from the Cache module re-running the supplier, not from Retry.

Two practical conclusions follow. A cache lookup that fails quickly is harmless; one that fails slowly (a Redis connection that hangs for its socket timeout) adds that delay to every call, so give the provider’s client a short timeout, or put a CircuitBreaker around the cache itself, as sketched in Near and Far Caches. And because a failing supplier may run twice, the decorated function must be safe to call twice, which is the same idempotency rule Retry has.

The readme.io page says that the JCache reference implementation (RI) should not be used in production, because it has concurrency problems, and recommends Ehcache, Caffeine, Redisson, Hazelcast, Ignite or another provider. The RI is the org.jsr107.ri:cache-ri-impl artifact; keep it out of production code, and use a real provider even in integration tests when you want them to resemble production.

Choosing a provider is the same decision as choosing a near or far cache in Near and Far Caches; the distributed ones add a network call that can fail or hang, which is exactly the case the failure rules above and a short client timeout are about.

The Cache module delegates all of it, so the JCache MutableConfiguration is where the real settings are:

Source: the JSR-107 specification page and its API repository. Run: the same configuration style with a 150 ms CreatedExpiryPolicy expired an entry, and the three variants of the price record behaved as described above.

There is no resilience4j.cache prefix. The 2.4.0 source of resilience4j-spring6, resilience4j-framework-common and resilience4j-spring-boot4 has no cache configuration class, no @Cache annotation, no auto-configured Cache bean and no Actuator endpoint for it. The only Spring Boot properties that touch a JCache are Spring’s own: with spring-boot-starter-cache, @EnableCaching and spring.cache.type=jcache, Boot builds a javax.cache.CacheManager and a JCacheCacheManager, and spring.cache.jcache.provider and spring.cache.jcache.config select the provider and its configuration resource. Those properties configure Spring’s cache abstraction, not Resilience4j; the isbnTitleCache bean above simply reuses the same manager, and the printed cache names were [book-prices, isbn-titles], one created by Boot from spring.cache.cache-names and one by the bean.

The two solve the same problem at different levels, and can coexist on the same JCache provider and manager.

Rule of thumb: use @Cacheable for ordinary application caching and Spring’s rich cache features, described in Caching: Spring Cache, Caffeine and Redis; use the Resilience4j Cache when you specifically need the cache failure to be absorbed and observed in the same decorator chain as a breaker and a retry, which is typical for a call to a remote, quota-limited API. Run: a @Cacheable bean on the cache book-prices loaded once for two calls, on the same Boot context in which the Resilience4j-decorated ISBN lookup ran.

CacheRegistryStore<E> implements the RegistryStore extension point (see Registries, Configuration & Decorators) on top of a JCache Cache<String, E>, with computeIfAbsent, find, remove, replace and values mapped to the JCache operations. It lets a registry keep its entries in a cache instead of a ConcurrentHashMap. The caveats are significant: registry entries (a CircuitBreaker, say) are live objects with state and threads, they are not Serializable, so the JCache must be configured with setStoreByValue(false) (with the default it fails with UncheckedIOException: Failed to serialize class …​CircuitBreakerStateMachine, as the run showed), and a distributed JCache cannot share the objects between JVMs anyway. In practice this is only useful for a local cache with custom eviction.

Source: CacheRegistryStore at v2.4.0.

Tail latency is the slowest few percent of calls. A service that answers in 20 ms nearly always, but in 800 ms one call in a hundred, makes a page that calls it ten times slow for one user in ten. Retry and TimeLimiter react to failure or to a timeout, both of which mean waiting for the long tail. A hedged request attacks it differently: if the first call has not answered after a short delay, send a second, identical call (to another replica, if the load balancer rotates), and use whichever answers first. The slow call is usually the one that hit a busy instance, and the second one rarely does.

The price is extra load: every hedge is a real request to the backend, so a hedge delay that is too short multiplies traffic. And because the same request is sent twice, hedging suits only idempotent calls: reads, and writes that carry an idempotency key. A hedged POST /orders without an idempotency key can create two orders. See Resilience Fundamentals for idempotency as the precondition for any repetition of a call.

resilience4j-hedge (copyright headers of 2021) implements this. There is no readme.io page for it; this section is documented from the v2.4.0 source and its tests, and every claim was run.

resilience4j-hedge is published to Maven Central as 2.4.0 like the other modules, but it is not in resilience4j-bom and not a dependency of resilience4j-all. Declare it with an explicit version, like resilience4j-spring-boot4:

Source: resilience4j-hedge at v2.4.0. Run: the sample project resolved it with the explicit version, and the module depends only on resilience4j-core. The module is also absent from every other module of the build: Decorators has no withHedge, and there is no Spring, Micrometer, Reactor or Kotlin integration for it. The only mentions of hedge in the 2.4.0 source tree outside the module are in settings.gradle.

Hedge follows the pattern of the other modules: Hedge.ofDefaults(), Hedge.ofDefaults(name), Hedge.of(config), Hedge.of(name, config), Hedge.of(name, config, tags) and the shortcut Hedge.of(Duration), which creates a hedge with a fixed delay. HedgeRegistry (HedgeRegistry.builder()…​build()) is an in-memory registry with hedge(name), hedge(name, config), hedge(name, configName) and the other usual overloads, shared configurations through addConfiguration, and the same semantics as the other registries: asking twice for the same name returns the same instance. The name defaults to UNDEFINED.

The execution methods are two:

There is no blocking Supplier decoration: hedging needs asynchronous results, so a synchronous call is wrapped in a Callable for submit.

The Hedge.of(Duration) shortcut and HedgeConfig.custom().preconfiguredDuration(…​) are the same thing. Run: the default HedgeConfig.ofDefaults() printed PRECONFIGURED, a null cutoff, factor 0, window 100, measure-errors true and concurrency 10.

There are no Spring Boot properties for hedging: no resilience4j.hedge prefix, no HedgeConfigCustomizer, no auto-configured HedgeRegistry. Create the registry as a bean yourself, or the hedge directly.

A delay close to the average hedges about half of the calls, because by definition half are slower than the average. That is rarely what you want; prefer a fixed delay near a high percentile, or an AVERAGE_PLUS amount that is a multiple of the typical latency.

HedgeImpl.decorateCaller is short, and its behaviour is worth stating precisely, because it differs from what the name suggests:

The hedge records the outcome of whichever call wins, in Hedge.Metrics and through Hedge.EventPublisher.

Consumers are registered with onPrimarySuccess, onPrimaryFailure, onSecondarySuccess and onSecondaryFailure. getMetrics() exposes getCurrentHedgeDelay(), getPrimarySuccessCount(), getSecondarySuccessCount(), getPrimaryFailureCount(), getSecondaryFailureCount() and getSecondaryPoolActiveCount(). The secondary success ratio is the number to watch: a ratio near zero says the delay is too long to matter, a ratio near one says the delay is below the normal latency and every call is being doubled. As with Cache, nothing is bridged to Micrometer or Actuator.

The catalogue page reads the price of a book from the pricing-service, an idempotent GET. The test stubs a replica that is slow (600 ms) for the first request and fast (20 ms) afterwards, and hedges after 100 ms. The circuit breaker is placed inside the hedge, so it sees every attempt, the hedged one included.

Source: derived from the v2.4.0 resilience4j-hedge source and tests, there being no official documentation page. Run: after a warm-up call, price("978-0") returned in under 400 ms instead of the 600 ms the slow replica took, with two requests to the stub, secondarySuccess 1 and primarySuccess 0; 800 ms later the breaker had recorded two successful calls, the hedged one and the slow primary that finished in the background.

Since Decorators has no hedge, composition is manual, and the position decides what each module measures.

The detailed rules for ordering are in Fallbacks & Composition.