Bucket4j: Production Checklist

A limit of 10,000 requests per hour per user sounds strict, but a bucket does not know about the hour: a client that sends 9,999 requests in the first seconds has used no more than it was given, and has caused a burst of about a thousand requests per second. A patient attacker can repeat that every hour and never reach the limit, so no alert fires. The remedy is a second, short bandwidth in the same bucket; the number of bandwidths has no effect on performance.

With a manual clock, the hourly bucket alone admitted all 9,999 requests in the same instant; with the second bandwidth only 20 were admitted, and 20 more after the clock moved one second. Source: Bucket4j reference: Be wary of long periods.

A token bucket guarantees its rate only over a long time. A bucket of 100 tokens per minute that is drained at the start of a minute is full again one minute later, so up to 200 tokens can be consumed in about a minute. That is inherent to the algorithm and the reference says so plainly: either accept bursts, or trade capacity for strictness by reducing capacity and refill together.

Both claims were run: the capacity-100 bucket admitted 200 tokens across a refill boundary, the capacity-50 bucket admitted 100 and refused a single tryConsume(51). The price is under-consumption over long periods, which is the trade the reference describes. Source: Bucket4j reference: Be wary of short-timed bursts.

bucket.asBlocking().tryConsume(1, maxWait) parks the calling thread until a token is available. That is the right behaviour for a batch step or an outbound call that may wait, and the wrong one for the inbound limiter of an interactive API: a client would rather be told 429 with Retry-After than be kept waiting, and a waiting request holds a platform thread, a connection and anything it has already opened. Under load a limiter that blocks turns a rejection into a thread-pool exhaustion.

The blocking call returned after more than 800 ms; the probe returned in under 100 ms with a one-second Retry-After. Blocking consumption stays appropriate for outbound work, and on virtual threads it is cheap while it waits; always give it a maximum wait. Source: Bucket4j reference: Blocking API. The fix in context is on the manual integration page and the Bucket API page.

A BucketProxy without optimizations holds no state, so building one per request is fine. A proxy built with withOptimization is different: it keeps the batched requests or the locally consumed tokens that have not been synchronized yet. If it is rebuilt for every request that state is thrown away, nothing is ever amortized, and the delayed tokens are lost. The optimization then costs complexity and delivers nothing.

Counting the commands the Lettuce client sent to Redis, 50 requests with a proxy per request made 101 round trips (50 EVAL and 51 GET), and the same 50 requests through one shared proxy made 3. The shared proxy is a singleton bean; make it thread-safe by construction, which it is, because the proxy is. Remember that delaying can over-admit by up to its token budget across replicas, so it is for large quotas such as the daily ISBN limit and not for login attempts. Source: Optimizations.java and DelayParameters.java at tag 8.21.0; the reference does not describe optimizations. The fix in context is Optimizations on the distributed page.

A bucket is created the first time a key is seen and, with the default ExpirationAfterWriteStrategy.none(), it stays in the store for ever. For a handful of well-known keys that is fine; for a key per client address or per user it is a memory leak that an attacker can drive: every new source address adds a row.

Against Redis, 1,000 per-address buckets created without expiration were all still there with a time to live of -1 (no expiry). With the strategy above, a bucket of the free plan (20 per minute) that had consumed one token had a time to live of 33,000 ms: the three seconds needed to refill that token plus the 30-second grace period. In a second run with a 500 ms grace period, 50 buckets had disappeared on their own within five seconds. basedOnTimeForRefillingBucketUpToMax drops a bucket exactly when it would be full again, so forgetting it changes nothing; fixedTimeToLive does not have that property and can forget consumed tokens. In addition, bound the key space (see the next two anti-patterns) and set a memory limit and eviction policy on the store itself, as described in Redis architecture, keyspace and eviction. Source: Bucket4j reference: Expiration policy. The fix in context is Expiration on the distributed page.

The key decides who shares a bucket, so it must come from something the client cannot choose. X-Forwarded-For and similar headers are set by the client unless a trusted proxy overwrites them; a limiter that keys on them gives every request its own bucket as soon as the attacker varies the header.

Sending 100 requests from one address with a different X-Forwarded-For each, the wrong key admitted all 100 of them (the 20 per minute limit never applied); the right resolver, given the same requests with a random X-API-Key in addition, admitted 20. request.getRemoteAddr() is the real client only when Spring Boot is configured to replace it from the forwarded headers of a trusted proxy; that configuration, and the four combinations that were run, are in the client address behind proxies. Hash secrets used as keys (the fingerprint above is the first eight bytes of a SHA-256 digest) so that API keys do not appear in Redis, in KEYS output or in logs. Source: Spring Boot how-to: Use behind a proxy server.

A servlet filter registered after the Spring Security chain only sees requests that Security let through. A wrong password is rejected by Security with a 401 or a redirect and never reaches such a limiter, so a per-user quota placed there does nothing against credential guessing. The measured difference is on the ordering section of the manual integration page: 25 wrong logins were 25 rejections by Security and none by the limiter placed after it.

The login limiter has to run before the credentials are evaluated. Placing it inside the Security chain, ahead of UsernamePasswordAuthenticationFilter, keeps CSRF and the rest of the chain in front of it and is easy to test; the security section below has the complete code.

The wrong version is a quota filter registered with FilterRegistrationBean at an order above Spring Security’s default of -100: every wrong password stops in the Security chain before that filter runs.

Source: Spring Security reference: Servlet architecture. See also Spring Security in the SpringBoot reference.

A refill is computed from elapsed time. For compare-and-swap, lock-based and JDBC select-for-update managers the default clientClock is TimeMeter.SYSTEM_MILLISECONDS, which is the wall clock of the application JVM, so every replica computes refills with its own clock. Entry-processor and server-side backends use the clock of the grid member unless a clientClock was set. If the machines disagree about the time, the replicas disagree about how many tokens a bucket holds.

On one Redis bucket of 10 tokens refilled at 10 per minute, drained by the replica with the correct clock, the replica whose clock was 30 seconds ahead read 5 tokens available: it refills early by exactly its skew. For a per-minute login or API limit that is large; for a per-day quota it is negligible. The remedies are operational (NTP or chrony on every node, and an alert on clock offset), architectural (a server-side backend, or one clientClock that reads a single trusted time source) and organizational (a rolling deployment that mixes library versions should also use backwardCompatibleWith). In tests, a controllable TimeMeter makes the refill deterministic; it must implement both currentTimeNanos() and isWallClockBased(). Source: AbstractProxyManager.getClientSideTime() and AbstractProxyManagerBuilder.clientClock at tag 8.21.0, and Bucket4j reference: Customizing time measurement. The fix in context is Clock skew on the distributed page.

A local Bucket limits one JVM. With three replicas a "100 per day" bucket is three buckets of 100, and the ISBN provider that allows 100 calls per day for the whole account sees up to 300. The limit also changes silently whenever autoscaling changes the number of replicas, so the contract with the provider depends on the replica count.

Per-instance limits are legitimate when they are labelled as such: a per-replica protection against overload, or a quota split evenly with a fixed replica count. They are an anti-pattern when documented or contracted as service-wide. Source: Bucket4j reference: Distributed concepts. The fix in context is the distributed rate limiting page; the Resilience4j RateLimiter is per-instance by design.

A shared limiter puts the store on the request path. When Redis stops answering, tryConsume throws an unchecked exception; uncaught, that is a 500 for every request, including the ones that never needed the limit. A hung store is worse than a refused connection, because nothing fails until the client’s own timeout expires, and every request thread waits for it.

With the Redis container paused (a hung server), the bare call failed with an exception after about 260 ms, which is the requestTimeout, and the guarded call returned UNAVAILABLE_DENIED after the same time under fail-closed and UNCHECKED_ALLOWED under fail-open; after the container was resumed the next check succeeded. Choose the policy per limit: fail closed for security limits (login attempts, one-time codes) and for quotas whose breach costs money, fail open for limits that only protect capacity. A Resilience4j CircuitBreaker around the check turns "every request waits 250 ms" into "requests fail at once", as shown on the manual integration page. Source: Bucket4j reference: Request timeout and Bucket4j reference: Distributed checklist.

The unit of planning is the number of round trips one consumption costs, and it depends on the backend.

This differs from the one-round-trip figure that is sometimes quoted for Bucket4j, which holds for the entry-processor backends and not for compare-and-swap Redis. The rate at which the store must answer is therefore

where is the number of limit checks per second (the request rate times the number of limited buckets per request) and the round trips per check from the table. A service at 5,000 requests per second with one Redis check per request asks Redis for about 10,000 round trips per second. A request that touches two buckets, for instance a per-address and a per-user one as the login limiter does, doubles that. Rejected requests still pay their round trip, which is why hostile traffic should be rejected earlier, at the edge, when it is expected.

The store must also be reachable with enough concurrent connections. By Little’s law the number of operations in flight is

with the checks per second and the time one check takes. Over loopback to a container the Redis check took between 0.3 and 1.2 ms in repeated runs and the PostgreSQL transaction about 5 to 7 ms (its commit waits for a disk flush); these are laptop figures, so measure your own. With at 5 ms and 500 checks per second a JDBC-backed limiter keeps about 2.5 connections busy, so a pool of 10 leaves headroom, and a pool shared with the business queries needs the limiter’s share added to its size. For JDBC remember that the limiter’s transaction holds a row lock: operations on the same key are serialized, so a single hot key such as the ISBN quota is limited to about checks per second regardless of pool size. The batching optimization, or a store with a one-round-trip operation, is the answer for hot keys; see latency and throughput.

The state of a bucket is small and fixed, whatever the request rate. Redis reported 168 bytes for the key and value of an API bucket and 200 bytes for a login bucket whose key holds a username digest and an address. A PostgreSQL row of the bucket table held 108 bytes of state in a 152-byte row, before index and page overhead. The number of keys alive at one moment is what matters, and with expiration it is the arrival rate of new keys times how long each one is kept:

where is the number of previously unseen keys per second, the time the bucket needs to become full, the grace period given to basedOnTimeForRefillingBucketUpToMax, and the interval of the cleanup job, which is zero for Redis (expiry is automatic) and positive for JDBC. For the per-address login bucket, 50 new addresses per second, a 15-minute refill and a 30-second grace period keep about keys alive, or roughly 9 MB at 200 bytes each. Without expiration the same traffic adds 50 keys per second for ever: 4.3 million keys and 860 MB after a day.

The Bookshelf login limiter keeps two buckets per attempt. One is keyed by the username and the address and allows five attempts per 15 minutes; the other is keyed by the address alone and allows 30, which catches one source trying many usernames (password spraying). The username goes into the key as a digest, lower-cased and trimmed, so that Alice and alice with a trailing space share a bucket, the key length is bounded, and no user-controlled separator can forge another key.

The filter is registered inside the security chain, as in the right-hand example of the fifth anti-pattern, and is deliberately not a @Bean: Boot registers every Filter bean as a servlet filter as well, which would run it twice. Source: Spring Security reference: Servlet architecture and Bucket4j reference: BucketProxy.

Through a real MockMvc request chain with form login, five wrong passwords for alice from one address each got the usual redirect to the error page, the sixth got 429 with a Retry-After header, and so did a seventh with the correct password: once the budget is spent, the credentials are not even examined. A different address had its own allowance. Forty attempts with forty different usernames from one address were refused 10 times, the 30-per-address limit. With two limiter instances on separate managers over one Redis, three attempts through the first and two through the second exhausted the five, and the sixth was refused by both, so the limit holds across replicas. The behaviour when the store is unavailable is the fail-closed branch above.

Several design points follow from the code and are worth stating rather than discovering.

An API key identifies a client for quota purposes, and a limiter built on it has three rules.

The generic and distributed checklists above still apply to every key: expire the buckets, bound the key space, decide the outage policy.