Throttling is the process of enforcing these limits to protect applications from excessive traffic, accidental misuse, or malicious attacks.
Why Do We Need Rate Limiting?
Consider a Spring Boot microservices application exposing a product search API.GET /api/productsEvery request consumes CPU, memory, database connections, and network bandwidth.
Without protection, excessive traffic can overwhelm the application, leading to increased latency, thread exhaustion, overloaded downstream services, and eventually service unavailability.
A rate limiter restricts how many requests a client can send within a given time period.
By preventing individual clients from consuming excessive resources, it protects the application and its backend services while ensuring fair resource usage for all users.
Rate Limiting vs Throttling
Rate limiting defines the maximum number of requests allowed during a specific time window.
Throttling is the action taken when the configured limit is exceeded. The application may reject requests immediately, delay them, or queue them for later processing.
Where Is Rate Limiting Applied?
Rate limiting can be implemented at multiple layers depending on the system architecture.In many microservices architectures, it is implemented at the API Gateway, allowing requests to be filtered before they reach backend services.
It can also be enforced by load balancers, reverse proxies, Spring Boot applications, or even individual microservices when different services require different limits.
Implementing rate limiting at the gateway is generally preferred because unnecessary requests are blocked before consuming backend resources.
Popular API gateways such as Spring Cloud Gateway, Kong, NGINX, AWS API Gateway, and Azure API Management provide built-in support for rate limiting.
Implementation
The following sections describe the four most commonly used algorithms.Fixed Window Algorithm
The Fixed Window algorithm counts requests within a fixed time interval, such as one minute.Suppose an API allows 100 requests per minute. As requests arrive, a counter is incremented. Once the counter reaches 100, every additional request during that minute is rejected.
When the next minute begins, the counter is reset to zero and requests are accepted again.
Minute 10:00:48
-------------------------
Request Count: 99
Next Request -> Accepted
Minute 10:00:50
-------------------------
Request Count: 100
Next Request -> Rejected
.
.
.
Minute 10:01:00
-------------------------
Request Count: 0
Next Request -> Accepted
This algorithm is simple, fast, and memory-efficient, but it can create traffic spikes at window boundaries because clients may send the maximum number of requests at the end of one window and again at the beginning of the next.
Sliding Window Algorithm
The Sliding Window algorithm continuously evaluates requests over the previous time interval instead of using fixed boundaries.If the limit is 100 requests per minute, every incoming request checks how many requests have been received during the previous sixty seconds.
Current Time
|
v
<----------------------------->
Previous 60 Seconds
Count Requests
Unlike the Fixed Window algorithm, the limit moves continuously with time, preventing sudden bursts at minute boundaries.
Sliding Window provides more accurate and fair rate limiting but requires additional memory and computation because timestamps or request counters must be maintained.
Token Bucket Algorithm
The Token Bucket algorithm is one of the most widely used rate limiting algorithms because it allows controlled bursts of traffic.A bucket contains a fixed number of tokens. Every request consumes one token, while new tokens are added at a constant rate until the bucket becomes full.
Each client typically has its own token bucket, allowing limits to be enforced independently for different users or API keys.
+----------------+
| Token Bucket |
| ************** |
+----------------+
|
One Token
|
v
API Request
If the application has been idle for some time, the bucket becomes full with 100 tokens. A client can immediately send a burst of up to 100 requests because sufficient tokens are available.
After the bucket becomes empty, new requests are accepted only as fresh tokens are generated. Since five new tokens are added every second, approximately five additional requests can be processed each second.
This algorithm supports occasional traffic bursts while maintaining a controlled average request rate, making it one of the most popular choices for API gateways and cloud services.
Leaky Bucket Algorithm
The Leaky Bucket algorithm processes requests at a constant rate regardless of how quickly they arrive.Incoming requests are placed into a queue, and the queue drains at a fixed processing speed.
Incoming Requests
|
v
+----------------+
| Queue |
+----------------+
|
Fixed Processing
Rate
|
v
Application
The excess requests are queued while the application continues processing requests at a steady rate.
If the queue becomes full because requests continue arriving faster than they are processed, additional requests are rejected.
Unlike the Token Bucket algorithm, which permits short bursts, the Leaky Bucket algorithm smooths incoming traffic into a constant flow, making it useful when downstream systems require a predictable and stable request rate.
HTTP 429 - Too Many Requests
When a client exceeds the configured limit, the server typically returns the following response.The Retry-After header tells the client when another request may be attempted.HTTP/1.1 429 Too Many Requests Retry-After: 60
Many APIs also return headers such as X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset so clients can monitor their current usage.
Rate Limiting in Spring Boot
A common way to implement rate limiting in Spring Boot is by using Bucket4j, an open-source Java library that implements the Token Bucket algorithm.Typically, a bucket is associated with each client, identified by attributes such as an IP address, API key, or authenticated user.
Common rate-limiting keys include client IP address, authenticated user ID, API key, OAuth client ID, or a combination of these depending on the application's requirements.Every incoming request attempts to consume a token from that client's bucket before the request reaches the business logic.
If a token is available, the request is processed normally. Otherwise, the request is rejected immediately with an HTTP 429 (Too Many Requests) response.
The following example creates a token bucket that allows a client to make up to 100 requests per minute.
Bucket bucket = Bucket.builder()
.addLimit(Bandwidth.simple(
100,
Duration.ofMinutes(1)))
.build();
if (bucket.tryConsume(1)) {
// Process request
} else {
// Return HTTP 429
}
Once all tokens have been consumed, subsequent requests are rejected with an HTTP 429 response until new tokens are replenished according to the configured refill rate.
Distributed Rate Limiting
In a microservices architecture, multiple Spring Boot instances often run behind a load balancer.If each application instance maintains its own in-memory request counter, rate limits are enforced independently on each server.
As a result, a client may exceed the intended limit simply because consecutive requests are routed to different instances.
To enforce a global limit across all instances, the token bucket or request counters are stored in a shared distributed store such as Redis, ensuring that every application instance enforces the same limit.
Load Balancer
|
+---------+---------+
| |
v v
Spring Boot 1 Spring Boot 2
| |
+---------+---------+
|
v
Redis
Libraries such as Bucket4j provide Redis-backed implementations, allowing token buckets to be shared across multiple Spring Boot instances with minimal configuration.
RedisClient redisClient =
RedisClient.create("redis://localhost:6379");
StatefulRedisConnection connection =
redisClient.connect(new ByteArrayCodec());
ProxyManager proxyManager =
LettuceBasedProxyManager.builderFor(connection).build();
BucketConfiguration configuration =
BucketConfiguration.builder()
.addLimit(Bandwidth.simple(
100,
Duration.ofMinutes(1)))
.build();
Bucket bucket = proxyManager.builder()
.build(clientId, configuration);
if (bucket.tryConsume(1)) {
// Process request
} else {
// Return HTTP 429
}
ProxyManager connects Bucket4j to Redis. When build(clientId, configuration) is called, Bucket4j looks up the bucket associated with clientId in Redis.
If it does not exist, it is created and stored there.
Subsequent requests from any Spring Boot instance retrieve and update the same bucket, ensuring that rate limits are enforced consistently across the entire cluster.
Advantages
1. Rate limiting protects applications from excessive traffic, improves system availability and overall service reliability, and ensures fair resource usage among clients.2. It also reduces the impact of denial-of-service attacks, prevents accidental misuse by clients, and helps control infrastructure costs by limiting unnecessary requests.
Limitations
1. Selecting appropriate limits requires understanding normal traffic patterns. Limits that are too restrictive may block legitimate users, while limits that are too generous may provide little protection.2. Distributed rate limiting introduces additional infrastructure and network overhead because application instances must coordinate through a shared store such as Redis.
Best Practices
1. Apply rate limiting as early as possible, preferably at the API Gateway, to prevent unnecessary requests from reaching backend services.2. Choose limits based on client type, API sensitivity, and expected traffic patterns rather than using a single global limit.
3. Use Redis or another distributed store when multiple application instances share the same rate limits.
4. Return meaningful HTTP 429 responses together with a Retry-After header so clients know when they can retry.
5. Monitor rate-limiting metrics and adjust limits as traffic patterns evolve.
Summary
Rate limiting controls how many requests a client can make within a specified period, while throttling is the mechanism used to enforce those limits.Algorithms such as Fixed Window, Sliding Window, Token Bucket, and Leaky Bucket offer different trade-offs between implementation simplicity, fairness, memory usage, and burst handling.
In Spring Boot microservices, rate limiting is commonly implemented using Bucket4j with Redis for distributed deployments, or at the API Gateway to protect backend services before requests reach the application.
