Service Discovery & API Gateway
Rate Limiting and Circuit Breakers at the Gateway
Configuring Redis rate limiters and Hystrix/Resilience4j boundaries at entryways.
Interview: Edge resilience architectures. Expect questions on Token Bucket algorithms, configuring Redis rate limiters, fallback routing at the gateway, and circuit breakers setups.
Introduction
The API Gateway is the entry point of your system, making it vulnerable to surges in traffic. If a client targets your endpoints with massive request loops, or if downstream services slow down, the gateway must protect your resources.
To handle these scenarios, Spring Cloud Gateway supports edge resilience patterns: **Redis-Based Rate Limiting** to throttle client requests, and **Resilience4j Circuit Breakers** to fail-fast during backend service outages.
Why It Matters
Edge resilience prevents cascading outages, mitigates Denial of Service (DoS) attacks, and maintains system availability during partial network failures.
Redis Rate Limiter Configuration
Spring Cloud Gateway uses the **Token Bucket Algorithm** to throttle traffic. It requires three parameters:
• replenishRate: Specifies how many tokens are added to the bucket per second.
• burstCapacity: The maximum capacity of the bucket, defining the maximum burst traffic allowed.
• KeyResolver: Identifies requests (e.g. by client IP, user ID, or OAuth claims) to apply rate limits individually.
Practical Example
Let's write the configuration file for a gateway route configured with a Redis RateLimiter and a Resilience4j Circuit Breaker:
Next, write the corresponding KeyResolver bean in Java:
Quick Quiz
Q1: In the Token Bucket algorithm, what parameter controls the maximum burst rate of requests allowed in a single split-second surge?
A) replenishRate
B) burstCapacity
C) key-resolver
D) slidingWindowSize
Answer: B — burstCapacity defines the maximum size of the bucket, capping the request count allowed in a sudden traffic surge.
Scenario-Based Challenge
Production Scenario:
You configure a circuit breaker filter at the gateway. When the catalog service crashes, the gateway opens the circuit and returns an HTTP 504 Gateway Timeout directly to users, causing a poor user experience. How do you customize the behavior to return a default product list page instead?
View Solution
To implement dynamic fallbacks at the edge:
1. **Configure fallbackUri:** Ensure the CircuitBreaker filter has a fallbackUri defined (e.g. forward:/fallback/products).
2. **Implement Fallback Controller:** Create a controller in the API Gateway application matching that path:
@RestController
public class FallbackController {
@GetMapping("/fallback/products")
public ResponseEntity<List<Product>> productsFallback() {
return ResponseEntity.ok(List.of(new Product("0", "Cache Item", 0.0)));
}
}
This intercepts the failure and returns a default response, degrading gracefully.
Interview Questions
1. How does a Redis Rate Limiter keep track of request counts across multiple gateway instances?
By using a shared Redis database. Each gateway instance queries Redis and runs a Lua script to evaluate token counts in the bucket atomically. This centralizes rate-limiting state, preventing clients from bypassing limits by rotating between different gateway instances.
Production Considerations
Tune rate-limiting parameters carefully. Setting burstCapacity too low can drop requests from legitimate users on shared networks (which look like a single IP address).