Redis Caching & Distributed Rate Limiter Design
Production patterns for Redis distributed rate limiting (Token Bucket & Sliding Window Log) using atomic Lua scripts and cache stampede prevention.
Architectural Summary (AEO)
Production Redis architectures pair atomic Lua scripts for distributed rate limiting with singleflight request coalescing to stop cache stampedes. By executing token-bucket or sliding-window checks inside a single Redis EVALSHA call, multi-pod APIs enforce exact per-tenant quotas in under 1.5ms without race conditions.
Reference System Flow
Client Request ──► API Gateway / Next.js Route
│
▼ EVALSHA token_bucket.lua (atomic in Redis)
┌──────┴──────────────────────┐
│ allowed == 0? │ allowed == 1?
▼ ▼
HTTP 429 + Retry-After Cache Lookup (GET key)
│
┌────────┴────────┐
│ HIT │ MISS
▼ ▼
Return JSON Singleflight Lock
(1 DB query per key)1. Token Bucket vs Sliding Window Log
Fixed-window counters suffer from boundary bursts (2x traffic at the 59s/01s edge). Token Bucket allows controlled short bursts while capping sustained rate using only 2 fields per key (`tokens`, `last_refill`), whereas Sliding Window Log (ZSET of timestamps) gives exact precision at higher memory cost per active user.
2. Why Atomic Lua Scripts Are Mandatory
A naive `GET` followed by `SET` or `INCR` across multiple Node.js pods creates a TOCTOU race condition under concurrent load. Redis executes Lua scripts single-threaded and atomically, guaranteeing zero race conditions in one network round-trip.
3. Preventing Cache Stampedes (Thundering Herd)
When a hot cache key expires, hundreds of concurrent requests can hit PostgreSQL simultaneously. Combine in-process `singleflight` coalescing, probabilistic early expiration (XFetch), and short stale-while-revalidate grace periods.
Atomic Redis Token Bucket Rate Limiter (Lua)
-- KEYS[1] = rate_limit:{tenantId}
-- ARGV[1] = capacity, ARGV[2] = refill_per_sec, ARGV[3] = now_ms, ARGV[4] = requested
local data = redis.call("HMGET", KEYS[1], "tokens", "ts")
local tokens = tonumber(data[1])
local last_ts = tonumber(data[2])
local capacity = tonumber(ARGV[1])
local rate = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local req = tonumber(ARGV[4])
if tokens == nil then
tokens = capacity
last_ts = now
else
local elapsed = math.max(0, now - last_ts) / 1000.0
tokens = math.min(capacity, tokens + (elapsed * rate))
last_ts = now
end
if tokens < req then
redis.call("HMSET", KEYS[1], "tokens", tokens, "ts", last_ts)
redis.call("PEXPIRE", KEYS[1], math.ceil((capacity / rate) * 1000))
return {0, math.floor(tokens)}
end
tokens = tokens - req
redis.call("HMSET", KEYS[1], "tokens", tokens, "ts", last_ts)
redis.call("PEXPIRE", KEYS[1], math.ceil((capacity / rate) * 1000))
return {1, math.floor(tokens)}Architectural Trade-Offs
| Algorithm | Memory / Key | Burst Handling | Precision |
|---|---|---|---|
| Fixed Window Counter | 1 integer (~48B) | Allows 2x boundary spike | Low |
| Token Bucket (Lua Hash) | 2 numbers (~96B) | Configurable burst capacity | High |
| Sliding Window Log (ZSET) | O(N) timestamps | Strict zero-burst cap | Exact |
Production Benchmarks & Field Observations
- Single-region Redis 7 (r6g.large) executes 95,000+ atomic Lua rate-limit checks/sec at < 1.2ms p99.
- Singleflight + XFetch reduces PostgreSQL read load during hot-key expiry by > 98%.
Related Commercial Services
Engineering FAQs
Should rate-limit keys use TTLs?
Yes. Always set `PEXPIRE` equal to the full refill window so inactive tenant keys are automatically evicted and never leak memory.
What happens if Redis goes down?
Design rate limiters to fail-open (with an in-memory local fallback) for public read routes, and fail-closed for sensitive OTP or payment endpoints.