Part 025 — Distributed Rate Limiting Foundation

Rate limiting is not just an API gateway feature.

In enterprise backend systems, rate limiting is a control mechanism for protecting scarce resources, preserving fairness, enforcing tenant boundaries, reducing abuse, and keeping downstream systems alive under load.

Redis is often used as the backend for distributed rate limiting because it is fast, atomic at command execution level, TTL-aware, and shared across application instances.

But a rate limiter is not correct just because it uses Redis.

The real design question is:

Which actor is being limited, against which resource, over which time window,
with what burst tolerance, with what failure behavior, and with what business consequence?

A weak rate limiter can cause two different classes of incidents:

Too loose  -> downstream overload, noisy neighbor, abuse, database pressure
Too strict -> customer impact, false rejection, failed workflows, support escalation

The goal of this part is to build the mental model before implementation.

Implementation details come in Part 026.

1. What Rate Limiting Actually Controls

Rate limiting controls the rate at which an actor can perform an action.

The actor can be:

• IP address
• authenticated user
• tenant
• customer account
• API client
• service account
• integration partner
• endpoint
• region
• cluster
• whole system

The action can be:

• HTTP request
• quote calculation
• price lookup
• order submit
• token refresh
• login attempt
• report export
• async job enqueue
• Kafka/RabbitMQ publish
• downstream API call
• expensive PostgreSQL query

A rate limiter is not only about security.

It is also about resource governance.

In a CPQ/order-management style system, expensive operations may include:

• quote recalculation
• product compatibility evaluation
• catalog rule evaluation
• order validation
• eligibility check
• pricing simulation
• external integration call
• bulk import
• retry storm from integration partners

These operations can be much more expensive than a normal CRUD request.

A limiter should be designed around cost, not only request count.

2. Rate Limiting vs Throttling vs Quota

These terms are related, but not identical.

A mature platform may use several controls together.

Example:

Per-user API limit       -> prevent accidental client loop
Per-tenant burst limit   -> protect shared service capacity
Global system limit      -> protect database or downstream dependency
Concurrency limit        -> protect expensive calculation engine
Daily quota              -> enforce commercial or contractual entitlement

Do not collapse all of these into one Redis key.

They answer different operational questions.

3. Why Distributed Rate Limiting Exists

A local in-memory limiter works only inside one process.

That may be fine for a single instance.

But enterprise services usually run multiple replicas:

Pod A
Pod B
Pod C
Pod D

If each pod allows 100 requests/minute locally, the actual system limit is not 100.

It is roughly:

100 * number_of_pods

This becomes unstable during scaling.

3 pods -> 300 requests/minute
8 pods -> 800 requests/minute
2 pods -> 200 requests/minute

A distributed limiter uses shared state so all pods observe the same limit.

Redis is commonly used for that shared state.

Java/JAX-RS Pod A ----\
Java/JAX-RS Pod B ----- Redis limiter keys
Java/JAX-RS Pod C ----/

The value is global coordination.

The cost is network dependency, Redis latency, and new failure modes.

4. Where Rate Limiting Can Live

Rate limiting can be implemented at multiple layers.

In a Java/JAX-RS service, rate limiting often appears as:

JAX-RS request filter
  -> extract user/tenant/endpoint
  -> call limiter
  -> reject with 429 or continue
  -> resource method
  -> service layer

But some limits belong deeper:

Quote recalculation service
  -> check tenant calculation quota
  -> check global calculation concurrency
  -> execute expensive rule engine

A gateway limit protects the platform.

A domain limit protects business invariants.

You may need both.

5. What a Limiter Decision Contains

A limiter should not return only true/false.

It should return a decision object.

Example:

public record RateLimitDecision(
    boolean allowed,
    String limitName,
    String subject,
    long limit,
    long remaining,
    long resetEpochMillis,
    long retryAfterMillis,
    String reason
) {}

A decision should support:

• HTTP response mapping
• audit logging
• metrics
• customer support diagnosis
• Retry-After header
• debug traces
• product entitlement explanation

A rejected request without enough metadata becomes difficult to debug.

6. Common Limiter Subjects

6.1 Per-IP Limit

Useful for unauthenticated endpoints.

Examples:

• login
• password reset
• registration
• public catalog lookup
• token endpoint

Risk:

• NAT/shared office IP can cause false positives
• attacker can rotate IPs
• IPv6 aggregation decisions are non-trivial

Use per-IP limits as one signal, not the only control.

6.2 Per-User Limit

Useful after authentication.

Examples:

• 100 quote reads/minute/user
• 5 export requests/hour/user
• 10 password reset attempts/hour/user

Risk:

• service accounts may represent many users
• user ID must be stable and non-sensitive
• user-level limits do not prevent tenant-wide overload

6.3 Per-Tenant Limit

Very important in multi-tenant enterprise systems.

Examples:

• 1,000 quote calculations/minute/tenant
• 50 order submissions/minute/tenant
• 10 concurrent bulk jobs/tenant

Risk:

• large tenants may need higher limits
• tenant ID must be extracted reliably
• tenant hierarchy may matter
• internal tenants/test tenants may need special profiles

6.4 Per-Endpoint Limit

Useful when endpoint cost differs.

Example:

GET /quotes/{id}           -> cheap read
POST /quotes/{id}/calculate -> expensive compute
POST /orders                -> business-critical command

Do not apply the same limit to cheap and expensive operations without thinking.

6.5 Global Limit

Protects shared dependency capacity.

Examples:

• max 5,000 pricing requests/minute across all tenants
• max 500 external tax API calls/minute
• max 1,000 PostgreSQL-heavy search requests/minute

Risk:

• can punish all tenants because of one noisy tenant
• requires fairness strategy
• should be combined with per-tenant limits

7. Key Dimensions of a Rate Limit

A rate limit is defined by more than a number.

subject     -> who/what is limited
resource    -> which operation/resource is protected
scope       -> endpoint, service, tenant, global, dependency
window      -> time interval or refill model
capacity    -> allowed volume
burst       -> short-term spike allowance
cost        -> weight per action
response    -> reject, delay, degrade, enqueue, fallback
failure     -> fail-open or fail-closed
observed by -> gateway, service, worker, dashboard

Example:

Limit name: quote-calculate-per-tenant
Subject: tenantId
Resource: POST /quotes/{id}/calculate
Window: sliding window 60 seconds
Limit: 300 requests
Burst: no explicit extra burst
Cost: 1 per request
Response: HTTP 429 with Retry-After
Failure mode: fail-open for low-risk tenants, fail-closed for abuse-sensitive endpoint

A rate limit without this definition is just a magic number.

8. Fixed Window

Fixed window is the simplest strategy.

Example:

Allow 100 requests per minute.
Counter key: rate:user:123:2026-07-11T10:31

All requests in the same minute increment the same counter.

If the counter exceeds 100, reject.

Strengths:

• simple
• cheap
• easy to understand
• easy to implement with Redis INCR and TTL

Weaknesses:

• boundary burst problem
• unfair near window edges
• can allow almost double traffic across adjacent windows

Boundary example:

10:00:59 -> 100 requests allowed
10:01:00 -> another 100 requests allowed

Actual load:

200 requests in ~2 seconds

Fixed window is acceptable for coarse limits.

It is weak for precise protection of expensive dependencies.

9. Sliding Window Counter

Sliding window counter approximates a rolling window by combining current and previous fixed windows.

Example:

previous window count weighted by overlap
current window count counted fully
estimated count = weighted_previous + current

Strengths:

• smoother than fixed window
• cheaper than sliding log
• good enough for many API limits

Weaknesses:

• approximate
• more complex than fixed window
• still not perfect fairness

Use when you need a balance between accuracy and cost.

10. Sliding Log

Sliding log stores each request timestamp.

With Redis, this is commonly done using a sorted set:

key: rate:user:123:quote-calculate
member: request-id or timestamp
score: timestampMillis

Decision flow:

remove entries older than window
count entries in current window
if count < limit -> add current request and allow
else reject

Strengths:

• accurate rolling window
• good fairness
• clear audit of recent events

Weaknesses:

• more memory
• cleanup required
• more CPU than fixed window
• can be expensive for very high traffic subjects

Sliding log is good for stricter limits with moderate cardinality.

It is risky for massive high-cardinality traffic without cleanup discipline.

11. Token Bucket

Token bucket models capacity as tokens that refill over time.

bucket capacity: 100 tokens
refill rate: 10 tokens/second
request cost: 1 token

If a token is available, consume it and allow.

If not, reject or delay.

Strengths:

• allows controlled burst
• smooth long-term rate
• natural for API traffic
• good for user experience

Weaknesses:

• implementation complexity
• needs accurate time calculation
• often requires Lua for atomicity

Token bucket is often a strong default for user-facing API burst control.

12. Leaky Bucket

Leaky bucket models a queue that drains at a fixed rate.

Incoming requests fill the bucket.

The bucket leaks at a constant rate.

If the bucket is full, new requests are rejected.

Strengths:

• smooth output rate
• protects downstream system
• useful when downstream cannot absorb burst

Weaknesses:

• may feel less flexible to clients
• queueing/delay semantics complicate HTTP path
• pure rejection variant resembles token bucket with different behavior

Leaky bucket is useful when you care about smoothing downstream throughput more than allowing burst.

13. Concurrency Limiting Is Different

Rate limiting controls actions per time.

Concurrency limiting controls in-flight work.

Example:

Allow max 10 concurrent quote calculations per tenant.

This protects CPU, thread pools, external dependencies, and DB connections.

Redis can implement distributed semaphores for concurrency, but the correctness concerns are closer to locking and leases.

Do not use request-per-minute limits as a substitute for concurrency limits.

A tenant can stay below per-minute quota but still start too many expensive concurrent operations.

14. Weighted Rate Limits

Not all requests have equal cost.

Example:

GET quote summary       cost = 1
GET quote details       cost = 2
POST calculate quote    cost = 10
POST bulk import        cost = 100

A weighted limiter subtracts different token amounts per operation.

This is useful when endpoint cost differs significantly.

Risk:

• cost model can become political
• incorrect weights cause unfairness
• business and platform teams must agree on cost meaning

Weighted limits should be documented.

15. Local vs Distributed Limiter

Local Limiter

State is inside one JVM.

Strengths:

• very fast
• no network dependency
• useful as first-line protection
• protects local CPU/thread pool

Weaknesses:

• not globally accurate
• changes with pod count
• state resets on restart

Distributed Limiter

State is shared, often in Redis.

Strengths:

• consistent across pods
• supports tenant/global limits
• works with horizontal scaling

Weaknesses:

• Redis dependency on request path
• higher latency
• failure mode must be designed
• hot limiter keys can overload Redis

Many production systems use both:

Local limiter       -> protect each pod from sudden burst
Distributed limiter -> enforce tenant/global fairness

This layered approach reduces Redis pressure.

16. Redis as Rate Limiter Backend

Redis is a good fit because:

• commands are fast
• single command execution is atomic
• TTL is native
• counters are easy
• sorted sets support sliding logs
• Lua supports atomic multi-step algorithms
• shared state works across pods

Redis is not magic because:

• Redis can be unavailable
• Redis can become hot-key bound
• Redis failover can affect requests
• Redis Cluster creates multi-key constraints
• time source matters
• memory can grow with high-cardinality keys
• Lua scripts can block the server if badly written

A Redis limiter is part of the critical request path.

Treat it as production infrastructure.

17. Rate Limiter Key Design

Limiter keys need careful design.

Example format:

rl:{env}:{service}:{limitName}:{subjectType}:{subjectId}:{window}

Examples:

rl:prod:quote-api:quote-calc:tenant:t-123:20260711T1031
rl:prod:auth-api:login:ip:203.0.113.8:20260711T10
rl:prod:order-api:submit:user:u-456

Design rules:

• never put raw PII in keys
• normalize endpoint names
• avoid unbounded arbitrary strings
• include tenant/user/IP only after validation/canonicalization
• use hash tags if Redis Cluster requires multi-key locality
• apply TTL to limiter keys
• consider cardinality before launch

Bad key:

rate:john.smith@example.com:/api/v1/orders?search=...

Better key:

rl:prod:order-api:search:user:u_7f3a:20260711T1031

18. High Cardinality Risk

Rate limiters can create many keys.

High-cardinality subjects:

• IP addresses
• anonymous sessions
• user IDs
• idempotency-like request IDs
• API keys
• endpoint+tenant combinations

If each subject creates several keys per minute, memory can grow quickly.

Example:

1,000,000 users/day
5 limiter dimensions
2 windows per dimension
= 10,000,000 keys/day before expiry behavior

TTL helps, but memory pressure and expiration churn still matter.

Measure cardinality before production rollout.

19. Hot Key Risk

Global limiters can create hot keys.

Example:

rl:prod:quote-api:global:quote-calc:20260711T1031

Every pod increments the same key.

This can become a bottleneck.

Mitigations:

• avoid overly aggressive global limiter frequency
• combine local pre-limit with distributed global limit
• shard global counters and aggregate approximately
• use gateway-level limit for coarse global protection
• monitor command rate per key if available

A global limiter protects the system, but can also overload Redis.

20. Rate Limiting and HTTP Semantics

For HTTP APIs, rate limiting usually maps to:

HTTP 429 Too Many Requests

Useful headers:

Retry-After: 12
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1783765920

Be careful with header semantics.

For security-sensitive endpoints, you may intentionally return less detail.

For enterprise API clients, good metadata helps integration teams implement backoff.

A useful 429 body may include:

{
  "error": "rate_limit_exceeded",
  "limitName": "quote-calculate-per-tenant",
  "retryAfterSeconds": 12,
  "correlationId": "..."
}

Never leak sensitive internal topology or tenant data in the error response.

21. Fail-Open vs Fail-Closed

What happens if Redis is unavailable?

Two options:

fail-open   -> allow request when limiter cannot decide
fail-closed -> reject request when limiter cannot decide

Fail-Open

Strengths:

• protects availability
• avoids customer impact due to Redis issue
• useful for low-risk business operations

Weaknesses:

• abuse can pass through
• downstream can be overloaded
• limiter outage removes protection

Fail-Closed

Strengths:

• protects sensitive systems
• prevents abuse when state cannot be checked
• useful for security endpoints

Weaknesses:

• Redis issue becomes user-facing outage
• can block legitimate traffic
• can cause broad incident

Choose by endpoint risk.

Example:

Do not leave this decision implicit.

22. Rate Limiting and PostgreSQL

Rate limiting often protects PostgreSQL.

Examples:

• expensive search queries
• quote recalculation reads
• catalog expansion
• batch import validation
• report generation

If PostgreSQL is the bottleneck, the limiter should reflect database cost.

But Redis limiter state is outside the DB transaction.

Important questions:

• Is the limiter checked before DB work?
• Is the limiter cost refunded if validation fails?
• Does every request cost the same?
• Is a rejected request visible in audit logs?
• Is the DB protected from cache-miss storm separately?

A rate limiter is not a substitute for query optimization.

It is a protective control.

23. Rate Limiting and Kafka/RabbitMQ

Rate limiting can protect message producers and consumers.

Producer-side examples:

Limit async job enqueue per tenant
Limit event publish rate per integration partner
Limit bulk import event production

Consumer-side examples:

Limit downstream API calls from consumers
Limit retry storm rate
Limit DLQ replay speed
Limit projection rebuild traffic to Redis/PostgreSQL

Important distinction:

HTTP rate limit protects synchronous request path.
Consumer rate limit protects asynchronous processing path.

A consumer can overload downstream systems even when HTTP traffic is normal.

Rate limits should exist near the pressure source.

24. Rate Limiting in Kubernetes

Kubernetes changes limiter behavior.

Scaling replicas changes local capacity.

Rolling deployments can create connection bursts to Redis.

Pod restarts reset local limiter state.

CPU throttling can make clients timeout and retry.

Important review points:

• total Redis connections = pod count * pool size
• local limiter capacity scales with pod count
• distributed limiter Redis key pressure increases with pod count
• retry policy can amplify Redis load
• HPA scaling can change limiter dynamics
• graceful shutdown should avoid partial request duplication

Rate limiter design must be tested under replica scaling.

25. Observability Requirements

A production limiter needs metrics.

Minimum metrics:

allowed_count{limitName, subjectType, endpoint}
rejected_count{limitName, subjectType, endpoint}
redis_error_count{limitName}
redis_latency_ms{limitName}
fallback_decision_count{mode}
remaining_tokens/sample
high_cardinality_key_count estimate

Useful logs on rejection:

correlationId
limitName
subjectType
subjectHash
endpoint
remaining
retryAfter
decision

Avoid logging raw IP/user/tenant if policy forbids it.

Use hashed or internal IDs where appropriate.

26. Failure Modes

Common rate limiter failure modes:

Most limiter incidents are design/configuration incidents, not Redis command incidents.

27. Debugging Questions

When a user reports unexpected rate limiting, ask:

Which limit rejected the request?
Who was the subject?
Which endpoint/action was limited?
What was the configured limit?
What was the observed count/token state?
Was Redis healthy?
Was there a deployment or scaling event?
Was traffic retried by client/gateway?
Was there a tenant-specific override?
Was the request authenticated consistently?
Was the limit local, distributed, or both?

When downstream overload happens despite limiter, ask:

Was the protected resource correctly identified?
Was the limit too high?
Was there an unprotected path?
Did async consumers bypass the limiter?
Did local limits multiply by pod count?
Did retry behavior amplify load?
Did cache misses create database pressure outside the limiter?

28. Design Checklist

Before implementing a Redis rate limiter, define:

• limit name
• business purpose
• protected resource
• subject type
• subject ID extraction logic
• endpoint/action mapping
• window/refill model
• limit value
• burst tolerance
• weighted cost model if needed
• Redis key naming
• TTL policy
• fail-open/fail-closed behavior
• HTTP response shape
• Retry-After behavior
• metrics and logs
• operational owner
• configuration override process
• test strategy

If these are not defined, implementation is premature.

29. Internal Verification Checklist

For CSG/team verification, check:

• Which services currently implement rate limiting.
• Whether rate limiting is done in gateway, JAX-RS filter, service layer, worker, or client wrapper.
• Redis client used for limiter.
• Limiter key naming convention.
• Whether keys include tenant/user/IP/endpoint safely.
• TTL policy for limiter keys.
• Whether limiter commands are atomic.
• Whether Lua scripts are used.
• Whether fail-open/fail-closed is documented.
• Whether 429 response includes Retry-After.
• Whether rate limit config is environment-specific.
• Whether tenant-specific overrides exist.
• Whether limiter metrics exist.
• Whether rejection logs are searchable by correlation ID.
• Whether Redis timeout/retry behavior can amplify incidents.
• Whether Kubernetes pod scaling changes limiter behavior.
• Whether Kafka/RabbitMQ consumers have separate throttling.
• Whether PostgreSQL-heavy operations are protected.
• Whether security-sensitive endpoints have stricter rules.
• Whether support/on-call has a runbook for false 429 incidents.

30. PR Review Checklist

When reviewing a PR introducing rate limiting, ask:

• What exact resource is being protected?
• Is the subject extraction correct and stable?
• Is this local or distributed?
• What happens as pod count changes?
• What Redis keys are created?
• Do all limiter keys expire?
• Is the algorithm fair enough?
• Is Redis operation atomic?
• Is there a hot key risk?
• Is there high-cardinality risk?
• What happens if Redis is down?
• Is 429 response useful but safe?
• Are metrics and logs added?
• Are tests covering boundary behavior?
• Are tenant overrides controlled?
• Can operations tune the limit without code change?
• Is this protecting PostgreSQL, Kafka/RabbitMQ, external API, CPU, or business quota?

A good rate limiter PR should make the operational behavior obvious.

31. Summary

Distributed rate limiting is a coordination problem.

Redis is a strong backend for rate limiter state, but the correctness comes from the design:

subject + resource + window + algorithm + TTL + failure behavior + observability

The most important architectural decisions are:

• what you limit
• who you limit
• where the limiter runs
• which algorithm matches the business need
• whether Redis failure allows or rejects traffic
• how you detect false positives and false negatives

Part 026 turns this foundation into concrete Redis implementation patterns.