blog.dopana

Back

Rate limiting protects APIs from abuse, brute-force attacks, and unexpected traffic spikes. This article breaks down popular rate limiting algorithms and demonstrates how to implement them in production.

AllowedExceededClientRate LimiterBackend API429 Too Many Requests

Why Do We Need Rate Limiting?#

Rate limiting acts like a nightclub bouncer for your backend servers:

  • Mitigate DDoS & Brute-Force Attacks: Slow down automated credential stuffing or scraping bots before they reach authentication handlers or databases.
  • Ensure Fair Resource Usage: Prevent a single aggressive consumer (“noisy neighbor”) from starving other users of bandwidth or compute capacity.
  • Shield Backends from Cascading Failures: Protect downstream databases, queues, and internal microservices from being overwhelmed during traffic spikes.
  • Control Third-Party API Costs: Keep operational costs predictable, especially when proxies wrap metered external APIs like OpenAI or Anthropic.

Core Algorithms#

1. Fixed Window Counter#

The simplest approach: count incoming requests within fixed time intervals (e.g., 60-second windows).

[!WARNING] Boundary burst issue: If a user sends 10 requests at 0:59 and another 10 requests at 1:00, they successfully execute 20 requests within 2 seconds, potentially doubling your intended server load.

2. Sliding Window Log#

Stores an exact timestamp for every request and calculates the number of requests that occurred during the preceding rolling window (e.g., past 60 seconds).

[!NOTE] Highly accurate with smooth boundary handling, but memory consumption grows linearly with request volume because every request timestamp is saved.

3. Sliding Window Counter (Redis)#

Combines the low memory footprint of fixed windows with the smoothness of sliding logs using Redis Sorted Sets (ZSET):

4. Token Bucket#

Imagine a bucket that holds up to NN tokens. New tokens are added at a constant refill rate RR (tokens per second). When a request arrives, it must consume 1 token to pass. If the bucket is empty, the request is rejected.

[!TIP] Key advantage: Accommodates brief bursts of traffic (up to the bucket capacity) while strictly bounding long-term average throughput. Widely adopted by AWS and Stripe.

5. Leaky Bucket#

Incoming requests enter a FIFO queue (the bucket) and leak out for processing at a strictly constant rate. If incoming requests exceed bucket capacity, excess requests leak over the edge and are immediately dropped or rejected.

Ideal for traffic shaping and smoothing background asynchronous job pipelines.

Implementation — Express Middleware#

A real-world Express server usually requires tiered rate limits depending on the route sensitivity and user tier:

Response Headers#

Always include rate limit metadata in HTTP response headers so clients can intelligently throttle retries using exponential backoff:

X-RateLimit-Limit: 100
X-RateLimit-Remaining: 45
X-RateLimit-Reset: 1718845200
Retry-After: 45
http

Cloudflare Rate Limiting#

Deploying rate limits directly to the network edge stops malicious traffic before it reaches your origin servers.

Create a rule via Wrangler CLI or Cloudflare Dashboard:

# Wrangler CLI command
wrangler ratelimit create my-limit --limit 100 --period 60
bash

Cloudflare Workers binding implementation:

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const key = request.headers.get('CF-Connecting-IP') || 'unknown';
    const { success } = await env.myRateLimiter.limit({ key });

    if (!success) {
      return new Response('Too Many Requests', { status: 429 });
    }

    return new Response('OK');
  },
};
typescript

Distributed Rate Limiting#

When running a cluster of API servers, local in-memory counters fail because requests from the same user hit different instances. A centralized in-memory store like Redis synchronizes state across the cluster:

Server 1Redis - Atomic CountersServer 2Server 3

Atomic Redis Lua script to prevent race conditions:

Best Practices#

  1. Always return standard headers: Enables well-behaved API clients to throttle themselves and respect Retry-After.
  2. Apply endpoint-specific policies: Keep public content open, but apply strict limits to compute-heavy queries and authentication endpoints.
  3. Use compound keys: Combine IP address, API key, and user ID (IP + apiKey + userId) to prevent bypassing limits by switching network connections.
  4. Leverage distributed stores: Use Redis with Lua scripts or connection pooling across multiple application instances.
  5. Implement graceful degradation: Fail open (allow requests) if your Redis cache temporarily becomes unreachable, rather than blocking legitimate users.
  6. Log violations: Continuously monitor 429 logs to detect coordinated bot attacks and evaluate whether threshold adjustments are needed.

Conclusion#

Rate limiting is not only a defensive security barrier; it is essential resource governance. Fixed window counters are quick to implement, sliding windows provide precision without boundary spikes, and token buckets deliver flexible burst tolerance. For multi-node architectures, use Redis atomic scripts; for global applications, enforce limits at the edge with Cloudflare Workers. Choose the algorithm that matches your traffic characteristics and infrastructure scale.

References#