Concepts

A rate limiter caps how many requests are accepted within a period of time. You set a rule such as "100 requests per minute per user" or "5 login attempts per minute per IP", and requests beyond it are rejected or delayed. This chapter explains why we limit requests, which algorithms exist, and the vocabulary used throughout the rest of the guide.

Why limit requests

Server resources are finite. When one client floods a service in a short time, everyone else gets slower responses, and in the worst case the whole service goes down. A rate limiter protects against:

• Overload: sudden traffic spikes or a buggy retry loop should not knock over your database and servers.
• Unfairness: no single user should be able to monopolize shared capacity.
• Abuse: password brute forcing, account enumeration and scraping become much slower.
• Runaway cost: calls that cost money, such as paid third-party APIs or SMS, stay within budget.
• Plan tiers: "60 requests per minute on Free, 1,000 on Pro" is a common way to differentiate products.

Vocabulary

Five classic algorithms

• Fixed window: cut time into slots such as one minute and keep one counter per slot. Simplest of all, but around a slot boundary a client can send up to twice the limit.
• Sliding window log: store the timestamp of every request and count only those within the last minute. Exact, but memory grows with the limit.
• Sliding window counter: blend the previous and current slot counters by how far into the current slot you are. Cheap and quite accurate.
• Token bucket: tokens drip into a bucket at a fixed rate and each request takes one. It enforces an average rate while allowing bursts up to the capacity.
• Leaky bucket: requests go into a queue that drains at a constant rate. Output is perfectly smooth, at the price of added waiting time.

A fixed window counter takes only a few lines. The following chapters fix its weaknesses one at a time.

import time
from collections import defaultdict

LIMIT, WINDOW = 5, 60          # 5 requests per minute
counters = defaultdict(int)    # (key, window number) -> count


def allow(key: str) -> bool:
    window_id = int(time.time() // WINDOW)
    counters[(key, window_id)] += 1
    return counters[(key, window_id)] <= LIMIT

This sketch never deletes old counters. Real code cleans up past windows or uses a store with expiry, such as Redis.

Choosing the key

The same algorithm behaves very differently depending on what you count against.

• User ID or API key: the fairest choice for authenticated traffic, and the natural place to attach per-plan limits.
• IP address: used before login (sign-in, sign-up, password reset). Watch out for NAT, where an office or a mobile carrier puts many people behind one address, and for IPv6, where a user often controls a whole /64 block and can rotate addresses freely.
• Endpoint: an expensive search API deserves a different limit than a cheap read.
• Combinations: keys such as user:42:POST /search give fine-grained control.

Behind a reverse proxy, be careful with X-Forwarded-For. Clients can set it to anything, so only trust entries appended by proxies you operate.

def client_ip(remote_addr: str, forwarded_for: str | None,
              trusted_proxies: set[str]) -> str:
    # Only trust the header when the direct peer is our own proxy
    if forwarded_for and remote_addr in trusted_proxies:
        hops = [h.strip() for h in forwarded_for.split(",")]
        # Walk from the right; the first untrusted hop is the real client
        for hop in reversed(hops):
            if hop not in trusted_proxies:
                return hop
    return remote_addr

After rejecting: HTTP 429 and Retry-After

Over HTTP, a request that exceeds the limit normally gets 429 Too Many Requests, a status code defined in RFC 6585. Adding a Retry-After header, either a number of seconds or an HTTP date, tells well-behaved clients exactly how long to back off. Many APIs also report the remaining quota and reset time in headers, such as GitHub's x-ratelimit-remaining.

HTTP/1.1 429 Too Many Requests
Retry-After: 12
Content-Type: application/json

{"error": "rate_limited", "message": "Too many requests. Retry in 12 seconds."}

Summary

• A rate limiter caps requests per key per period to prevent overload, monopolization, abuse and runaway cost.
• The classic algorithms are fixed window, sliding window log, sliding window counter, token bucket and leaky bucket.
• Picking the right key (user, IP, endpoint) matters as much as picking the algorithm.
• When you reject, answer with 429 and Retry-After so clients know when to try again.