Why Rate Limiting Matters Every public API eventually faces abuse.
It could be a malicious script hammering your endpoints, a misconfigured client retrying in a loop, or just a sudden spike in traffic.
Without rate limiting, your backend can get overwhelmed, leading to slow responses or crashes.
Rate limiting protects your service, keeps costs predictable, and ensures fair usage among all consumers.
I remember my first production incident: a client accidentally sent thousands of requests per second to a search endpoint.
The database CPU spiked to 100%, and the whole app became unresponsive for minutes.
A simple rate limit would have prevented that entirely.
What Is Rate Limiting?
Rate limiting controls how many requests a client can make within a given time window.
It's a policy that defines a threshold and an action when the threshold is exceeded.
Common actions include rejecting the request with or delaying it.
The key concepts are: Limit: Maximum requests allowed in a window (e.g., 100 requests per minute).
Window: The time period for the limit (e.g., 1 minute, 1 hour).
Identifier: Who is being limited?
Usually an IP address, API key, or user ID.
Simple Fixed Window Algorithm The simplest approach is the fixed window.
You track requests per identifier in a time bucket.
For example, allow 10 requests per minute.
When a request comes in, you increment the counter for the current minute.
If the counter exceeds 10, reject.
Here's a minimal implementation in Node.js using an in-memory map: This works for simple cases, but has a flaw: a client can burst at the end of one window and the start of the next, effectively doubling the rate.
For example, 10 requests at 59 seconds, then 10 more at 61 seconds.
Sliding Window Log A more accurate approach is the sliding window log.
You store timestamps of each request and count how many fall within the last .
This avoids the burst issue but uses more memory.
This is more precise but can be slow if the list grows large.
For high-traffic APIs, you'd want a more efficient structure like a token bucket.
Token Bucket Algorithm The token bucket is popular because it allows bursts while smoothing out long-term rate.
Think of a bucket that holds up to tokens.
Each request consumes one token.
Tokens are added at a fixed rate (e.g., 1 token per second).
If the bucket is empty, the request is rejected.
Here's a simple implementation: This allows a client to make 10 requests immediately, then 1 per second afterwards.
It's a good balance between responsiveness and protection.
Real-World Considerations In production, you rarely write your own rate limiter.
Libraries like for Node, or services like Redis-based limiters (e.g., ) handle distributed scenarios.
But understanding the basics helps you choose the right one.
Key points to consider: Distributed systems: In-memory maps don't work across multiple instances.
Use a shared store like Redis with atomic operations.
Identify clients properly: IP addresses can be shared (NAT, proxies).
Prefer API keys or user IDs when available.
Response headers: Include , , and so clients know how to behave.
Graceful degradation: When a limit is hit, return a clear error with a header so clients can back off.
Testing Your Rate Limiter Always write tests.
For a simple limiter, test that: Requests within the limit pass.
Requests over the limit get
429.
The window resets correctly.
Here's a quick test using Node's built-in test runner: Final Thoughts Rate limiting is a fundamental tool for any API developer.
Start with a simple fixed window if you're prototyping, but for production, consider a token bucket or a battle-tested library.
The most important part is being intentional: know your limits, document them, and handle the 429s gracefully on the client side as well.
Your future self, and your users, will thank you when the inevitable traffic spike doesn't take down your service.