Vibe Engines is free, interactive AI engineering & system design — runs in your browser. Pick a learning path
Rate Limit Designer
A rate limiter designer. Enter the sustained rate you want to allow and how big a burst to tolerate, and get concrete token-bucket parameters (refill rate and capacity), the effective peak burst, and how the choice compares to fixed-window and sliding-window approaches — including the boundary-burst pitfall of fixed windows.
Algorithm comparison
- Token bucket (recommended)
refill 1.67 / sec, capacity 150Smooth average rate with a controlled burst. Cheap state (count + timestamp).
- Fixed window
100 per 60s, hard resetBoundary bug: up to 200 requests across a reset edge. Simplest, but spiky.
- Sliding window
100 over a moving 60sNo boundary spike, most precise — but more state (timestamps / weighted counters).
A rate limit is a promise about traffic, and a token bucket turns that promise into two numbers a server can enforce. You state a quota, such as 100 requests per minute, and how much burst to tolerate. This designer converts them into a refill rate and a bucket capacity, and sets the same quota beside a fixed window and a sliding window.
What it computes
There are three inputs: the limit L in requests, the window T in seconds, and the burst allowance b, a multiple of the quota.
refill rate r = L / T tokens added per second
capacity C = max(1, ⌈L × b⌉) tokens the bucket holds
fixed window worst case = 2 × L across one reset edgeEach request spends one token, and an empty bucket rejects. A real limiter stores only the token count and the time of its last update, and tops the bucket up lazily whenever a request arrives: tokens = min(C, tokens + r × elapsed). The refill label prints two decimals below 10 per second, whole numbers above that, and switches to "1 every N s" when the rate is under one per second. A bound follows from the two parameters: in any span of t seconds, a token bucket admits at most C + r × t requests.
A worked example
The defaults are 100 requests per 60 seconds with a burst allowance of 1.5.
- Refill:
100 / 60 = 1.6667tokens per second, shown as 1.67 / sec, which is one token every 0.6 seconds. - Capacity:
⌈100 × 1.5⌉ = 150, the max burst a client can send after sitting idle. - Fixed window: 100 requests at 11:00:59 and 100 more at 11:01:00 put 200 through in about a second, which the comparison flags as the boundary bug.
- Sliding window: 100 over any moving 60 seconds, with no edge to exploit.
The most each scheme lets through, for a client that starts with a full bucket and sends as fast as it is allowed:
| Span | Token bucket, C = 150 | Fixed window | Sliding window log |
|---|---|---|---|
| 1 s | 151 | 200 | 100 |
| 10 s | 166 | 200 | 100 |
| 60 s | 250 | 200 | 100 |
The bucket is far gentler than the fixed window over a few seconds, yet over a full minute it admits up to 250, because a 1.5× burst rides on top of a minute of refill. That is what a bucket sized for bursts is meant to do, and it is worth knowing before you describe 100 per minute as a hard cap.
Assumptions and limits
- One token per request. Requests of different cost, such as LLM calls priced by tokens, need a weighted bucket where each request spends its own cost.
- Continuous refill. The bucket fills smoothly. Implementations that add tokens in discrete ticks admit slightly different bursts at the tick edges.
- One limiter. Ten servers each holding their own bucket allow ten times the rate unless the state is shared or the limit is divided.
- Reject, not queue. An empty bucket rejects. A leaky bucket queue that delays requests instead smooths output but adds latency, and is not modeled.
- Edge inputs. A limit or window of zero gives no refill, which the label shows as "1 every Infinity s"; capacity never drops below one token.
How to use the result
- The refill rate is the long-run promise. Keep it, times the number of clients you expect to be active at once, below what the backend can actually serve.
- Size capacity from real client behavior. A page load that fires 20 API calls needs at least 20 tokens, or legitimate users are rejected on their first click. A small capacity protects a fragile downstream; a large one suits batch clients.
- Meet a hard quota with headroom. If a vendor allows 100 per minute over any moving window, size your client bucket so
C + r × T ≤ 100, for example 10 tokens of capacity refilling at 1.5 per second. - Tell clients when to retry. Return HTTP 429 with a Retry-After header. The wait for one token is 0.6 seconds at the defaults, which rounds up to 1 in whole seconds.
How it works
- Refill rate r = limit ÷ window; capacity C = ⌈limit × burst allowance⌉.
- In any span of t seconds a token bucket admits at most C + r × t requests.
- Fixed windows can pass 2 × the limit across a single reset edge.
- Sliding windows cap every moving window at the limit, at the cost of more state.
Frequently asked questions
How do I convert a rate limit into token bucket parameters?
Divide the limit by the window to get the refill rate, and multiply the limit by the burst allowance to get the capacity. For 100 requests per 60 seconds with a burst allowance of 1.5, the bucket refills at 100 / 60 = 1.67 tokens per second, one every 0.6 seconds, and holds 150 tokens, rounded up to a whole token. Each request spends one token and is rejected when none are left, so the refill rate is the sustained throughput and the capacity is the largest burst after an idle period.
Why is a token bucket usually preferred over a fixed window?
A fixed window, such as 100 requests per minute that resets on the minute, has a boundary problem: a client can send 100 requests at 11:00:59 and another 100 at 11:01:00, landing 200 in about a second without breaking the per-minute rule. A token bucket has no reset edge. In any span of t seconds it admits at most its capacity plus the refill rate times t, so short bursts are capped by the capacity and the long-run average by the refill rate. It needs only a token count and a timestamp per client.
What is the difference between a sliding window and a token bucket?
A sliding window counts requests over a moving interval, so no 60-second span can ever contain more than the limit. The exact version, a sliding log, stores a timestamp for every request in the window; the common approximation, a sliding window counter, weights the previous fixed window's count by how much of it still overlaps. A token bucket is cheaper and deliberately allows a burst on top of the average, so over a full window it can admit more than the limit. Choose a sliding window when the quota is a hard contract.
What is the difference between a token bucket and a leaky bucket?
A leaky bucket used as a queue accepts requests into a fixed-size buffer and releases them at a constant rate, so the output never bursts; arrivals that find the buffer full are dropped, and queued requests wait. A token bucket admits a burst immediately, up to its capacity, and enforces only the average rate over time. Used as a meter instead of a queue, the leaky bucket is equivalent to a token bucket with the same rate and size, and the generic cell rate algorithm (GCRA) is a common way to implement either.
How do I rate-limit across many servers?
A per-server in-memory bucket lets a client exceed the global limit by spreading requests across servers: ten servers with their own buckets allow ten times the rate. For a global limit, keep the bucket in a shared store such as Redis and update it atomically, for example with a Lua script that refills and spends in one step, trading a network round trip for a correct cluster-wide count. If traffic is evenly balanced, dividing the limit across servers is a cheaper approximation. The algorithm is the same; the state has to live where every limiter can see it.
What should an API return when a request is rate limited?
Return HTTP status 429 Too Many Requests, defined in RFC 6585, with a Retry-After header saying how many seconds to wait. With a token bucket the honest wait is the time to refill one token, which is 0.6 seconds at 100 requests per minute, so Retry-After, which takes whole seconds, rounds up to 1. Many APIs also send the limit and the remaining count in headers so clients can slow down before they are rejected. Clients should back off with jitter rather than retry in lockstep.