Token Bucket Simulator
A live token-bucket rate limiter you can play. Set the bucket capacity and refill rate, then send requests by hand or turn on auto traffic — watch the bucket drain when you burst, refill when you pause, and reject requests once it hits empty. The algorithm behind almost every production rate limiter, made tactile.
Two knobs: burst and sustained rate
A rate limiter has to answer one question millions of times a second: let this request through, or not? The token bucket is the algorithm behind most of them because it separates two things you actually care about. A bucket holds up to capacity tokens and refills at a steady rate per second. Every request spends one token; if the bucket is empty, the request is rejected. Capacity controls how big a burst you tolerate; refill rate controls the sustained requests-per-second over the long run — and you tune them independently.
Why bursts are a feature
Real traffic is spiky. A user idle for a while, then firing ten quick calls, isn't abusive — it's normal. The token bucket banks tokens during quiet periods (up to the capacity ceiling) so that burst sails through, then settles the client back to the refill rate. That's the key difference from a leaky bucket, which forces a perfectly constant output rate and would needlessly throttle that harmless burst.
Watch it drain
Send requests faster than the bucket refills and you'll watch the level fall to zero and requests start bouncing (the red pips). Pause, and the bucket climbs back up at exactly the refill rate. Turn on auto traffic above the refill rate and the accept rate settles at roughly refill ÷ traffic — the limiter shedding exactly the excess.
How it's really built
Production limiters don't run a background timer per client — that wouldn't scale to millions of keys. Instead they store just two numbers per key, usually in Redis: the current token count and the timestamp of the last refill. On each request they lazily add (now − last) × rate tokens (capped at capacity), then check for one. Stateless between calls, cheap, and shareable across every server behind a load balancer.
How it works
- Each request spends one token; an empty bucket rejects (or delays) it.
- Capacity = burst you tolerate; refill rate = sustained requests/sec.
- Idle time banks tokens up to capacity, spendable as a burst.
- At scale it’s two numbers in Redis (count + last-refill time), refilled lazily.
Frequently asked questions
How does the token bucket algorithm work?
A bucket holds up to "capacity" tokens and refills at a fixed rate. Each request removes one token; if the bucket is empty the request is rejected (or delayed). The bucket size sets how big a burst you allow, while the refill rate sets the sustained request rate over the long run — the two knobs give you burst tolerance and a steady-state limit independently.
What is the difference between token bucket and leaky bucket?
Token bucket allows bursts: if the bucket is full you can spend all its tokens at once, then wait for it to refill. Leaky bucket smooths output to a constant rate regardless of input bursts, like a queue draining at a fixed speed. Token bucket is the more common choice for API rate limiting because it tolerates short spikes gracefully.
How do capacity and refill rate interact?
Refill rate is your sustained requests-per-second ceiling; capacity is how much slack accrues while you are idle, which you can spend in a burst. A capacity of 10 with a refill of 5/s lets a quiet client fire 10 immediately, then settle to 5/s. Bigger capacity means burstier traffic is accepted; a smaller one enforces a smoother rate.
How are token buckets implemented at scale?
Rather than a background timer, production limiters store two numbers per key — the token count and the last-refill timestamp — usually in Redis, and lazily add (now − last) × rate tokens on each request before checking. That makes the limiter stateless between calls and cheap to run across many servers sharing one bucket.