Vibe Engines
YouTube

Vibe Engines is free, interactive AI engineering & system design — runs in your browser. Pick a learning path

Token Bucket Simulator

A live token-bucket rate limiter you can play. Set the bucket capacity and refill rate, then send requests by hand or turn on auto traffic — watch the bucket drain when you burst, refill when you pause, and reject requests once it hits empty. The algorithm behind almost every production rate limiter, made tactile.

Send faster than the refill rate and the bucket drains — extra requests get rejected until it tops back up.
10/ 10tokens available
Send some requests to see accepted (green) vs rejected (red)…
0Accepted
0Rejected
100%Accept rateof 0 requests
5/sSustained limitburst up to 10

A token bucket decides, for every request, whether to let it through, using only two numbers: how many tokens the bucket can hold and how fast it refills. This simulator runs a real bucket in your browser so you can burst it, starve it and watch it recover.

What it computes

The bucket has a capacity C of 1 to 50 tokens and a refill rate r of 1 to 50 tokens per second, and it starts full. A request is accepted if at least one whole token is available, and spends it; otherwise it is rejected and nothing is spent.

every 100 ms   tokens = min(C, tokens + r × 0.1)
on a request   if tokens ≥ 1: tokens −= 1, accepted += 1
               else:          rejected += 1
accept rate  = accepted / (accepted + rejected) × 100

Auto traffic adds rate × 0.1 to an accumulator on every tick and fires a request each time it passes 1, so 8 requests per second arrive evenly spaced rather than at random. The gauge shows tokens to one decimal, the strip shows the last 24 outcomes, and the sustained-limit card restates r and the burst ceiling C. One bound follows: over any t seconds, a bucket that starts full accepts at most C + r × t requests.

A worked example

The defaults are a capacity of 10, a refill rate of 5 per second and an auto traffic rate of 8 requests per second. Turn auto traffic on and the bucket loses 8 − 5 = 3 tokens a second, so it runs dry after 10 / 3 ≈ 3.3 seconds, which is when the first red pip appears. From then on, every accepted request is a freshly refilled token. Stepping the simulator's own loop gives:

ElapsedTokensAcceptedRejectedAccept rate
1 s680100%
3 s0240100%
5 s034685%
10 s0592174%
60 s030917164%

After a minute the bound predicts 10 + 5 × 60 = 310 accepted; the run gets 309 because the first two refills arrive while the bucket is still full and are capped away. The accept rate keeps falling toward 5 / 8 = 62.5%, the share a 5-per-second limit can serve from 8 per second of demand. Stop the traffic, wait two seconds, and the bucket is full again: ten quick clicks all pass, and an eleventh is rejected until the next whole token lands, at most 200 ms later.

Assumptions and limits

  • Ticked refill. Tokens arrive in 100 ms steps, half a token per step at 5 per second, and a request needs a whole one, so the gauge can read 0.5 and still reject.
  • Even traffic. Auto traffic is perfectly regular. Real arrivals clump, which drains a bucket sooner than the average rate suggests and makes capacity worth more.
  • One client, one bucket. A production limiter keeps a bucket per key, such as a user, API key or IP address, and shares it across servers.
  • Rejected means gone. Real clients retry, and retries without backoff add load exactly when the bucket is empty.
  • Unit cost. Every request costs one token; weighted limiters charge expensive calls more.

How to use the result

  • Read overload as refill ÷ traffic. When demand exceeds the refill rate, the limiter sheds exactly the excess: at 8 per second against 5, three requests in eight are rejected for as long as it lasts.
  • Size capacity to the bursts you want to forgive. A burst at rate λ runs for C / (λ − r) seconds before rejections start, 3.3 seconds here; double the capacity and the cushion doubles.
  • Keep the refill rate below what the backend sustains. Capacity forgives spikes, but the refill rate is the load you have promised to absorb indefinitely.
  • In production, refill lazily. Store the count and the last refill time per key, add (now − last) × r on each request, cap it at C, then spend. No timers, and the same decisions without the 100 ms steps.

When to use it, and when not

Use it to get a feel for the numbers before you commit to them: how long a burst lasts, what share of an overload gets through, and how fast a quiet client earns its burst back. Apart from the 100 ms steps, what you see is exactly what a lazily refilled production bucket does. To turn a written quota such as 100 requests per minute into C and r, use the rate limit designer.

A token bucket is the wrong tool in three cases. If requests differ widely in cost, you need a weighted bucket. If a contract says no more than L in any window, the bucket can break it, because it admits up to C + r × T in a window of T seconds; use a sliding window. And if you want to smooth traffic rather than reject it, use a queue drained at a fixed rate, or a cap on requests in flight.

Trade-offs and limits

ChoiceGainCost
Large capacityAt C = 20 the default overload runs about 6.7 s before the first rejectionA quiet client can land 20 requests on the backend in one instant
C = 1Output never bursts above rTwo requests closer than 1 / r, 200 ms at 5 per second, and the second is rejected
Bucket in a shared storeOne limit across every serverA round trip per request, and the store sits on the hot path
Bucket per serverNo network costThe real limit is r times the number of servers

Capacity buys time, not throughput. Run the default overload for 60 seconds with C = 20 instead of 10 and the simulator accepts 319 requests instead of 309: exactly the 10 extra tokens, spent once. The accept rate still heads to 62.5%.

Common mistakes

  • Rounding tokens to integers on each refill. At 5 per second with a request every 100 ms, each lazy refill adds 0.5 tokens. Floor it and move the timestamp forward anyway, and the bucket never refills at all. Store fractional tokens.
  • Each server using its own clock. Lazy refill multiplies elapsed time by r, so a server whose clock runs ahead hands out extra tokens. Take the time from the shared store, for example Redis TIME, inside the same atomic script.
  • Buckets that reset on deploy. In-memory buckets restart full, so every rolling deploy hands every client a fresh burst of C, often exactly when the backend is warming up.
  • Rejecting without saying when to retry. Clients that retry at once turn 3 rejections a second into extra load on an empty bucket. Send Retry-After; the wait for one token is 1 / r.

Check yourself

Capacity 20, refill 5, auto traffic 8. When does the first rejection come?

The bucket loses 8 − 5 = 3 tokens a second, so 20 / 3 ≈ 6.7 s. The simulator's first red pip comes at 6.5 s, because the first refill arrives while the bucket is still full and is capped away.

Raise auto traffic from 8 to 15. Why does the accepted count after 10 seconds stay at 59?

Accepted requests are capped by tokens, not by demand: at most 10 + 5 × 10 = 60 in 10 seconds, minus the refill lost at the start. Only rejections grow, to 91, and the accept rate falls toward 5 / 15 ≈ 33%.

At the defaults, how many requests can pass in 2 seconds?

From a full bucket, 10 + 5 × 2 = 20. From an empty one, only the refill: 10. The same client gets twice the throughput depending on whether it was quiet first.

The bucket is empty and the refill rate is 2. How long until a click is accepted?

Each 100 ms tick adds 0.2 tokens, so the count reaches 1 after five ticks: 500 ms, which is 1 / r.

refill r = 5/s 8 req/s ineven arrivals C = 10 tokens1 per request accept 5/stoken ≥ 1 reject 3/stoken < 1 full bucket lasts 10 / (8 − 5) ≈ 3.3 s, then 5 of 8 pass any t seconds admits at most C + r × t = 10 + 5t
The default overload in steady state. Capacity decides how long the burst lasts; the refill rate decides how much gets through after that.

How it works

  • Each request spends one whole token; an empty bucket rejects it.
  • Capacity = burst you tolerate; refill rate = sustained requests/sec.
  • Idle time banks tokens up to capacity, spendable as a burst.
  • Above the refill rate, the long-run accept rate tends to refill ÷ traffic.
  • At scale it’s two numbers in Redis (count + last-refill time), refilled lazily.

Frequently asked questions

How does the token bucket algorithm work?

A bucket holds up to a fixed number of tokens, its capacity, and gains new tokens at a steady refill rate until it is full. Each request removes one token. If no whole token is left, the request is rejected, or delayed in some designs. Capacity sets how large a burst can pass after a quiet period, and the refill rate sets the sustained number of requests per second over the long run, so the two knobs control burst tolerance and the steady-state limit independently.

What is the difference between a token bucket and a leaky bucket?

A token bucket lets bursts through: a full bucket can spend all of its tokens at once and then waits for the refill. A leaky bucket used as a queue smooths output to a constant rate whatever the input does, holding requests in a buffer and dropping them when it overflows. Token buckets are the more common choice for API rate limiting because they forgive short, harmless spikes. Used as a meter rather than a queue, the leaky bucket makes the same accept and reject decisions as a token bucket.

How do capacity and refill rate interact?

The refill rate is the sustained ceiling, and capacity is the slack a client banks while idle and can spend in a burst. With a capacity of 10 and a refill rate of 5 per second, a quiet client can fire 10 requests at once and then settle to 5 per second. Under steady traffic faster than the refill rate, the bucket empties after capacity divided by the excess rate: 10 tokens against 8 requests per second last about 3.3 seconds before rejections begin.

What accept rate should I expect when traffic exceeds the limit?

Over a long run, the accepted share approaches the refill rate divided by the offered rate. At the defaults, 8 requests per second against a refill of 5, that is 5 / 8, or 62.5 percent: five requests a second pass and three are rejected. The simulator reads higher at first because the starting 10 tokens are spent before any rejection, so after 10 seconds it shows 59 accepted and 21 rejected, 74 percent, and after a minute 309 and 171, 64 percent.

Why does the bucket show tokens but still reject a request?

Requests need a whole token, and the refill arrives in fractions. The simulator adds tokens every 100 milliseconds, so at 5 per second each tick adds half a token, and the gauge can read 0.5 while the next request is still rejected. A token only becomes spendable when the count reaches 1. Production limiters that refill lazily from timestamps behave the same way: a fractional balance builds up silently until it is large enough to pay for a request.

How are token buckets implemented at scale?

Production limiters do not run a timer per client. They store two numbers per key, the token count and the time of the last refill, usually in Redis, and on each request add the elapsed time multiplied by the refill rate, cap the result at the capacity, and then try to spend a token. Doing the refill and the spend in one atomic step, for example in a Lua script, stops concurrent servers from double-spending, so every server behind a load balancer can share one bucket per key.