Vibe Engines is free, interactive AI engineering & system design — runs in your browser. Pick a learning path
Little's Law Calculator
Little's Law ties the three numbers every capacity plan rests on: the average number of in-flight requests, the throughput, and the latency. Solve for whichever you don't know — size a connection pool from QPS and latency, back out the latency a queue depth implies, or find the throughput a fixed worker count can sustain. Includes a utilisation read-out so you can see when you're driving the system into the danger zone.
Hot. Past ~80% utilisation, small bursts cause large queue spikes. Keep steady-state headroom for variance.
L = λ × W holds for the long-run average of any stable system. It sizes for the mean — add headroom for the tail (p99 spikes, bursts) it deliberately ignores.
Little’s Law ties together the three numbers that describe any busy system: how many requests are in flight, how fast they arrive, and how long each one takes. Pick any two and the third is fixed. This calculator solves for whichever one you choose, then compares the concurrency with the number of servers, workers or connections you have.
What it computes
L = λ × W Solve for: Concurrency
λ = L ÷ W Solve for: Throughput
W = L ÷ λ Solve for: Latency
ρ = L ÷ servers utilization of your slots- L is the average number of requests inside the system at once, whether being served or waiting.
- λ (lambda) is the average arrival rate in requests per second. In a stable system arrivals equal completions, so this is also your throughput.
- W is the average time a request spends inside the system. You enter it in milliseconds and the tool divides by 1,000, so the units multiply out: requests per second × seconds = requests.
- servers is how many requests you can serve at once: worker processes, threads, pool connections or GPU slots. Utilization ρ is L as a share of them.
The utilization bar has three bands: below 80% reads “Healthy headroom”, 80% up to 100% reads “Hot”, and 100% or more reads “Over capacity”. Results of 1,000 or more display as whole numbers, 10 to 999 with one decimal, and anything smaller with two.
A worked example
The defaults solve for concurrency with a throughput of 500 requests per second, a latency of 40 ms and 24 servers.
- Convert latency:
40 ms ÷ 1,000 = 0.04 s. - Concurrency:
L = 500 × 0.04 = 20, shown as 20.0 in-flight requests. - Utilization:
ρ = 20 ÷ 24 = 0.833, shown as 83% (“20.0 of 24 slots”). That is past the 80% line, so the panel reads Hot. - Switch to Throughput and the same numbers come back the other way: 20 in flight at 40 ms is
20 ÷ 0.04 = 500.0requests per second. Switch to Latency:20 ÷ 500 × 1,000 = 40.0 ms.
| Change from the defaults | Concurrency L | Utilization of 24 | Band |
|---|---|---|---|
| None: 500 req/s at 40 ms | 20.0 | 83% | Hot |
| Latency doubles to 80 ms | 40.0 | 167% | Over capacity |
| Traffic falls to 300 req/s | 12.0 | 50% | Healthy |
| Servers raised to 30 | 20.0 | 67% | Healthy |
| Traffic rises to 600 req/s | 24.0 | 100% | Over capacity |
The second row is the one to remember. A downstream slowdown with no change in traffic doubles the concurrency you need, which is how a slow database turns into an exhausted worker pool.
Assumptions and limits
- A stable system. The law holds for long-run averages when arrivals and completions balance. If the queue is growing because arrivals exceed capacity, there is no steady state and the averages keep drifting.
- Averages, not tails. Concurrency fluctuates around L. Bursts and slow requests push the instantaneous count well above it, so the mean is a floor, not a size.
- One boundary for all three numbers. λ, W and L must describe the same system. Time measured at the load balancer includes network and queueing that a database pool never sees.
- Utilization is approximate. L ÷ servers is the true busy fraction only when W is the time a request actually occupies a slot. If W includes time spent waiting for a slot, L counts the waiting requests too, and the ratio measures demand rather than busy time.
- Latency depends on load. The tool treats W as a fixed input, but real latency climbs as utilization nears 100%. In the simplest single-server queue model (M/M/1), the average time in the system is the service time divided by
1 − ρ: 5 times the service time at 80% busy and 10 times at 90%.
How to use the result
- Size a pool or worker count. Take peak throughput and the latency at peak, compute L, then divide by your target utilization. For the defaults,
20 ÷ 0.75 = 26.7, so 27 slots keeps you at 75%. - Find the ceiling of what you have. Rearranged, maximum throughput is
servers ÷ W: 24 slots at 40 ms complete at most 600 requests per second, or 480 at 80% busy. - Read a queue as a wait.
W = L ÷ λturns a backlog into time: 50 jobs in the system with 200 arriving per second means each spends 250 ms inside. - Sanity-check dashboards. If measured in-flight requests are far from throughput × latency, the metrics are measuring different boundaries or the system is not stable.
- Keep headroom for variance. The 80% line is a rule of thumb. Spiky traffic and long tails need more margin; smooth traffic spread across many servers can run hotter.
How it works
- L = λ × W: concurrency = throughput × latency, with latency entered in milliseconds.
- Solve for any one of the three from the other two.
- Utilization ρ = L ÷ servers; 80% and above is flagged as hot, 100% as over capacity.
- Exact for long-run averages of a stable system, not for bursts or tails.
Frequently asked questions
What is Little's Law?
Little's Law says that in any stable system, the average number of items inside it (L) equals the average arrival rate (λ) times the average time each item spends inside (W): L = λW. John Little published a general proof in 1961. It holds whatever the arrival pattern, service order or number of servers, which is why it underpins so much capacity planning. If 500 requests per second each take 40 ms, then 500 × 0.04 = 20 requests are in flight on average.
How do I use Little's Law to size a connection pool or thread pool?
Multiply peak throughput by the time each request holds a connection or thread, in seconds. That product is the average number of slots in use. Then divide by a target utilization so bursts have room: at 500 requests per second and 40 ms you need 20 slots on average, and 20 ÷ 0.75 rounds up to 27 slots at 75% busy. Too few and requests queue for a slot, which raises latency. Far too many wastes memory and can overload whatever sits downstream.
What does the utilization percentage mean?
It is the average concurrency L divided by the servers or workers you entered: the share of your slots in use. The tool marks under 80% as healthy, 80% to just under 100% as hot, and 100% or more as over capacity. Past roughly 80%, queueing delay grows quickly. In the simplest single-server queue model, the average time in the system is the service time divided by one minus the utilization, so 5 times the service time at 80% and 10 times at 90%.
How do I convert milliseconds for Little's Law?
Keep the units consistent: if throughput is in requests per second, latency must be in seconds. 40 ms is 0.04 seconds, so 500 requests per second times 0.04 seconds gives 20 requests in flight. Mixing units is the most common mistake, and it gives answers off by a factor of 1,000. This calculator takes latency in milliseconds and divides by 1,000 internally, and when it solves for latency it converts back to milliseconds, so you can type the numbers your dashboards show.
Does Little's Law work with p99 latency?
Not directly. The law is exact for averages: mean concurrency equals mean throughput times mean latency. Plugging in a p99 latency does not give you a p99 concurrency, but it does give a deliberately pessimistic size, as if every request were slow at once. A practical approach is to size from the mean with the law, then check that your headroom also covers peak traffic at tail latency, and treat the gap between the two numbers as your safety margin.
Why does Little's Law not match my metrics?
Usually because the three numbers are measured at different boundaries or over different windows. Latency measured at the client includes network and queueing time that the server's in-flight gauge never sees, and a short window during a burst is not a steady state. The law also needs a stable system: if requests arrive faster than they complete, the queue grows and the averages drift. Measure all three at the same point over the same period, ideally several minutes long, and the relationship should hold closely.