Vibe Engines is free, interactive AI engineering & system design — runs in your browser. Pick a learning path
Cache ROI Calculator
A cache ROI calculator. Enter your request rate, cache hit rate, and the latency of a hit versus a miss to see your effective average latency, how much faster that is than going to the origin every time, and how much backend load the cache removes. Add a per-origin-request cost to estimate the money saved.
A cache pays off in two currencies, latency and backend load, and both follow from one number: the hit rate. This calculator takes your request rate, hit rate and the latency of a hit and a miss, and returns the average latency users see, how much traffic still reaches the origin, and what the avoided origin requests are worth each month.
What it computes
h = hit rate ÷ 100 clamped to 0..1
effective = h × hit latency + (1 − h) × miss latency
faster by = (miss latency − effective) ÷ miss latency
origin after = request rate × (1 − h)
load reduction = 1 ÷ (1 − h) ∞ at a 100% hit rate
saved per month = request rate × h × cost per request × 2,628,000- Request rate: requests per second arriving at the cache.
- Hit rate: the share of requests the cache answers without touching the origin.
- Hit latency and miss latency: end-to-end time on each path. A miss usually pays the cache lookup and the origin, so include both.
- Cost per origin request: what one origin request costs in compute, database load or egress. Leave it at 0 to skip the dollar figure.
The "faster than all-miss" percentage compares the blend with a world where every request took the miss path. The month is 2,628,000 seconds, a 365-day year divided by 12.
A worked example
The defaults are 5,000 requests per second, a 90% hit rate, 2 ms hits, 80 ms misses and $0.00002 per origin request.
- Effective latency:
0.9 × 2 + 0.1 × 80 = 1.8 + 8.0= 9.8 ms. - Saving:
(80 − 9.8) ÷ 80 = 87.75%, shown as 88% faster than all-miss. - Origin traffic:
5,000 × 0.1 = 500requests per second, shown as 5,000 → 500 with 90% of traffic offloaded. - Load reduction:
1 ÷ 0.1= 10.0× less backend load. - Money:
4,500 hits/s × $0.00002 = $0.09a second, and$0.09 × 2,628,000= $236,520 a month.
Holding everything else at the defaults and moving only the hit rate:
| Hit rate | Effective latency | Origin req/s | Load reduction | Saved per month |
|---|---|---|---|---|
| 50% | 41.0 ms | 2,500 | 2.0× | $131,400 |
| 80% | 17.6 ms | 1,000 | 5.0× | $210,240 |
| 90% | 9.8 ms | 500 | 10.0× | $236,520 |
| 95% | 5.9 ms | 250 | 20.0× | $249,660 |
| 99% | 2.8 ms | 50 | 100.0× | $260,172 |
The dollar column grows with hits, so it barely moves from 90% to 99%. The load column grows with the inverse of misses, so the same nine points cut origin traffic tenfold. Hit rate near the top is mostly a capacity and latency win, not a billing one.
Assumptions and limits
- Averages only. Effective latency is a mean. Every percentile above the hit rate is a miss, so at 90% the p95 and p99 are still 80 ms.
- A steady hit rate. Real hit rates dip after deploys, cold starts, evictions and TTL expiry, and vary by key. Measure at peak, not over a quiet day.
- Gross savings. The dollar figure counts avoided origin requests only. The cache's own servers, memory and operations are not subtracted.
- Two fixed latencies. Every hit and every miss costs the same. Large values, hot shards and slow serialization are not modeled.
- No failure mode. If the cache goes down, all 5,000 requests per second reach an origin that was sized for 500.
How to use the result
- Chase misses, not hits. Near the top each point of hit rate removes a large share of the remaining misses: 90% to 95% halves origin load. Longer TTLs, more memory and caching negative results are where those points come from.
- Size the origin for the cache-down case, or protect it with request coalescing, rate limits and load shedding, so a cache outage becomes a slowdown rather than an outage.
- Check the tail separately. If the objective is on p99, the hit rate has to exceed 99% before the cache improves it at all.
- Net out the cache's cost. Subtract the monthly cost of the cache cluster from the saving. If little remains, the latency gain has to justify the added complexity on its own.
How it works
- Effective latency = hit% × hit latency + (1 − hit%) × miss latency.
- Origin traffic is the miss rate; load reduction = 1 ÷ (1 − hit rate).
- Shows origin QPS before vs after the cache.
- Optional per-request cost gives gross dollars saved over a 2,628,000-second month.
Frequently asked questions
How is effective cache latency calculated?
It is the weighted average of the two paths: hit rate times hit latency plus miss rate times miss latency. If 90% of requests hit a 2 ms cache and the other 10% take 80 ms from the origin, the average is 0.9 times 2 plus 0.1 times 80, which is 9.8 ms. That blend is the mean latency users see, and it explains why even a moderate hit rate cuts the average sharply when misses are expensive.
Why does a higher cache hit rate matter more than it looks?
Because the misses carry the cost. Going from an 80% to a 90% hit rate halves the miss traffic, from 20% to 10%, which halves both the slow requests and the origin load. Going from 90% to 99% cuts misses tenfold while the hit rate rises only nine points. In the default example that move takes average latency from 9.8 ms to 2.8 ms and origin traffic from 500 to 50 requests per second.
How much backend load does a cache remove?
The share of requests removed equals the hit rate, so the origin sees the miss rate. The reduction factor is 1 divided by the miss rate: a 90% hit rate leaves 10% of traffic and means 10 times less backend load, while 99% means 100 times less. The calculator shows origin QPS before and after, so you can check whether the cache lets the existing backend cope, and how far the backend would have to scale without it.
Does a cache improve p99 latency?
Only once the miss rate falls below 1%. Under a simple two-latency model, every percentile above the hit rate is a miss, so with a 90% hit rate the p95 and p99 are still the full miss latency even though the average looks fast. If your service level objective is set on p99, the hit rate has to exceed 99% before the cache moves that number at all, which is why tail latency targets often need a faster origin as well as a cache.
How is the monthly cost saving calculated?
It multiplies the hits per second by the cost of one origin request and by 2,628,000 seconds, a 365-day year divided by 12. With the defaults, 4,500 hits per second at $0.00002 each is $0.09 a second, or $236,520 a month. This is a gross figure: it counts only the origin work avoided and does not subtract what the cache cluster itself costs to run, so the net return is lower.
What does a cache not help with?
A cache helps repeated reads of data that can tolerate some staleness. It does nothing for unique or uncacheable requests, which still pay the full origin cost, and it brings its own problems: invalidation and stale data, a thundering herd of misses when a hot key expires, and an origin that may be overwhelmed if the cache fails. The return shown here assumes a healthy hit rate, and achieving and keeping it is the real work.