Caching & Performance
Making systems fast — caching strategies, CDNs, rate limiting and low-latency design — with a capacity estimator and the systems that live or die on cache hit rates.
Handbooks 3
The Redis Handbook
Redis as an in-memory data-structure server, not just a cache — the core structures (strings, hashes, lists, sets, sorted sets), why single-threaded is fast, RDB vs AOF persistence, eviction and TTL, the classic patterns (cache, rate limiter, leaderboard, session, queue, pub/sub), high availability with replication, Sentinel and Cluster, and when it's the wrong tool.
Redis vs Memcached
Both put data in RAM by a key, so they look interchangeable — but Memcached does one thing (a lean multithreaded cache) while Redis is a data-structure server with persistence, replication and pub/sub. The one-job-vs-many split, a feature table, and why most teams now default to Redis.
The Caching Patterns Handbook
Why the fastest query is the one you never run. A cache serves requests from a fast store and falls back to a slow source on a miss, so average latency is a hit-rate-weighted blend of the two paths — h·t_cache + (1−h)·t_source, dominated by the miss term. That makes hit rate the one number that decides a cache's value. The patterns (cache-aside, write-through, write-back), the eviction policy, and the two famously hard problems — invalidation/staleness and the cache stampede — that turn a speedup into an outage if you get them wrong. With worked math and runnable code.
System Designs 19
Design a URL Shortener
Build a URL shortener (think Bitly or TinyURL). Learn how to mint unique short codes with base62, make the read-heavy redirect path sub-millisecond with caching, shard billions of mappings, and track clicks asynchronously.
Design Twitter
Build Twitter's news feed. Learn the social graph, why fan-out on write beats read-time merging, how a precomputed timeline cache makes feed reads O(1), how ranking surfaces the best tweets, and how a hybrid model solves the celebrity problem.
Design YouTube
Build a planet-scale video platform. See how raw uploads become an adaptive-bitrate ladder through an async transcoding pipeline, how a CDN serves immutable segments from the edge, how tiered blob storage holds exabytes affordably, and how view counts stay async.
Design a Rate Limiter
Build a distributed rate limiter. Learn where to put the check, how to key and tier limits, the token-bucket and sliding-window algorithms, why shared atomic counters in Redis avoid race conditions, and the fail-open vs fail-closed trade-off.
Design Typeahead (Autocomplete)
Build a search typeahead. See how a prefix trie turns search into an O(L) walk, how a prefix-keyed cache answers the skewed common case in under a millisecond, how precomputed top-k ranking surfaces the best ten, how an offline builder keeps suggestions fresh from query logs, and how to personalize and shard.
Design a Distributed Cache
Build a distributed cache like Redis or Memcached. See how cache-aside shields the database, how consistent hashing shards across nodes, how TTL and LRU/LFU eviction bound memory, how replication and invalidation keep it correct, and how to survive cache stampedes and hot keys.
Design a News Feed
Build a social news feed. See how the social graph, pull vs push fan-out, a precomputed feed cache, relevance ranking, the hybrid model that tames the celebrity problem, blending multiple content sources, and cursor pagination fit together.
Design Netflix
Build a planet-scale video streaming service. See how the play path splits authorization from byte delivery, how origin storage and CDN edges serve immutable segments, how an adaptive-bitrate ladder adapts to any connection, how a parallel transcoding pipeline builds it, and how Open Connect, recommendations and QoE events fit together.
Design Instagram
Build a photo-sharing app. See how splitting media from metadata, a CDN for immutable images, async image processing, a precomputed feed cache, asynchronous like/view counters, TTL-based ephemeral stories, and sharding fit together to serve billions of photos.
Design Pastebin
Build a text-paste service. See how base62 key generation, separating metadata from blob storage, a cache and CDN for the read-heavy immutable path, TTL expiration, privacy flags and size limits, and async analytics fit together.
Design a Stock Exchange
Build a stock exchange matching engine. See how an in-memory order book with price-time priority matches orders, how pre-trade risk checks guard the core, how a single sequencer and single-threaded engine give deterministic low-latency matching, and how market-data feeds, journaling/replay, clearing and symbol partitioning fit together.
Design a CDN
Build a Content Delivery Network like Cloudflare or Akamai. See why distance is the enemy, how edge PoPs cache content near users, how anycast/GeoDNS routes to the nearest edge, how Cache-Control TTLs and tiered shield caches protect the origin, how versioned URLs and purge solve invalidation, and how stale-while-revalidate keeps it fast and up.
Design DNS
Build the Domain Name System. See why one central name→IP table can't work, how a delegated hierarchy of root, TLD and authoritative servers splits the namespace, how a recursive resolver walks it, how caching with TTLs makes lookups instant, how replication and anycast make it DDoS-proof, and how GeoDNS, DNSSEC and propagation work.
Design a Feature Flag System
Build a feature flag / experimentation platform like LaunchDarkly. See why redeploying to toggle a feature is slow and risky, how a flag store + dashboard lift decisions out of code, how an SDK evaluates flags locally with zero network calls, how consistent-hash bucketing gives sticky percentage rollouts, how local cache + streaming + edge deliver a millisecond kill switch, and how exposure events turn a flag into an A/B experiment.
Design a Leaderboard
Build a real-time leaderboard for millions of players. See why SQL ORDER BY + COUNT rank melts at scale, how a sorted set (Redis ZSET) gives O(log N) update/rank and O(log N + K) top-K, how relative "around me" boards work, why sharding a global ranking is uniquely hard, how time-windowed boards reset cleanly, and how a durable store backs the rebuildable in-memory index.
Design a Distributed Counter
Build a distributed counter for view counts and likes at massive scale. See why a single UPDATE row melts under write contention, how sharding into N sub-counters trades read cost for parallel writes, how batching through a stream cuts write volume, how a cached total makes reads O(1), how HyperLogLog counts uniques in a few KB, and how idempotency keys make increments exactly-once.
Design a Live Streaming System
Build a live video streaming system like Twitch or YouTube Live. See why it optimizes for cheap CDN fan-out instead of ultra-low latency, how ingest accepts one broadcaster push, how transcoding builds an adaptive bitrate ladder, how HLS/DASH segmenting turns live video into cacheable HTTP files, how a CDN fans them to millions, the latency-vs-scale dial, and how live chat and DVR/VOD fit in.
Design a Flash Sale System
Build a flash-sale / limited-inventory system for a massive concurrent drop. See why read-then-decrement oversells, how admission control and a virtual waiting room pace the thundering herd, how an atomic inventory decrement guarantees exactly N buyers win, how reserve-then-confirm with a hold TTL and async order processing work, and how to tame the single hot inventory key with fairness and anti-bot defenses.
Design a Hotel Booking System
Build a hotel/room booking platform like Booking.com or Marriott. See why read-then-decrement inventory oversells rooms, how a multi-night stay needs an all-or-nothing atomic transaction across every date, how reserve-then-confirm with a hold TTL prevents both overselling and cart-abandonment loss, why dynamic pricing is cached rather than computed live, how search is split from the strongly-consistent booking path (CQRS), and how controlled overbooking is a business policy the same atomic system enforces.
AI System Designs 1
Coding Challenges 8
LRU Cache
The eviction policy behind every size-limited cache: when you run out of room, throw out whatever was used least recently. The trick is O(1) get and put — an ordered hash map (Python dict / JS Map) gives you exactly that. Solve it in Python or TypeScript.
Streaming Median
Latency dashboards do this every second: maintain the median of a stream without re-sorting per event. The classic two-heap trick — a max-heap for the low half, a min-heap for the high half, the median always at the boundary. Solve it in Python or TypeScript.
Token Bucket Rate Limiter
The algorithm inside most production rate limiters — and it never runs a timer. Refill the bucket lazily from the time elapsed since the last request, cap at capacity, spend one token or reject. Two numbers of state per client, exactly like the Redis version. Solve it in Python or TypeScript.
Sliding-Window Rate Limiter
Allow at most N requests per rolling window — the rate limiter that guards real APIs. A sliding log of accepted timestamps gives exact limits without fixed-window bursts. Decide accept/reject for a stream. Solve it in Python or TypeScript, with hidden tests.
LFU Cache
The cache that evicts what you use least often — and, on ties, least recently. Harder than LRU: track frequency and recency together, still O(1) per op. Replay get/put operations. Solve it in Python or TypeScript, with hidden tests.
Bloom Filter
A tiny bit-array that answers "have I seen this?" in a fraction of a set’s memory — with occasional false positives but never a false negative. Build one with double hashing. Provided hashes keep both languages in sync. Hidden tests.
HyperLogLog Cardinality Estimator
Count how many distinct items a stream held — billions of them — using a few kilobytes, not a giant set. Turn "the longest run of leading zeros" into an estimate. Powers COUNT(DISTINCT) in Redis and BigQuery. Solve it in Python or TypeScript, with hidden tests.
KV-Cache Eviction (Attention Sinks)
An LLM’s KV cache grows every token, so long chats must drop old entries without wrecking quality. StreamingLLM keeps the first few "attention sink" tokens plus a sliding window, evicting the middle. Compute the survivors. Solve it in Python or TypeScript, with hidden tests.
Labs 2
The Eviction Race: LRU vs LFU
Don't read about cache eviction — race the policies. Feed the same access stream to three caches of equal size and watch them make different choices: FIFO evicts the oldest, LRU evicts the least-recently-used, LFU evicts the least-frequently-used. On a hot-key-plus-scan workload, LFU keeps the hot item while LRU and FIFO throw it away — and the hit rates diverge. Step the stream and watch, made playable, with theory and a quiz.
Backpressure: The Bounded Queue
Don't read about backpressure — feel the queue fill. A fast producer feeds a slow consumer through a bounded buffer. Without backpressure the queue overflows and you silently drop work; with backpressure you slow the producer to the consumer's pace so nothing is lost — you trade latency for reliability. Toggle backpressure and watch drops turn into controlled waiting, made playable, with theory and a quiz.
Interactive Tools 6
Interactive Capacity Estimator
Slide DAU, requests per user, payload size and read/write ratio and watch QPS, storage per year, bandwidth, shard count and cache memory recompute live — every number with the formula behind it. The napkin math interviewers expect, made interactive.
Percentile Calculator
Drop in a list of latencies (or any numbers) and get p50, p90, p95 and p99 instantly, plus min, max and mean — with the sorted distribution drawn as a histogram and each percentile marked on it. See exactly why the average hides your worst requests and the tail is what users feel.
Token Bucket Simulator
A live token-bucket rate limiter you can play. Set the bucket capacity and refill rate, then send requests by hand or turn on auto traffic — watch the bucket drain when you burst, refill when you pause, and reject requests once it hits empty. The algorithm behind almost every production rate limiter, made tactile.
Latency Budget Builder
Assemble a chat request stage by stage — network, gateway, retrieval, rerank, prefill, decode — and watch the waterfall: time to first token, full response time, a stacked bar of where the milliseconds went, and which stage to attack first when the budget blows.
Cache ROI Calculator
A cache ROI calculator. Enter your request rate, cache hit rate, and the latency of a hit versus a miss to see your effective average latency, how much faster that is than going to the origin every time, and how much backend load the cache removes. Add a per-origin-request cost to estimate the money saved.
Latency Numbers Explorer
An interactive version of the classic "latency numbers every programmer should know." Compare the real time of an L1 cache reference, a memory read, an SSD read, a same-datacenter round trip and a cross-continent round trip — then scale them all to human time (where one CPU cycle is a second) to feel just how enormous the gaps really are.
About caching & performance
Almost every fast system is fast because it avoids work — and caching is how. A cache keeps the results of expensive operations close and cheap to fetch, turning a database query or a network round-trip into a sub-millisecond lookup. The art is in what to cache, how to evict (LRU, LFU, TTL), and how to stay correct as the underlying data changes.
Beyond caches, performance is about the whole path: CDNs that move content to the edge, rate limiters that protect a service from overload, and capacity math that sizes it all. This topic collects the tools and systems where hit rate and tail latency are the difference between a snappy product and a slow one.