50 System Design Interview Questions.

Drive it in four moves: clarify requirements and scope (functional and non-functional), estimate scale with back-of-the-envelope math, sketch a high-level architecture, then deep-dive the one or two components the constraints make interesting. Close by naming trade-offs and failure modes. Interviewers grade your process and communication far more than a "correct" diagram.

Functional requirements are what the system does ("users can post and follow"); non-functional ones are how well it must do it — latency, availability, durability, consistency, scale, cost. The non-functionals drive almost every architectural decision, so pinning them down early ("p99 under 200ms," "99.99% available," "eventual consistency is fine") is what turns a vague prompt into a designable problem.

Start from users and behavior: daily active users × actions/user = requests/day, divided by ~86,400 seconds for average QPS, then multiply by a peak factor (often 2–5×). Multiply request rate by payload size for bandwidth, and data-per-item by item count for storage — remembering replication and retention. Round aggressively; the goal is the right order of magnitude (is this 100 QPS or 100k QPS?), which decides whether a single database suffices or you need sharding and caching.

Latency is how long one request takes; throughput is how many you handle per second. Batching, buffering, and queuing raise throughput but add latency; conversely, optimizing for low latency (no batching, more parallelism) can waste capacity. Always reason in percentiles — p50, p95, p99 — because averages hide the tail latency that actually hurts users.

Vertical scaling (a bigger machine) is simple and needs no code changes, but hits a hardware ceiling and is a single point of failure. Horizontal scaling (more machines) is effectively unbounded and fault-tolerant, but demands statelessness, load balancing, and a way to partition data. Most systems scale vertically first for simplicity, then horizontally once they must.

An SLI is a measured indicator (e.g. request success rate); an SLO is your internal target for it (99.9% success); an SLA is the contractual promise to customers, usually looser than the SLO, with penalties if breached. The gap between SLO and 100% is your error budget — spend it on releases and risk; when it's exhausted, freeze changes and focus on reliability.

It spreads incoming requests across a pool of servers, giving you horizontal scale, fault tolerance (route around dead nodes via health checks), and a stable entry point that hides the fleet behind one address. It also enables zero-downtime deploys by draining connections from instances before they're replaced.

L4 balances on TCP/UDP — fast and protocol-agnostic, but blind to the payload. L7 understands HTTP, so it can route by path, header, or cookie, terminate TLS, and do content-based routing — at higher CPU cost. L7 is the norm for web apps; L4 shines where raw throughput or non-HTTP traffic matters.

Round-robin for uniform servers and cheap requests; least-connections when request durations vary widely; weighted variants when capacities differ; and consistent hashing when you need the same client or key to keep landing on the same node (sticky caches, sharded state) while minimizing reshuffling as nodes come and go.

It maps both keys and nodes onto a ring so that adding or removing a node only remaps the keys near it — about K/N keys — instead of nearly all of them as with plain hash(key) % N. Virtual nodes (many ring points per physical node) smooth out the load distribution. It's the backbone of distributed caches and databases like Cassandra and DynamoDB.

A content delivery network caches content at edge locations near users, cutting latency and offloading your origin. It's ideal for static assets (images, JS, video) and cacheable API responses; you control freshness with TTLs and cache-control headers, and purge or version URLs when content changes. Modern CDNs also run compute at the edge for lightweight dynamic logic.

Sticky sessions pin a user to one server so in-memory session state stays local — simple, but it undermines even load balancing and loses the session when that server dies. The preferred pattern is stateless servers with session state pushed to a shared store (Redis) or a signed token (JWT), so any server can handle any request and you scale freely.

Caching trades staleness for speed and load reduction by keeping hot data close to where it's used. You can cache at many layers: the browser, a CDN, an application/in-memory tier (Redis, Memcached), and inside the database (query and buffer caches). The closer to the user, the bigger the win — and the harder invalidation gets.

Cache-aside (lazy): the app reads the cache, falls back to the DB on a miss, and populates the cache — simple and common. Write-through: writes go to cache and DB together, keeping them consistent at write-latency cost. Write-back: writes hit the cache and flush to the DB asynchronously — fastest writes, but risks data loss if the cache dies before flushing.

LRU evicts the least recently used entry — a good default assuming recency predicts reuse. LFU evicts the least frequently used, better for stable hot sets but slow to forget old favorites. TTL expires entries after a fixed time, bounding staleness regardless of access. Real caches often combine them (e.g. LRU with per-key TTLs).

Because you must decide when cached data is stale across many replicas without a single source of truth for freshness. Strategies include short TTLs (simple, tolerates staleness), explicit invalidation on write (precise but needs reliable delivery to every cache), and versioned keys (change the key when data changes). Each trades some mix of consistency, complexity, and hit rate.

When a hot key expires, many requests miss simultaneously and hammer the database at once. Fixes: a lock or single-flight so only one request recomputes while others wait, early/probabilistic refresh before expiry, and serving stale-while-revalidate so users get old data while one worker refreshes. Jittering TTLs stops many keys expiring together.

Memcached is a lean, multi-threaded key-value cache — great for simple, high-throughput string caching. Redis adds rich data structures (lists, sets, sorted sets, streams), persistence, replication, pub/sub, and Lua scripting, so it doubles as a lightweight datastore, queue, or rate-limiter. Reach for Redis unless you specifically want Memcached's simplicity.

Choose SQL for structured, relational data needing ACID transactions, flexible ad-hoc queries, and strong consistency. Choose NoSQL when you need to scale writes horizontally, store flexible/semi-structured documents, or optimize for a known access pattern (key-value, wide-column, graph). The honest interview answer is that it depends on access patterns and consistency needs — not "NoSQL is faster."

An index (usually a B-tree) keeps a sorted structure over one or more columns so lookups and range scans are O(log n) instead of a full table scan. The cost is extra storage and slower writes, since every insert/update must maintain the index. Over-indexing hurts write-heavy tables; index the columns you actually filter, join, and sort on.

Replication keeps copies of data on multiple nodes for read scaling and fault tolerance. Single-leader (writes to one primary, reads from replicas) is simplest but has replication lag, so replicas can serve stale reads. Multi-leader and leaderless improve write availability but introduce write conflicts you must resolve. Synchronous replication is durable but slow; asynchronous is fast but can lose recent writes on failover.

Sharding partitions data across independent databases by a shard key so writes and storage scale horizontally. A good key spreads load evenly and keeps common queries within one shard; a bad one creates hot shards (e.g. sharding by country when one country dominates). Range-based keys ease range queries but skew easily; hash-based keys spread evenly but scatter range scans.

Normalization removes redundancy so each fact lives once — clean writes, but reads need joins. Denormalization duplicates data to serve reads without joins — fast reads at the cost of larger storage and keeping copies in sync on write. Normalize by default; denormalize deliberately for read-heavy hot paths, often materializing precomputed views.

Use object storage (S3, GCS) for large, immutable binary blobs — images, video, backups, model weights — where you need cheap, durable, effectively unlimited capacity but not transactional queries. The common pattern is to store the blob in object storage and its metadata in a database, serving the bytes directly from the store or a CDN.

A WAL records every change durably before applying it to the main data structures, so after a crash the database replays the log to recover a consistent state. It's the foundation of durability and atomic commits, and the same idea powers replication (ship the log to replicas) and log-structured storage engines.

When a network partition occurs, a distributed system can preserve at most one of strong Consistency (every read sees the latest write) or Availability (every request gets a non-error response). Since partitions are inevitable, the real choice is CP (reject requests to stay consistent) vs AP (stay available, reconcile later). With no partition, you can have both.

Strong consistency means every read reflects the latest committed write — required for balances, inventory, and unique-username checks. Eventual consistency lets replicas converge over time, tolerating brief staleness — fine for like counts, feeds, and DNS. Many systems offer a spectrum in between (read-your-writes, monotonic reads, causal consistency) so you pick per operation.

ACID (Atomicity, Consistency, Isolation, Durability) is the transactional guarantee of relational databases — correctness first. BASE (Basically Available, Soft state, Eventually consistent) is the looser posture many NoSQL systems take to maximize availability and scale. They're two ends of a trade-off spectrum, not a strict binary.

With N replicas, require W nodes to ack a write and R nodes to serve a read. If W + R > N, every read overlaps at least one node with the latest write, giving strong consistency. Tuning them trades latency for guarantees: W=1 is fast writes but weak reads; larger quorums are safer but slower and less available during failures.

Consensus algorithms like Raft and Paxos let a cluster agree on a single value or an ordered log despite failures, as long as a majority is alive. They underpin leader election, distributed locks, and strongly-consistent metadata stores (etcd, ZooKeeper, Consul). The cost is a majority round-trip per decision, so you keep consensus off the hot path and use it for coordination, not bulk data.

Networks retry, so the same request may arrive twice — an idempotent operation produces the same result whether applied once or many times, preventing double charges or duplicate orders. You achieve it with idempotency keys (dedupe on a client-supplied ID), conditional writes, or naturally idempotent operations (set-to-value rather than increment). It's what makes at-least-once delivery safe.

A queue decouples producers from consumers: it absorbs traffic spikes (buffering), lets each side scale and fail independently, and enables async work so a user request returns immediately while heavy processing happens later. It also smooths load and provides retries and dead-letter handling. The cost is added latency, eventual consistency, and operational complexity.

Kafka is a distributed, partitioned log: consumers track their own offset, messages are retained and replayable, and it excels at high-throughput streaming and event sourcing. RabbitMQ is a traditional broker with flexible routing (exchanges, bindings) and per-message acks, better for complex routing and task queues. Rough rule: Kafka for streams and replay, RabbitMQ for routed work distribution.

At-most-once may drop messages but never duplicates; at-least-once never drops but may duplicate (the common, practical default). True exactly-once is expensive and often an illusion — real systems approximate it with at-least-once delivery plus idempotent consumers or transactional dedupe. When asked, say exactly-once end-to-end is hard and idempotency is the pragmatic path.

In a point-to-point queue each message is consumed by exactly one worker — used to distribute tasks. In pub/sub each message fans out to every subscriber — used to broadcast events so many independent services react. Kafka blends both: consumers in the same group share partitions (queue-like), while different groups each get the full stream (pub/sub-like).

Backpressure is what happens when a consumer can't keep up with a producer. Handle it by bounding queues and applying flow control, shedding load (drop or reject low-priority work), scaling consumers, or buffering to durable storage. The failure to plan for it — unbounded in-memory queues — is a classic cause of cascading out-of-memory crashes.

A DLQ collects messages that repeatedly fail processing after a retry limit, so a poison message can't block the main queue forever. It isolates bad data for inspection, alerting, and manual or automated reprocessing — a small but telling detail interviewers like to see, because it shows you plan for failure, not just the happy path.

The go-to is the token bucket: tokens refill at a steady rate up to a burst capacity, and each request spends one or is rejected. Alternatives are the leaky bucket (smooths output) and sliding-window counters (accurate boundaries). In a fleet, keep the counters in a shared store like Redis (atomic increments or Lua scripts) so limits hold globally, not per server.

A circuit breaker wraps calls to a dependency and trips open after too many failures, failing fast (or serving a fallback) instead of piling requests onto a struggling service. After a cool-down it goes half-open to test recovery, then closes if calls succeed. It stops one failing dependency from cascading into a full outage.

Retry only idempotent or safely-deduped operations, use exponential backoff with jitter so clients don't retry in lockstep and create a retry storm, and cap total attempts. Combine with a circuit breaker and per-request timeouts. Naive tight-loop retries against an overloaded service turn a blip into an outage.

Rather than failing entirely when a dependency is down, the system drops to a reduced but useful mode — serve stale cache, hide a non-critical widget, disable personalization, or queue writes for later. Designing explicit fallbacks per feature keeps the core experience alive during partial outages, which users forgive far more than a blank error page.

Metrics are cheap numeric time series for dashboards and alerts (QPS, error rate, p99). Logs are detailed discrete events for debugging a specific incident. Traces follow one request across services to find where latency is spent. Together — the "three pillars of observability" — they answer is it broken, what broke, and where.

Remove any component whose failure takes down the system: run redundant instances behind a load balancer, replicate data across nodes and availability zones, use health checks with automatic failover, and eliminate shared bottlenecks (a lone leader, a single cache, one region). Then test it — chaos experiments and failover drills prove the redundancy actually works.

Generate a short key by base-62 encoding a unique ID (from a counter or key-generation service), or hashing the URL and handling collisions. Store the key → long URL mapping in a key-value store, front it with a cache since reads dominate writes massively, and serve redirects (301/302). It shards cleanly by key, and analytics can be captured asynchronously via a queue.

The core choice is fan-out on write (push each post into followers' precomputed feeds — fast reads, expensive for celebrities) vs fan-out on read (assemble the feed at request time — cheap writes, slow reads). Most systems use a hybrid: push for normal users, pull for accounts with huge followings, merged at read time — then cache aggressively.

Use persistent connections (WebSocket) via a gateway that tracks which server holds each user's connection. Messages route through that gateway, persist to a store partitioned by conversation, and deliver in order (per-conversation sequence numbers). Offline users get messages on reconnect; delivery/read receipts and presence ride the same channel. Scale by sharding conversations and using a pub/sub backbone between gateways.

Partition keys across nodes with consistent hashing (plus virtual nodes for balance), replicate hot partitions for availability, and pick an eviction policy (LRU) with TTLs. Clients hash keys to find the right node directly, avoiding a central bottleneck. Handle node failures by remapping only affected keys, and guard against stampedes with single-flight refresh.

Accept notification requests onto a queue, then have workers fan out per channel through provider adapters (APNs/FCM, an email service, an SMS gateway). Add user preferences and rate limiting to avoid spamming, template rendering, retries with dead-letter handling for provider failures, and idempotency keys so a retried request doesn't double-send. Track delivery status asynchronously.

Detect the skew, then spread the load: add a local cache in front of the hot key, replicate that key across nodes and read from any, or split the key (e.g. append a random suffix and aggregate). For hot writes like a global counter, shard the counter into N sub-counters and sum on read. The theme is: never let one key's traffic land on one node.

Options: a UUID (random, no coordination, but large and unsorted), a Snowflake-style ID (timestamp + machine ID + sequence — sortable and compact, the common choice), or a central ticket/range server that hands out blocks of IDs to each node. The trade-off is coordination vs sortability vs size; Snowflake IDs hit a sweet spot for most systems.