LEARNING PATH · Systems & Backend

Reliability & Scale (SRE)

For engineers who keep systems up as traffic grows.

Advanced ~2h 15m5 lessons8 steps

The reliability engineer’s toolkit: turn nines into real downtime, size capacity before it bites, then study the patterns — quorums, rate limiting, caching, queues — that absorb load and keep services alive under pressure.

Translate availability targets into real downtime budgets
Estimate capacity and headroom before launch
Tune quorum and consistency trade-offs deliberately
Apply rate limiting, caching and queues to shed and absorb load

0 / 5 done · 0%

How this path works

This path is for engineers who carry the pager or want to: SREs, platform and infrastructure engineers, and backend developers who have become the person who keeps the service up. It covers the design side of reliability, meaning the decisions that set how a system behaves before anything goes wrong, rather than on-call process or incident management.

Reliability work, and the design rounds of an SRE interview, test whether you can reason with numbers. What does 99.95 percent availability mean in minutes per month? How much capacity does a launch need if traffic peaks at three times the average? How many replicas can fail before writes stop? And when load arrives anyway, which component sheds it, which absorbs it, and which falls over first?

The steps go in that order. After the fundamentals, two calculators turn availability targets and traffic into downtime budgets and node counts. Partitioning and the quorum explorer show how redundancy buys back uptime and what it costs in consistency. The last three designs are the load-handling patterns themselves: a rate limiter that rejects work early, a cache that absorbs reads, and a queue that turns a spike into a backlog the system can drain at its own pace.

What each step gives you

The System Design Fundamentals Handbook
The shared vocabulary, read with a reliability lens: what CAP says your system does during a partition, why stateless services are the ones you can scale and replace freely, and how replication and consensus keep data alive when machines die. It prepares you to answer "what happens if this node disappears?" for every box in a diagram, which is the habit the rest of the path depends on.
Availability (Nines) Calculator
Chain dependencies in series and in parallel, then read off end-to-end availability, the number of nines and the yearly and monthly downtime. It trains the arithmetic behind availability targets: three nines is roughly 8.8 hours of downtime a year, each extra nine is about ten times harder, serial dependencies multiply, and parallel replicas multiply their unavailabilities instead, which is why redundancy on the critical path pays.
Interactive Capacity Estimator
Turn daily users, request rates, payload sizes and a peak-to-average factor into peak QPS, storage per year with replication, bandwidth and node counts, with every formula visible. Use it before a launch review: size for the spike rather than the average, see how a higher peak factor or replication setting changes the node count, and practice stating the headroom you would provision and why.
Partitioning, Sharding & Replication
How data is split and copied so that losing one node is an inconvenience rather than an outage: partitioning schemes, consistent hashing with virtual nodes, hot partitions caused by a bad shard key, replication topologies and quorums. It prepares you to spot the reliability risks in a data layer, such as one hot shard that saturates first or a rebalance that moves far more data than expected.
Quorum (N/R/W) Explorer
Dial N, R and W and see how many failures reads and writes each survive, whether reads are guaranteed fresh, and what the setting does to latency. For reliability work the key readings are N minus W and N minus R: practice choosing settings for a store that must keep accepting writes when a replica is lost, and explaining what consistency you give up to get there.
Design a Rate Limiter
Load shedding designed properly: rejecting work at the edge before it costs anything, choosing whose traffic to limit, token buckets versus windows, one correct counter across many gateways, regional trade-offs, and failing open or closed when the limiter's own store is down. It prepares you for the reliability question hidden inside every limiter design, which is what happens to your users when the protection itself fails.
Design a Distributed Cache
A cache is a reliability component as much as a speed one, and this design covers the failure side: replication and failover, invalidation, hot keys, and surviving a thundering herd by coalescing misses. Practice explaining what happens to the database behind it after a cold start or a mass flush, since a cache that quietly carries most of your read load is also a single point of failure.
Design a Message Queue
The pattern for absorbing spikes: producers append to a durable log, consumers drain it at their own pace, and partitions, replication with in-sync replicas, retention and delivery guarantees decide what survives a broker failure. It prepares you to reason about consumer lag, slow consumer groups and replays, and to explain why at-least-once delivery with idempotent consumers is usually the practical target.

A study plan

Week 1
Read System Design Fundamentals, then model a real service in the Availability Calculator: every synchronous dependency in series, every replica set in parallel. Compare the result with the target you have promised or been promised, and write down the one dependency that costs you the most nines.
Week 2
Use the Capacity Estimator to size that same service for its busiest day, then read Partitioning, Sharding and Replication and find where its data layer would hit a hot partition first. Finish in the Quorum Explorer by choosing N, R and W for one store and justifying the choice in three sentences.
Week 3
Work through the rate limiter, the distributed cache and the message queue. For each, write a short failure table: what fails, what users see, and what sheds or absorbs the load. Then answer each page's interviewer questions about failure modes out loud.

Questions about this path

How do you calculate downtime from an availability percentage?

Multiply the unavailable fraction by the length of the period. At 99.9 percent the unavailable fraction is 0.001, so a year of about 8,760 hours allows roughly 8.8 hours of downtime, or about 43 minutes a month. At 99.99 percent it drops to about 52 minutes a year, and at 99.999 percent to about 5 minutes. For a whole system, multiply the availabilities of dependencies that are all required, and for redundant replicas multiply their unavailabilities instead. The Availability Calculator in this path does both.

What should I study for an SRE interview?

SRE loops usually mix several areas: Linux and networking troubleshooting, coding or scripting, system design with a reliability focus, and questions about monitoring, incident response and on-call practice. This path covers the design and quantitative part, meaning availability math, capacity planning, replication and quorums, and the patterns that shed or absorb load. Pair it with hands-on troubleshooting practice and a few prepared stories about incidents you handled, including what you changed afterward, since those conversations are hard to improvise.

How much capacity headroom should a service have?

There is no universal number, but the method is consistent. Estimate average load, multiply by a realistic peak-to-average factor for your traffic, then add room to lose a node or a zone without overloading the rest. If you run several instances and want to survive one failure, the remaining instances must carry the peak on their own. Revisit the estimate before launches and seasonal events, and confirm it with load tests, because a back-of-the-envelope figure is a starting point rather than a measurement.

Why do rate limiters, caches and queues matter for reliability?

They are the three standard ways to keep a service alive when demand exceeds capacity. A rate limiter sheds excess work at the edge and shares capacity fairly between clients. A cache absorbs repeated reads so the database sees only a fraction of the traffic. A queue turns a sudden spike into a backlog that consumers drain at a steady rate. Each brings its own failure modes, such as a limiter whose store goes down or a cache flush that stampedes the database, which is why this path studies all three as complete designs.

Keeping systems up at scale

Reliability engineering is the discipline of designing for failure instead of hoping to avoid it. This path starts from the fundamentals, then gets quantitative with the tools that turn "should be fine" into numbers — availability composition, capacity estimation, quorum trade-offs — before applying them to the systems where reliability is the whole point: rate limiters, caches and queues.

The mindset it builds: every dependency is a tax on uptime, redundancy is how you buy it back, and you can't operate what you don't measure. That's what separates a system that survives its first traffic spike from one that doesn't.

← All learning paths

The System Design Fundamentals Handbook

Availability (Nines) Calculator

Interactive Capacity Estimator

Partitioning, Sharding & Replication

Quorum (N/R/W) Explorer

Design a Rate Limiter

Design a Distributed Cache

Design a Message Queue