The big idea

What is an email system, systems-wise?

Email is unusual among the systems in this series: it's federated. Your mail server has to talk to millions of OTHER organizations' mail servers you don't control, over a protocol (SMTP) with no guaranteed delivery, no built-in spam prevention, and no shared trust — while still giving your own users a fast, reliable, always-in-sync inbox.

We'll build both sides: the inbound path (accept mail from an untrusted internet, filter it, store it) and the outbound path (send reliably to servers that may reject, delay, or vanish), plus what makes an inbox actually usable at scale — sharded storage, search, threading, and sync across devices.

How to read this: Each step opens with a real design decision — make the call before I show you what ships. Watch the diagram grow, hover the boxes, and at the end kill the spam filter and flood a phishing campaign to see what fails safe and what absorbs the attack. Hit Begin.

Step 1 · The baseline

Accept mail, store it

Simplest version: accept any incoming SMTP connection, take the message, and write it straight into the recipient's mailbox. What's obviously missing?

Design decision: Mail arrives and is written directly into the mailbox with no intermediate checks. What's the glaring gap?

The call: There's no spam or sender-authentication filtering at all — anyone on the internet can put anything directly into a user's inbox, including spam, phishing, and spoofed sender addresses. — Without any filtering between "accept the SMTP connection" and "it's in the inbox," every inbox is wide open to spam, phishing, and spoofing at internet scale. This is the single most important thing missing from the naive version.

The glaring gap is trust: SMTP alone provides no sender verification and no content screening, so a direct accept-and-store pipeline puts every inbox one connection away from spam, phishing, and spoofed senders. Everything received from the open internet needs to pass through filtering before it's considered "delivered."

The internet is an untrusted input: Any system that accepts input from arbitrary external parties — which describes inbound email about as purely as anything — has to treat that input as untrusted by default and validate/filter before it reaches anything durable or user-facing.

Step 2 · Sending is unreliable — plan for it

Queue and retry, don't send-and-forget

A user sends an email. The recipient's mail server is temporarily down, or slow, or greylisting the message (a common anti-spam tactic: reject once, accept on retry). Should the send simply fail if the first attempt doesn't succeed?

Design decision: The first delivery attempt to a recipient's mail server fails or times out. What should happen?

The call: Queue the message durably and retry with exponential backoff over a bounded period (often up to several days), only reporting a hard failure — and generating a bounce message back to the sender — once genuinely permanent failure is confirmed or the retry window is exhausted. — Because delivery across the open internet to servers you don't control is inherently unreliable, a durable queue with backoff-retry absorbs the common transient failures automatically. Only after real, sustained failure (or an explicit permanent rejection like "mailbox doesn't exist") does the system give up and notify the original sender via a bounce.

Queue outbound mail durably and retry with exponential backoff over a bounded window — this absorbs the common, transient delivery failures (temporary unavailability, greylisting) automatically. Only once delivery is genuinely, permanently impossible (or the retry window expires) does the sender get notified, via a bounce message.

Design for an unreliable network you don't control: Unlike most systems in this series where every component is under one organization's control, email's outbound path genuinely depends on the availability and behavior of servers run by other organizations entirely — durable queuing and backoff-retry are the standard answer to "this dependency will sometimes, unpredictably, not be immediately reachable."

Step 3 · Layered filtering

Cheap sender checks before expensive content analysis

Every accepted message needs to be screened for spam and phishing before it reaches a mailbox. Should every message run through the same, presumably expensive, ML content-classification model?

Design decision: Screening every inbound message for spam/phishing — run the expensive ML model on all of it?

The call: Layer cheap, deterministic sender-authentication checks first (SPF, DKIM, DMARC — is this sender actually authorized to send as this domain?), and reserve the expensive ML content-analysis model for messages that pass those and still need a judgment call. — SPF/DKIM/DMARC verify sender authenticity essentially for free and catch a large fraction of spoofed and bulk-spam traffic instantly. Only the messages that pass basic authentication — genuinely ambiguous cases where content actually needs to be judged — reach the costly ML stage, keeping overall throughput far higher.

Layer the filter: SPF/DKIM/DMARC sender-authentication checks run first — cheap, deterministic, and effective at rejecting a large share of spoofed/bulk traffic instantly. Only messages that pass those checks proceed to ML-based content analysis, which is reserved for the genuinely ambiguous cases that actually need judgment.

The same cheap-before-expensive pattern, a third domain: This is the identical principle from the adaptive tutor's hybrid grader and the code review bot's static-analysis-first pipeline, applied to email: reach for a deterministic, near-free check first, and reserve the expensive probabilistic model for what actually requires it.

Step 4 · Shard the mailbox by user

Every query is "my mailbox," never a cross-user join

Billions of users, each with thousands of messages. How should the mailbox store be partitioned so that reading "my inbox" stays fast at any scale?

Design decision: How should mailbox storage be partitioned for a system with billions of users?

The call: Shard by user id — every user's mail lives together on a small number of partitions, since essentially every real query is "give me MY messages," never a query spanning multiple users. — Email access is almost perfectly partition-friendly: a user only ever reads their own mailbox. Sharding by user id means any single user's query touches exactly one (or a small, known set of) partition, and the system scales by adding more partitions as the user base grows — no query ever needs to fan out across the whole dataset.

Shard the Mailbox Store by user id. Because virtually every real access pattern is "show me my own mail" — never a query spanning multiple users' mailboxes — user-id sharding means any single query touches only the partition(s) holding that one user's data, and the system scales horizontally simply by adding more partitions as the user base grows.

Shard by your actual access pattern: The right partition key is whatever dimension your real queries actually filter on. Email's "always exactly one user's data" access pattern makes user-id sharding almost embarrassingly effective — a much cleaner fit than, say, a geospatial or time-based key would be for THIS particular workload.

Step 5 · Search and threading

Group a conversation without trusting the subject line

A user wants to see an entire email conversation as one thread. Subject lines can collide ("Re: Meeting" exists in a thousand different, unrelated conversations) or get manually edited. How do you reliably group messages into the SAME conversation?

Design decision: How do you reliably group messages into one conversation thread, not just by matching subject text?

The call: Use the message's References and In-Reply-To headers — each reply explicitly links back to the Message-ID(s) it's replying to, forming a reliable causal chain that reconstructs the true thread structure regardless of subject text. — Every properly-formed reply email includes headers that explicitly reference the Message-ID of the message(s) it's replying to — this is a direct, protocol-level causal link, not a text-matching heuristic. Following these references reconstructs the actual conversation tree correctly even when subjects collide or get edited.

Reconstruct threads from the References and In-Reply-To headers — each reply explicitly links to the Message-ID(s) it's responding to, forming a reliable causal chain. Combine this with a per-user search index (full-text over subject/body, scoped to that user's shard) so both search and threading stay fast and accurate at scale.

Use protocol-level structure over text heuristics: Whenever a reliable, structural signal exists (Message-ID references, in this case), prefer it over a fuzzy text-matching heuristic (subject line similarity) — structural signals don't collide, don't drift, and don't need constant tuning the way heuristics do.

Step 6 · Never lose a message, never bounce forever

Delivery dedup and the bounce loop

A retry after a network blip might have actually succeeded the first time — risking a duplicate delivery. And a permanently-failing message generates a bounce back to the sender — but what if the bounce ITSELF can't be delivered?

Attach a unique Message-ID to every message and use it for delivery deduplication — the recipient system recognizes and discards a true duplicate rather than delivering it twice. For bounces specifically: a bounce message is marked with a special empty return-path (so it can never itself generate another bounce), which structurally prevents an undeliverable-bounce from looping forever between two mail servers.

Design the failure path so it can't recurse: A naive "on failure, send an error message" pattern is dangerous if the error message ITSELF can fail and trigger another error message — this is a classic infinite-loop trap. The fix (a bounce that structurally cannot generate a further bounce) is a general pattern for any failure-notification mechanism: make the failure path a dead end, not a cycle.

Step 7 · Multi-device sync

The same inbox, several devices, always agreeing

A user reads an email on their phone. It should show as read on their laptop too, without the laptop re-downloading the entire mailbox to notice.

Use an incremental sync protocol (the same principle IMAP uses): each device tracks a position/token representing what it has already seen, and asks for "what changed since then" — new messages, flag changes (read/unread, starred), folder moves. A device that reconnects after being offline for days still just requests one delta, not a full mailbox re-fetch.

The same sync-token pattern, a second domain: This is architecturally identical to the incremental sync design used for multi-device calendar sync — a durable, append-only change log plus a per-device cursor into it. Different data, same proven pattern for "keep N independent clients agreeing on one source of truth cheaply."

Step 8 · The sharp edges

Attachments, floods, and bounce loops

Three more real-world pressures: large binary attachments shouldn't live inline in structured mailbox storage, spam campaigns can flood the system, and the bounce-loop risk from Step 6 needs to actually hold under adversarial conditions.

Store attachments in object storage (not inline in the mailbox database), with the mailbox record holding just a reference — the same large-binary-blob pattern used everywhere structured storage and large files meet. Under a spam/phishing flood, the layered filter from Step 3 does the heavy lifting: cheap sender-authentication checks reject the bulk of it before the expensive ML stage is ever touched, keeping filter throughput high under attack. And the bounce-loop prevention from Step 6 — a bounce that structurally can't generate another bounce — holds regardless of how adversarial or malformed an incoming message is.

Design for the unhappy path: Spam filter down → fail closed, mail queues rather than bypassing filtering. Phishing flood → cheap checks absorb the bulk before the expensive stage. Large attachments → object storage, not inline. An email system that only works for small, well-behaved, cooperative traffic is a demo; one that stays correct and fast under adversarial internet-scale conditions is a product.

You did it

You just designed an email system.

• I — n
• O — u
• S — p
• M — a
• T — h
• M — e
• M — u
• A — t