The whole design, in writing
Every step of the build above, written out: the problem each piece solves, the option that was taken and the ones that were not, the numbers, and how it fails in production.
The big idea
How does Dropbox sync your files?
You drop a file in a folder on your laptop. Moments later it’s on your phone, your tablet, and a colleague’s machine — and if you tweak one paragraph, only that change travels, not the whole file again.
Sync is really two problems wearing one coat: moving bytes efficiently, and keeping a consistent view of the namespace (the file tree, names, versions) across devices that come and go. We’ll build those two halves and the machinery that keeps them agreeing.
What the new pieces do
- Laptopdevice
- A device where files change. A local client watches a folder and uploads only what's new, then keeps it in sync.
- Phonedevice
- Another device on the same account. Must learn about the laptop's change quickly and pull it down.
Step 1 · The skeleton
Files = metadata + bytes
A "file" is two things: where it lives and what it’s called (metadata), and the actual content. Conflating them makes both sync and storage harder.
A "file" is its name/path (metadata) plus the actual bytes (content). How do you model it for sync?
Conflating the tiny mutable metadata with the giant immutable bytes makes both sync and storage harder: every rename rewrites bytes, and you can’t sync "what changed" cheaply. Split them.
A Metadata DB owns folders, names and versions; content goes elsewhere. Devices sync the small metadata first to learn exactly what changed, then fetch only the bytes they’re missing. The file tree, not the files, is kept consistent.
Device-local trees can’t agree across machines that come and go — you need an authoritative server-side namespace to sync against. The metadata is the shared source of truth.
Split them. A Metadata Service (backed by a Metadata DB) owns the folder tree, names, and versions. Content goes elsewhere. Uploading a file becomes: store the bytes, then record a row saying "this path, this version, is made of these bytes."
Why this piece earns its place
Splitting the file in two means one upload now writes to two stores with no transaction across them, so the ordering is a design decision, not a detail. Bytes go first and the metadata row commits second: until that row lands the file does not exist, and a device that dies mid-upload has merely left blocks nobody points at. Do it the other way and the tree advertises a file whose bytes were never stored — a download that fails on something the user can plainly see in their folder. The cost of that ordering is garbage, because every abandoned upload leaves blocks behind. You cannot reclaim them by deleting whatever the last version dropped, either: the same bytes may be referenced by an older version you still owe the user, or by a completely different account. Collection has to run from the metadata side — reference counts, or a sweep over live block lists — and it has to treat a block that is being written right now as live, or it will cheerfully delete an upload that is halfway through.
What the new pieces do
- App Serveredge
- The front door for every device. Routes metadata operations and coordinates uploads and downloads.
- Metadata Serviceservice
- Owns the logical view: folders, file names, versions, and which blocks make up each file. The brain of sync.
- Metadata DBstore
- Durable record of the file tree, versions, and block lists. The source of truth for what the namespace looks like.
Step 2 · Chunk & dedup
Don’t move the whole file
Re-uploading a 2 GB file because you changed one slide is absurd — and storing the same attachment a thousand users share a thousand times is wasteful.
Re-uploading a 2 GB file for a one-slide change is absurd, and storing a shared attachment 1000× is wasteful. Fix?
Compression shrinks one upload but you still re-send the entire file on any change, and identical files from different users are stored separately. You need to move and store only the parts that are actually new.
Per-user whole-file copies miss global dedup (the same block shared across users) and still move whole files. Go finer-grained: content-addressed blocks.
A file becomes an ordered list of block hashes; a block named by its content hash is stored once globally, so if its hash already exists you skip the upload. Identical data — across files and users — is stored a single time.
The Block Service splits every file into fixed-size blocks (say 4 MB), hashes each, and stores blocks in a content-addressed Block Store. A file’s metadata becomes an ordered list of block hashes. If a block’s hash already exists, you skip the upload entirely — deduplication.
Why this piece earns its place
Block size is the knob this step introduces, and 4 MB is a compromise rather than a constant. Push it up and each block is cheaper to track but blunter to reuse: change one byte and you re-send the whole block, and two files have to agree over a much longer run of bytes before they dedup at all, so the win evaporates exactly where near-identical files should have paid off. Push it down and reuse gets sharper, but a 2 GB file becomes an enormous list of hashes that the metadata row has to carry, sync and version — the metadata you split out precisely to keep small — and every block is one more lookup into a global index keyed by hash, which means no locality at all: consecutive blocks of one file land on unrelated machines. Set it from measured file and edit sizes, not taste. And the hash must be cryptographic, because here the hash is the name: if two different blocks can be made to collide, one account can overwrite what another account’s tree points at.
- 4 MBblock size
- hash= address
- store oncededup
What the new pieces do
- Block Serviceservice
- Splits files into fixed-size blocks, hashes each one, and stores only blocks it hasn't seen before.
- Block Storestore
- Object storage holding the actual file bytes as deduplicated, content-addressed blocks.
Back of the envelope
- file = ordered list of block hashes
- 4 MB content-addressed blocks
- hash already exists ⇒ skip upload
- identical data stored once, globally
- shared attachment ×1000
- one stored copy
Step 3 · Delta sync
Send only what changed
Even within one file, you usually touch a tiny fraction. Sending every block on every save wastes bandwidth and battery.
Even within one file you usually touch a tiny fraction. How do you avoid sending every block on every save?
Because a file is a list of content-addressed blocks, "what changed" is a set difference of hashes — delta sync. The network only ever carries genuinely new bytes; metadata records a new version pointing mostly at existing blocks.
Labeling it small doesn’t reduce what’s sent — you still move every block. The win comes from computing the block-level diff and transferring only the changed blocks.
A text/line diff doesn’t generalize to binary files (images, videos, zips) and is fragile across formats. Block-hash set difference works for any file type uniformly.
On a change, the client re-chunks the file, compares the new block hashes against the old list, and uploads only the new blocks. The metadata service records a new version pointing mostly at existing blocks plus a few new ones. Downloads work the same in reverse — fetch only the blocks you don’t already have.
Why this piece earns its place
This step rests on an assumption the previous one made quietly: that block boundaries stay put. With fixed-size blocks they only stay put for overwrites. Insert a paragraph at the top of a document and every boundary after it slides along, so every hash downstream of the edit is new and the set difference degenerates into the entire file — delta sync silently becomes a re-upload, and the user watches a one-word change cost what the original save cost. The fix is to stop cutting at fixed offsets and let the bytes choose the cuts: run a rolling hash across the file and end a block wherever that hash hits an agreed pattern, so a boundary is a property of the data around it and an insertion disturbs only the block it landed in. That is what production sync engines do, and it is worth its price — the client must read and hash the file to compute the diff at all, so delta sync is really local I/O spent on every save to avoid spending bandwidth.
- ~few %of file moves
- set diffof hashes
- versionedmetadata
Back of the envelope
- re-chunk ⇒ set-diff the hashes
- only new blocks travel
- ~few % of a file moves per save
- the rest already exist
- new version = mostly old blocks
- + a few new ones
Step 4 · Multi-device
Tell the other devices, now
The laptop uploaded a change — but the phone has no idea. Polling "anything new?" every few seconds is laggy and hammers your servers with millions of empty checks.
The laptop uploaded a change but the phone has no idea. How do other devices learn quickly without hammering your servers?
Millions of devices polling is laggy (you wait a poll interval) and floods your servers with empty checks. You want the server to tell devices the instant something changes.
Wrong channel and far too slow/noisy for file sync — and it doesn’t drive an automatic pull. Devices need a lightweight real-time nudge, not a notification to a human.
On commit, the metadata service emits a sync event; the notifier pushes "something changed" to affected online devices, which pull the new metadata and missing blocks. Push (light) then pull (heavy) — and via the event stream the writer never needs to know who’s online.
Add a Notification Service that holds a long-lived connection (or long-poll) to every online device. When the metadata service commits a change, it emits a sync event; the notifier pushes a nudge to the affected devices, which then pull the new metadata and missing blocks.
Why this piece earns its place
A push is a hint, not a delivery guarantee, and treating it as one is what keeps it cheap. Sockets drop, a nudge is lost, a laptop sleeps mid-event — so every client keeps a slow periodic poll underneath the connection as a floor, and the push only decides whether sync is instant or one poll interval late. That also makes the notifier restartable, which matters because its interesting failure is not going down, it is coming back: every device on every account reconnects in the same second and each immediately pulls metadata, so the metadata service takes its worst read spike at the moment it is least ready for one. Randomised reconnect backoff is half the defense; keeping the nudge payload-free is the other half, because a device that receives ten of them in a burst collapses them into a single pull. Fan-out is the number to watch here — one write into a shared folder wakes every member’s devices, so this tier scales with how many people subscribe, not with how much actually changed.
What the new pieces do
- Notification Svcservice
- Holds long-lived connections to every device and tells them the instant something they care about changes.
- Sync Eventsbus
- A stream of change events. Lets the metadata service announce updates without knowing which devices are online.
Step 5 · Serve downloads
Popular files, everywhere
When a shared file goes viral inside a company, thousands of devices pull the same blocks from the origin block store at once — slow for far-away users and heavy on the origin.
A shared file goes viral — thousands of devices pull the same blocks from origin at once. How do you serve downloads?
A block named by its hash can never change, so it caches anywhere with zero invalidation. Downloads fetch hot blocks from the nearest edge; origin only serves misses and the long tail — downloads scale almost for free.
Replicas add origin throughput but distant users still pay the long round-trip, and you re-serve the same hot block repeatedly. Cache it at the edge near users instead.
Throttling punishes legitimate use to mask a scaling gap. Immutable blocks are the perfect cache target — push them to a CDN rather than slowing everyone down.
Put a CDN / cache in front of the block store. Blocks are immutable and content-addressed — a perfect cache target. Downloads fetch hot blocks from the nearest edge; the origin only serves cache misses and the long tail.
Why this piece earns its place
Content addressing is what makes the edge nearly free, and it is also what makes the edge dangerous: a block’s name is its content, so whoever holds the hash can ask for the bytes. Permissions live in the metadata tree, and the CDN cannot read that tree — it sees an opaque key and a request. So authorisation has to happen before the fetch rather than during it: the app server checks the share on the metadata side and hands back a short-lived signed URL, while the edge caches on the block key alone and ignores the signature, so two people who are both entitled to the same block still share one cached copy. Keep the expiry short, because an issued URL cannot be recalled — revoking someone from a folder does not un-issue the link they already hold, and the signature’s lifetime is the only bound on that window. Encrypting blocks on the client before they leave changes the shape again: the edge then caches ciphertext, which dedups fine inside one key scope and not at all across accounts.
What the new pieces do
- CDN / Cacheedge
- Caches frequently downloaded blocks near users so popular files don't pull from the origin every time.
Step 6 · Scale the metadata
Billions of files
Block storage scales easily, but the metadata — every file, folder, and version for hundreds of millions of users — outgrows a single database.
Block storage scales easily, but metadata for 100s of millions of users outgrows one DB. How do you partition it?
Random per-file sharding scatters one user’s tree across many shards, so listing a folder or syncing becomes a cross-shard fan-out. Keep an account’s files together.
Caching helps reads but the authoritative metadata still outgrows one machine’s write capacity and storage. You must partition the data itself.
A user’s files are accessed together and rarely cross into another user’s tree, so user id is a clean shard key: even load, operations local to one shard, no cross-shard transactions for everyday ops. Read replicas handle listing-heavy load.
Shard the metadata DB by user (or workspace), so all of one account’s tree lives together and operations stay local to a shard. Add read replicas for listing-heavy workloads. The block store, already object storage, scales horizontally on its own.
Why this piece earns its place
Sharding by user assumes a tree belongs to one person, and the sharing feature one step from here says it does not. The moment a folder is shared, one subtree is reachable from several users’ trees, and those users hash to different shards — so either the folder is duplicated per member and every write fans out across shards, or it lives in one place and a listing crosses shards after all. The cleaner shape is to shard by namespace instead of by person: a folder tree is the unit, an account is a set of mounted namespaces, and the shard key follows the namespace so a shared folder stays whole. The second thing to get right is the mapping itself. Hashing an id modulo the shard count turns adding capacity into a rebalance of everything, and at billions of files that is not an operation anyone gets to run. Keep an explicit lookup table from namespace to shard, so a move is one row and you can drain a hot shard — the one workspace far larger than any normal account — a namespace at a time.
- 100s of Musers
- billionsfiles
- shardby user
Back of the envelope
- shard metadata by user id
- a tree lives on one shard
- everyday ops = single-shard
- no cross-shard transactions
- + read replicas
- for listing-heavy load
Step 7 · The sharp edges
Conflicts, offline & sharing
Two devices edit the same file while one is offline. Whose version wins? And how do permissions work when a folder is shared with a team?
Track versions and detect divergence; rather than silently losing data, keep both as a conflicted copy and let the user reconcile. Queue changes made offline and replay them on reconnect. Layer sharing and permissions onto the metadata tree so access is checked on every operation.
Why this piece earns its place
A conflicted copy answers the easy half. Two devices edit the same bytes, keep both, done — but a device returning after a week offline is replaying namespace operations too, and a rename, a move or a delete has no second version to keep. Someone deleted the folder your queued edit belongs in; someone renamed the parent you are moving a child into; two devices moved directories into each other and replaying both honestly produces a cycle. So the offline queue cannot be a bag of file writes: each operation carries the version it was made against, so the server can tell a clean replay from a real divergence, and namespace operations are applied server-side in an order the server picks against the tree as it exists now. Where the outcome is genuinely ambiguous, choose the recoverable mistake — resurrect the file inside the deleted folder rather than let the delete win, because a user can throw away a copy they did not want and cannot recover bytes you dropped. And watch the blast radius: a conflicted copy inside a shared folder syncs to everyone in it.
You did it
You just designed Dropbox.
Everything you assembled, in order
- Split metadata (the file tree) from content (the bytes).
- Chunk files into content-addressed blocks and deduplicate.
- Delta sync moves only the blocks that actually changed.
- A notification service pushes change events; devices then pull.
- A CDN caches immutable hot blocks for fast downloads everywhere.
- Shard the metadata DB by user to scale to billions of files.
- Versioning, conflict copies, offline queues and sharing for real life.