The whole design, in writing
Every step of the build above, written out: the problem each piece solves, the option that was taken and the ones that were not, the numbers, and how it fails in production.
The big idea
What is a collaborative editor?
Two people open the same document and type at the same time. Each sees their own edits instantly and the other’s appear live, and somehow both end up looking at the exact same text. No “file is locked”, no overwriting each other’s work.
Stop syncing whole documents and start syncing tiny operations — “insert ‘x’ at position 5”. A server orders everyone’s ops and resolves conflicts so every copy converges to the same state. Get that one idea right and the rest is plumbing.
What the new pieces do
- Editor Aclient
- One person’s browser. It applies edits locally for instant feedback, then sends them to the server — it can’t wait for a round-trip per keystroke.
Step 1 · The model
Edits are operations
If a save means “upload the whole document”, two people’s saves clobber each other and the last one wins. You can’t merge intent from snapshots — by then the information about what changed is gone.
Two people edit the same doc simultaneously. What does the client send on each change?
Whole-document saves clobber each other — last write wins — and the information about WHAT changed is gone, so you can’t merge intent. Concurrent editing needs the changes themselves.
Coarse diffs against a stale base still race and lose intent when positions shift, and they’re expensive to merge. You want the fine-grained edits as they happen.
Model the doc as an ordered sequence and stream small ops applied locally for instant feedback. Ops preserve intent and are cheap to send, order and merge — the op stream is the source of truth.
Model the doc as an ordered sequence and every edit as a small op (insert/delete at a position). The editor applies ops locally for instant feedback and streams them to an Edit Server over a persistent connection.
Why this piece earns its place
The knob this introduces is op granularity, and one op per keystroke is not the obvious answer it looks like. Coalescing a run of typed characters into a single op — flushed on a short timer, and immediately on a caret move, a paste, or a delete — cuts the message count enormously and still feels instant. Widen that window too far and the cost lands in three places at once: peers watch you type in lumps, the remote cursor drawn for you jumps instead of moving, and whatever is sitting unflushed is the only work in the system that nobody has written down yet. Make it too narrow and every editor emits a message per keypress, which everything downstream then multiplies. The other half of granularity is what each op carries. A delete has to record what it removed, not just where — otherwise you cannot invert it, and undo, along with anything else that walks the history backwards, stops being something the op stream can give you.
What the new pieces do
- Edit Serverbackend
- Holds a persistent connection to every editor of a doc. It receives edits, orders them, resolves conflicts, and relays the result to everyone.
- Document Modelservice
- The doc as a sequence of characters/objects with stable positions. Edits are tiny ops — insert at p, delete at p — not whole-document saves.
Step 2 · The hard part
Resolving concurrent edits
A and B both edit from the same starting text. A inserts at position 2; B deletes at position 1. By the time A’s op reaches B, B’s positions have shifted — apply A’s op blindly and the docs diverge. Forever.
A inserts at position 2; B deletes at position 1. By the time A’s op reaches B, positions have shifted. How do they stay identical?
Locking kills real-time collaboration — it’s back to "file is locked," the exact thing you’re replacing. Concurrent edits must be allowed and reconciled, not serialized.
OT rewrites an incoming op against ops it didn’t see (shift A’s "insert at 2" past B’s delete); CRDTs give each char a stable orderable id so edits merge commutatively. Both guarantee convergence — same ops, any order, same doc.
Applying ops blindly when their positions were computed against different states makes the documents diverge permanently. Concurrent ops must be transformed/merged, not applied as-is.
Use Operational Transformation (OT) or CRDTs. OT rewrites an incoming op against ops it didn’t see (shift A’s “insert at 2” to account for B’s delete). CRDTs give each character a stable, orderable ID so concurrent edits merge commutatively — no transform needed.
Why this piece earns its place
The honest caveat is that OT’s correctness lives in a pile of transform functions, and their failure is silent. Plain inserts and deletes are the easy pairs; the ones that break are a bold span and a delete crossing it, a paste landing inside a table cell, two deletes that overlap. Get one pair subtly wrong and nothing throws — the two screens simply hold different text from that moment on, and somebody finds out when they reload. Two defenses. First, keep a single server as the ordering authority, which is what this design is already doing, so an op only ever has to be transformed against one linear history instead of against an arbitrary mesh of concurrent orderings; that second property is the genuinely hard one, and centralizing the order is how you avoid needing it. Second, don’t trust the transforms: have each client periodically send a cheap checksum of its text with the version it is at, compare server-side, and force a client that disagrees to resynchronize from the server’s copy. CRDTs swap this bill for another — deleted characters leave tombstones the structure has to keep carrying.
- convergentend state
- intentpreserved
- any ordersame result
What the new pieces do
- OT / CRDTservice
- The conflict-resolution core. It rewrites concurrent edits so that applying them in any order yields the same document on every screen.
Back of the envelope
- same ops, any order ⇒ same doc
- the convergence guarantee
- OT: transform vs unseen ops
- small ops, the server orders them
- CRDT: stable per-char ids
- commutative merge, no central transform
Step 3 · Make it live
Broadcast to everyone
A’s edit is resolved on the server — but B is still staring at stale text. Collaboration only feels real if the other person’s changes show up in well under a second.
A’s edit is resolved on the server, but B still sees stale text. How does B’s screen update live?
Polling is laggy and wasteful — collaboration needs sub-second updates, and most polls return nothing. The server should push the moment an op is accepted.
Re-downloading the doc is heavy, loses B’s cursor/scroll, and still lags. You want just the new ops pushed incrementally, not full reloads.
After accepting an op the server pushes it to every other editor’s open socket; when editors are on different servers, pub/sub relays it to whoever holds each connection. Real-time = push, not poll.
After accepting an op, the server broadcasts it to every other editor of the doc over their WebSockets. When editors sit on different servers, a pub/sub layer relays the op to whichever server holds each connection.
Why this piece earns its place
Broadcast is where the work stops being linear in what you type. One op becomes a message for every other editor of that doc, so a busy room costs the product of editors and edits, and the relay hop pays it again for every server holding one of those connections. Two things keep that affordable. Per connection, gather whatever ops are pending and write them as one frame on a short tick rather than a socket frame per op — the peer applies them in the same order either way, and that tick is latency you are choosing to spend, so keep it far under the sub-second promise. Then decide in advance what happens to a connection that cannot keep up: a phone on a bad network, a laptop that was suspended mid-session. The tempting answer is to buffer for it, and it is the wrong one, because unbounded per-connection queues turn one struggling client into server memory pressure that everybody in the document feels. Drop it, close the socket, and let it come back and catch up — nothing it missed is gone.
What the new pieces do
- Editor Bclient
- A second person editing the same document at the same time. The whole challenge is making A’s and B’s concurrent edits converge to one consistent doc.
- Op Broadcastbus
- Pushes each accepted op to all other editors of the doc in real time. Across servers, a pub/sub layer relays ops to whoever holds the other connections.
Step 4 · Never lose a keystroke
Persist: op log + snapshots
The live document lives in server memory. A crash, and hours of collaborative work vanish. But you also can’t rebuild a long doc by replaying millions of ops on every open.
The live doc is in server memory and a long doc has millions of ops. How do you avoid losing work AND loading fast?
The append-only op log is the truth (nothing lost on a crash); periodic snapshots let you load the latest checkpoint and replay only ops since — fast recovery, full history, free version control.
Rewriting the whole doc per keystroke is enormous write amplification and still loses concurrent intent. Append the small op, and materialize full state only occasionally.
Memory-only loses everything on a crash, and replaying millions of ops from scratch on every open is far too slow. You need durability (log) plus checkpoints (snapshots).
Append every accepted op to a durable Op Log, and periodically write a Snapshot of the full state. To load a doc, take the latest snapshot and replay only the ops since — fast recovery, full history, and free version control.
Why this piece earns its place
Two decisions hide inside “append it to the log”. The first is when you tell the client its op is safe. Acknowledge on receipt and you have opened a window in which the editor shows a character as saved that a crash would erase — precisely the thing this step exists to prevent — so the ack has to follow the durable write, not the receipt. Paying a disk round trip per keystroke would be miserable, so group them: ops arriving within a few milliseconds are written and acked together, which is one flush amortized across a whole room instead of one per keypress. The second is snapshot cadence, and it is a straight trade. Snapshot rarely and opening a busy document means replaying a long tail of ops while someone waits at a blank page. Snapshot constantly and you are back to rewriting the full document over and over, which is the write amplification you just rejected. Set it from replay cost rather than from a clock — checkpoint after a number of ops, build it off the serving path, and write it from a version you already hold rather than pausing the editors to take it.
What the new pieces do
- Op Logstore
- An append-only history of every operation. Replaying it rebuilds the document and powers undo, version history, and catching up after disconnects.
- Snapshotsstore
- Periodic materialized copies of the doc so you don’t replay millions of ops from scratch — load the latest snapshot, then apply ops since.
Back of the envelope
- append op = the truth
- nothing lost on a crash
- snapshot = checkpoint of the log
- load latest + replay ops since
- log ⇒ undo, history, catch-up
- free version control
Step 5 · Who else is here?
Presence & cursors
Collaboration feels lonely — and people collide — if you can’t see where others are. Those colored cursors and name flags are core to the experience, but they’re high-frequency and disposable.
Colored cursors and "3 people viewing" update constantly and are disposable. How do you handle them?
Persisting a flood of high-frequency cursor updates would swamp the durable log with data nobody replays. Presence is throwaway — it shouldn’t touch document storage.
Mixing disposable cursor traffic into the durable op pipeline risks the document during a presence storm. Keep ephemeral signals on a separate path.
Track who’s connected and each cursor/selection, broadcast constantly, but never store it — if everyone leaves, presence evaporates while the doc remains. A cursor flood can’t risk the op log.
Track Presence separately from the document: who’s connected, and each person’s cursor/selection. Broadcast these constantly but never persist them — if everyone leaves, presence simply evaporates while the doc remains.
Why this piece earns its place
Presence quietly assumes a connection tells you whether a person is there, and half-open connections are exactly where that stops being true. A lid closes, a tunnel swallows a socket, and the server still holds a connection with nobody on the far end — so the room keeps showing a cursor for someone who left, and it stays until something finally tears that connection down. Which means presence entries should expire on their own rather than waiting to be deleted: the client heartbeats, each entry carries a short lifetime, and anything that stops being refreshed disappears by itself. Key those entries per connection rather than per person, too, or the same user with the document open in two tabs overwrites their own cursor and it flickers between two places. The freeing part of this path is that it is the one stream you are allowed to lose. A cursor update is pure latest-value state, so the server can throttle it to roughly a frame rate and discard everything in between — doing that to an op would be data loss. Presence being disposable isn’t only a storage decision; it’s what lets you treat it as cheap when the system is under pressure.
What the new pieces do
- Presenceservice
- Tracks who’s viewing, where their cursor and selection are, and broadcasts those colored carets — ephemeral state that never touches durable storage.
Step 6 · The flaky network
Offline & reconnection
Connections drop in tunnels and on planes. A user keeps typing offline, then reconnects — meanwhile others changed the doc. Their queued edits must merge cleanly, not overwrite or get lost.
A user edits offline in a tunnel, then reconnects — others changed the doc meanwhile. How do the edits merge?
On reconnect the client sends queued ops and pulls everything from the op log after its last-seen version; OT/CRDT merges the two streams. Offline editing is just real-time editing with a longer delay — same convergence machinery.
Overwriting discards everyone else’s changes made while the user was offline. Concurrent edits must merge, not clobber — exactly what OT/CRDT exists to do.
Forbidding offline edits breaks the experience on planes and in tunnels, and isn’t necessary — buffered ops plus version-based catch-up merge cleanly on reconnect.
Each client tracks the last version it saw. On reconnect it sends its buffered ops and pulls everything from the Op Log since that version; OT/CRDT merges the two streams. The same convergence machinery that powers live editing powers catch-up.
Why this piece earns its place
Catch-up assumes the server can still produce every op after the client’s version, and that assumption expires. Op history isn’t retained at full resolution forever — that is what the snapshots are for — so a client that has been away long enough sits before the oldest op you still keep, and no amount of replay will reach it. Detect that case explicitly and hand it the latest snapshot plus the ops after that, rather than letting a request for ancient versions return an empty range that looks exactly like “nothing happened while you were gone”. The buffered side has its own edge. Those queued ops live in a browser tab, so they need writing to local storage as they are made, or the offline work survives the tunnel and not the closed lid. And bound the queue. Someone returning after days of solo editing is asking you to transform a very long buffer against an equally long stream, and what comes out of merging two documents that have drifted that far is rarely what either party wanted. Past that bound, stop calling it a merge: resynchronize from the server’s current state and keep their local work as a separate recoverable copy they can look at and decide about.
Back of the envelope
- client tracks last version N
- sync = "give me ops after N"
- buffered ops ⇄ pulled ops
- OT/CRDT merges both streams
- offline = real-time + delay
- one mechanism for both
Step 7 · Permissions & scale
The sharp edges
Not everyone may edit — some can only view or comment — and a viewer must not be able to sneak in an op. And a server can only hold so many live docs and connections before it tips over.
Check Permissions on join and on every op, so the server rejects edits from non-editors. Scale by sharding by document: all editors of one doc land on the same server (consistent hashing), keeping its in-memory state authoritative; pub/sub bridges across shards.
Why this piece earns its place
Pinning a document to one server buys simple ordering and pays for it with a handoff problem. That server will be redeployed, will lose its network, will pause long enough to look dead — and something has to let another server take the document over. Ownership as a lease from a coordination service is the usual shape, but the lease alone does not make it safe: the old owner can be paused past its expiry, wake up still believing it owns the doc, and append ops it ordered against a version the new owner has already moved past. The append is what has to be conditional. Every write to the Op Log carries the version it expects to extend and the ownership term it was made under, so a stale owner’s write is rejected by the log itself rather than trusted because a timer said so. The log arbitrates; the lease only decides who tries. Permissions have a smaller version of the same problem — a check on join is a check made once, and access gets revoked while the socket is still open. Re-check per op against state the server holds, and push the revocation to that live connection instead of waiting for the person to reconnect.
What the new pieces do
- Permissionsstore
- Who may open, comment on, or edit a doc. Checked when joining and on every op, so a viewer’s stray edit is rejected at the server.
You did it
You just designed Google Docs.
Everything you assembled, in order
- Edits are tiny operations on an ordered model, not whole-document saves.
- OT or CRDTs resolve concurrent edits so every copy converges.
- Accepted ops broadcast over WebSockets (and pub/sub) for live collaboration.
- A durable op log plus periodic snapshots give recovery, history and fast loads.
- Presence (cursors, who’s here) runs on a separate, ephemeral path.
- Versioned op logs make offline editing and reconnection sync just work.
- Server-side permissions and per-document sharding handle security and scale.