The big idea
Why is this hard?
“Give me a unique ID” sounds trivial — until it must work across hundreds of machines, millions of times a second, with zero collisions, and ideally produce IDs that sort by time. A single counter can’t keep up, and randomness has its own costs.
Generate IDs locally on each machine with no per-ID coordination, by carving a 64-bit integer into fields — time, machine, sequence — that together guarantee uniqueness. This is Twitter’s Snowflake, and it’s mostly a packing problem.
How to read this: We add one piece at a time, problem then fix, and the diagram grows. Hit Begin.
Step 1 · The naive options
Local, no round-trip
A database auto-increment is one counter — a single point of failure and a write bottleneck the whole fleet must queue behind. Calling a central service per ID adds a network hop to every insert.
Design decision: You need unique IDs millions/sec across hundreds of machines. Where do you generate them?
The call: A local in-process library that mints 64-bit IDs. — Each service generates IDs locally with no network call — speed is the point. The rest is just deciding how to fill 64 bits so they never collide.
Make the generator local: a library inside each service that produces a 64-bit ID in-process, instantly, with no network call. The rest of the design is just deciding how to fill those 64 bits so they never collide.
Why 64 bits: A 64-bit integer is half the size of a UUID, sorts and indexes efficiently in databases, and is plenty of room to encode time, machine and a counter. Smaller IDs mean smaller, faster indexes.
Step 2 · Put time first
Timestamp in the high bits
Random IDs (like a UUIDv4) are unique but unordered — inserting them scatters writes all over a database index, hurting cache locality and range queries. You lose all sense of “newer” vs “older”.
Design decision: You want IDs that don’t scatter DB index writes and that sort by creation time. How?
The call: Put a millisecond timestamp in the high bits. — Later IDs are numerically larger, so they’re roughly time-sortable and new inserts append to the end of the index. 41 bits of ms covers ~69 years from a custom epoch.
Place a millisecond timestamp in the top bits. Now IDs minted later are numerically larger, so they’re roughly time-sortable and new inserts append to the end of the index. 41 bits of milliseconds covers ~69 years from a custom epoch.
k-sorted IDs: IDs aren’t perfectly ordered across machines (clocks differ slightly), but they’re “k-sorted” — close enough to creation order to sort feeds and range-scan by time without a separate timestamp column.
Step 3 · Separate the machines
A machine ID field
Two machines minting an ID in the same millisecond would produce the same timestamp bits — and collide. Coordinating between them on every ID would re-introduce the network hop we just removed.
Design decision: Two machines minting in the same millisecond share timestamp bits and collide. Fix without a network hop?
The call: Give each generator a unique machine-id field. — Same millisecond, different machine bits → different IDs. Each node owns its slice of every millisecond, so they physically can’t collide — coordination happens once at assignment, not per ID. 10 bits = 1,024 generators.
Give each generator a unique machine ID in the middle bits. Same millisecond, different machines, different IDs — uniqueness by construction, with no per-ID coordination. 10 bits allows 1,024 distinct generators.
Partition the ID space: Each machine owns its own slice of every millisecond. Disjoint slices mean two nodes physically cannot produce the same number — coordination happens once (at machine-id assignment), not per ID.
Step 4 · Many IDs per millisecond
The sequence counter
One machine often needs more than one ID per millisecond. Timestamp + machine bits alone would repeat for every ID minted within the same millisecond on the same node.
Design decision: One machine often needs many IDs in the same millisecond. Timestamp + machine bits would repeat. Fix?
The call: A per-millisecond sequence counter in the low bits. — A counter that increments per ID and resets each millisecond lets one machine mint up to 4,096 IDs/ms (12 bits, ~4M/sec). Only if it overflows do you wait for the next tick.
Add a sequence counter in the low bits that increments per ID and resets each millisecond. With 12 bits, a single machine can mint 4,096 IDs per millisecond (~4M/sec). If it overflows, wait for the next millisecond tick.
The full layout: [1 sign | 41 time | 10 machine | 12 seq] = 64 bits. Time keeps it sorted, machine keeps nodes disjoint, sequence handles bursts within a millisecond. That’s the whole trick.
Step 5 · Hand out machine IDs
Assign node bits safely
The whole scheme breaks if two generators are accidentally given the same machine ID — then they collide every millisecond. So who assigns those 1,024 ids, and how do you avoid duplicates as machines come and go?
Design decision: The scheme breaks if two generators get the same machine id. Who assigns the 1,024 ids?
The call: A registry leases a free machine id at startup (ZooKeeper/etcd/config). — Each generator claims a free id once on startup and holds it via a lease. That’s the only coordination in the whole system — after it, every node runs independently and collision-free.
Use an ID Registry: on startup each generator claims a free machine id from a coordination service (ZooKeeper/etcd) or static config, holding it via a lease. The only coordination in the entire system happens here, once per process.
Coordinate once, not per ID: Push all the coordination cost to startup. After a generator has its machine id, it runs fully independently — fast, local, and collision-free for as long as it holds its lease.
Step 6 · When the clock lies
Clock skew & rewind
IDs trust the wall clock — but NTP can step the clock backward. If time moves back, a machine could reuse timestamps it already used and mint duplicate IDs. The one assumption Snowflake makes is the one thing clocks violate.
Design decision: NTP can step the wall clock backward. What stops the generator minting duplicate IDs?
The call: Track the last timestamp; refuse to mint if time goes backward. — A clock guard remembers the last value used and stalls/refuses until the clock passes it — never issuing an ID for a time it already served. A tiny rare pause buys an ironclad no-duplicates guarantee.
A Clock Guard tracks the last timestamp used. If the clock goes backward, the generator refuses to mint (or briefly stalls) until the clock catches up past the last value — never issuing an ID for a time it has already served.
Monotonic, not just current: The generator cares about monotonically increasing time, not absolute time. Detecting rewind and waiting it out trades a tiny, rare pause for an ironclad no-duplicates guarantee.
Step 7 · Other ways
Alternatives & trade-offs
Snowflake gives unique, k-sorted IDs without coordination — but it leaks creation time (a privacy/competitive signal) and isn’t strictly monotonic across machines. Sometimes you need different properties.
Pick the tool for the job: a DB ticket server or range allocator (hand out blocks of ids per machine) gives strict ordering at the cost of a central component; UUIDv7 offers time-ordered randomness with no machine-id assignment. Snowflake wins when you want local, fast, sortable, and 64-bit.
No free lunch: Every scheme trades among coordination, ordering, size and information leakage. Knowing why Snowflake’s fields exist lets you reason about which alternative fits a given workload.
You did it
You just designed an ID generator.
- IDs are generated locally, in-process, with no per-ID network round-trip.
- A timestamp in the high bits makes 64-bit IDs roughly time-sortable.
- A machine-id field keeps same-millisecond IDs on different nodes disjoint.
- A per-millisecond sequence counter handles bursts (4,096 IDs/ms/node).
- A registry assigns unique machine ids once at startup — the only coordination.
- A clock guard refuses to mint when the wall clock rewinds, avoiding duplicates.
- Ticket servers, range allocators and UUIDv7 are alternatives with other trade-offs.