The big idea

Why is this hard?

“Give me a unique ID” sounds trivial — until it must work across hundreds of machines, millions of times a second, with zero collisions, and ideally produce IDs that sort by time. A single counter can’t keep up, and randomness has its own costs.

Generate IDs locally on each machine with no per-ID coordination, by carving a 64-bit integer into fields — time, machine, sequence — that together guarantee uniqueness. This is Twitter’s Snowflake, and it’s mostly a packing problem.

How to read this: We add one piece at a time, problem then fix, and the diagram grows. Hit Begin.

Step 1 · The naive options

Local, no round-trip

A database auto-increment is one counter — a single point of failure and a write bottleneck the whole fleet must queue behind. Calling a central service per ID adds a network hop to every insert.

Design decision: You need unique IDs millions/sec across hundreds of machines. Where do you generate them?

The call: A local in-process library that mints 64-bit IDs. — Each service generates IDs locally with no network call — speed is the point. The rest is just deciding how to fill 64 bits so they never collide.

Make the generator local: a library inside each service that produces a 64-bit ID in-process, instantly, with no network call. The rest of the design is just deciding how to fill those 64 bits so they never collide.

Why 64 bits: A 64-bit integer is half the size of a UUID, sorts and indexes efficiently in databases, and is plenty of room to encode time, machine and a counter. Smaller IDs mean smaller, faster indexes.

Step 2 · Put time first

Timestamp in the high bits

Random IDs (like a UUIDv4) are unique but unordered — inserting them scatters writes all over a database index, hurting cache locality and range queries. You lose all sense of “newer” vs “older”.

Design decision: You want IDs that don’t scatter DB index writes and that sort by creation time. How?

The call: Put a millisecond timestamp in the high bits. — Later IDs are numerically larger, so they’re roughly time-sortable and new inserts append to the end of the index. 41 bits of ms covers ~69 years from a custom epoch.

Place a millisecond timestamp in the top bits. Now IDs minted later are numerically larger, so they’re roughly time-sortable and new inserts append to the end of the index. 41 bits of milliseconds covers ~69 years from a custom epoch.

k-sorted IDs: IDs aren’t perfectly ordered across machines (clocks differ slightly), but they’re “k-sorted” — close enough to creation order to sort feeds and range-scan by time without a separate timestamp column.

Step 3 · Separate the machines

A machine ID field

Two machines minting an ID in the same millisecond would produce the same timestamp bits — and collide. Coordinating between them on every ID would re-introduce the network hop we just removed.

Design decision: Two machines minting in the same millisecond share timestamp bits and collide. Fix without a network hop?

The call: Give each generator a unique machine-id field. — Same millisecond, different machine bits → different IDs. Each node owns its slice of every millisecond, so they physically can’t collide — coordination happens once at assignment, not per ID. 10 bits = 1,024 generators.

Give each generator a unique machine ID in the middle bits. Same millisecond, different machines, different IDs — uniqueness by construction, with no per-ID coordination. 10 bits allows 1,024 distinct generators.

Partition the ID space: Each machine owns its own slice of every millisecond. Disjoint slices mean two nodes physically cannot produce the same number — coordination happens once (at machine-id assignment), not per ID.

Step 4 · Many IDs per millisecond

The sequence counter

One machine often needs more than one ID per millisecond. Timestamp + machine bits alone would repeat for every ID minted within the same millisecond on the same node.

Design decision: One machine often needs many IDs in the same millisecond. Timestamp + machine bits would repeat. Fix?

The call: A per-millisecond sequence counter in the low bits. — A counter that increments per ID and resets each millisecond lets one machine mint up to 4,096 IDs/ms (12 bits, ~4M/sec). Only if it overflows do you wait for the next tick.

Add a sequence counter in the low bits that increments per ID and resets each millisecond. With 12 bits, a single machine can mint 4,096 IDs per millisecond (~4M/sec). If it overflows, wait for the next millisecond tick.

The full layout: [1 sign | 41 time | 10 machine | 12 seq] = 64 bits. Time keeps it sorted, machine keeps nodes disjoint, sequence handles bursts within a millisecond. That’s the whole trick.

Step 5 · Hand out machine IDs

Assign node bits safely

The whole scheme breaks if two generators are accidentally given the same machine ID — then they collide every millisecond. So who assigns those 1,024 ids, and how do you avoid duplicates as machines come and go?

Design decision: The scheme breaks if two generators get the same machine id. Who assigns the 1,024 ids?

The call: A registry leases a free machine id at startup (ZooKeeper/etcd/config). — Each generator claims a free id once on startup and holds it via a lease. That’s the only coordination in the whole system — after it, every node runs independently and collision-free.

Use an ID Registry: on startup each generator claims a free machine id from a coordination service (ZooKeeper/etcd) or static config, holding it via a lease. The only coordination in the entire system happens here, once per process.

Coordinate once, not per ID: Push all the coordination cost to startup. After a generator has its machine id, it runs fully independently — fast, local, and collision-free for as long as it holds its lease.

Step 6 · When the clock lies

Clock skew & rewind

IDs trust the wall clock — but NTP can step the clock backward. If time moves back, a machine could reuse timestamps it already used and mint duplicate IDs. The one assumption Snowflake makes is the one thing clocks violate.

Design decision: NTP can step the wall clock backward. What stops the generator minting duplicate IDs?

The call: Track the last timestamp; refuse to mint if time goes backward. — A clock guard remembers the last value used and stalls/refuses until the clock passes it — never issuing an ID for a time it already served. A tiny rare pause buys an ironclad no-duplicates guarantee.

A Clock Guard tracks the last timestamp used. If the clock goes backward, the generator refuses to mint (or briefly stalls) until the clock catches up past the last value — never issuing an ID for a time it has already served.

Monotonic, not just current: The generator cares about monotonically increasing time, not absolute time. Detecting rewind and waiting it out trades a tiny, rare pause for an ironclad no-duplicates guarantee.

Step 7 · Other ways

Alternatives & trade-offs

Snowflake gives unique, k-sorted IDs without coordination — but it leaks creation time (a privacy/competitive signal) and isn’t strictly monotonic across machines. Sometimes you need different properties.

Pick the tool for the job: a DB ticket server or range allocator (hand out blocks of ids per machine) gives strict ordering at the cost of a central component; UUIDv7 offers time-ordered randomness with no machine-id assignment. Snowflake wins when you want local, fast, sortable, and 64-bit.

No free lunch: Every scheme trades among coordination, ordering, size and information leakage. Knowing why Snowflake’s fields exist lets you reason about which alternative fits a given workload.

You did it

You just designed an ID generator.

• I — D
• A —
• A —
• A —
• A —
• A —
• T — i