Vibe Engines
YouTube

Vibe Engines is free, interactive AI engineering & system design — runs in your browser. Pick a learning path

System Design Fundamentals  /  Concurrency & Locking
Fundamental 02~13 min readIntermediate
Deep Dive

Two writers,
one truth. Locks and isolation levels keep it honest.

The instant two operations touch the same data at the same time, correctness is on the line — lost updates, dirty reads, double-spends. Concurrency control is the set of tools that decide which interleavings are allowed. Here's the full toolkit, from row locks to fencing tokens.

01

The races you're preventing

A race condition is when the result depends on the timing of operations that should have been independent. The classic is the lost update: two transactions read a balance of 100, each add 10, each write 110 — one update vanishes. Concurrency control exists to make these impossible.

AnomalyWhat happens
Dirty readYou read data another transaction wrote but hasn't committed — and it may roll back.
Non-repeatable readYou read the same row twice in one transaction and get different values.
Phantom readYou re-run a range query and new rows have appeared.
Lost updateTwo read-modify-write cycles overlap; one overwrites the other.
Write skewTwo transactions read overlapping data, then write disjoint rows that together break an invariant.
02

Pessimistic vs optimistic locking

The strategy you choose comes down to one question: how often do operations actually collide?

Two philosophies of conflict

Pessimistic — "assume conflict"

  • Lock before reading/writing
  • SELECT … FOR UPDATE
  • Others block and wait their turn
  • No wasted work, but risks deadlocks
  • Best for hot rows, high contention

Optimistic — "assume no conflict"

  • Read a version number, do the work
  • Compare-and-swap at commit time
  • No blocking; retry if version changed
  • Wasted work only when conflict is real
  • Best for read-heavy, low contention

Pessimistic locks come in shared (read) and exclusive (write) flavours; the danger is the deadlock — two transactions each holding a lock the other needs. Databases detect cycles and abort a victim, but you avoid them by acquiring locks in a consistent order. Optimistic concurrency avoids deadlocks entirely by never holding locks — at the price of a retry path you must actually write.

A deadlock is precisely a cycle in the wait-for graph, and consistent lock ordering makes the cycle impossible:

T1 holds A wants B · T2 holds B wants A  →  T1→T2→T1 = cycle  ·  everyone acquires A→B  →  T2 waits T1, no cycle

Whoever holds the lower-ordered lock always makes progress, so no one waits in a circle. The runnable version below detects the deadlock and shows the fix.

→ Interview tip

"How do you stop two people booking the same seat?" Name the trade-off out loud: a pessimistic row lock is simplest at low scale; an optimistic version check or a short-lived distributed lock scales better but needs a retry. The wrong answer is to never mention the race.

03

Isolation levels — the anomaly dial

SQL isolation levels are a contract: each one promises to prevent certain anomalies, in exchange for less concurrency. Higher is safer and slower.

Isolation levelDirty readNon-repeatablePhantom
Read UncommittedPossiblePossiblePossible
Read CommittedPreventedPossiblePossible
Repeatable ReadPreventedPreventedPossible*
SerializablePreventedPreventedPrevented

* In the ANSI standard. PostgreSQL's Repeatable Read (snapshot isolation) actually prevents phantoms but allows write skew; true Serializable closes that too.

04

MVCC — readers and writers stop fighting

Multi-Version Concurrency Control is how modern databases (PostgreSQL, MySQL InnoDB, Oracle) avoid the read-write blocking that naïve locking causes. Instead of overwriting a row, a write creates a new version tagged with the transaction that made it. Each transaction reads the version that was valid as of its start — a consistent snapshot.

Snapshot isolation via versions
Txn A starts
(sees v1)
→
Txn B writes
v2
→
Txn A reads
still v1
→
A commits
no block
Readers never block writers and vice-versa; old versions are garbage-collected once no transaction needs them.

The result: readers don't block writers, writers don't block readers. The cost is version bloat (the VACUUM problem in Postgres) and the subtle write-skew anomaly that only true serializable isolation removes.

05

What your database actually gives you

The isolation level names are standard. What each database does under those names is not.

Vendors built the ANSI levels on very different machinery, so two databases can both say "Repeatable Read" and disagree about whether a lost update can happen. Know your defaults:

DatabaseDefaultWhat the levels really are
PostgreSQLRead CommittedRead Committed takes a fresh snapshot per statement. Repeatable Read is snapshot isolation: updating a row that another transaction changed after your snapshot fails with a serialization error. Serializable is SSI. Read Uncommitted behaves as Read Committed.
MySQL InnoDBRepeatable ReadPlain SELECTs read a snapshot fixed at the first read. Locking reads, UPDATE and DELETE read the latest rows and take next-key locks (the row plus the gap before it), which blocks phantom inserts. Lost updates between a plain read and a later write go undetected. Serializable turns plain SELECTs into locking reads when autocommit is off.
OracleRead CommittedIts "Serializable" is snapshot isolation, so write skew is possible; write conflicts raise ORA-08177.
SQL ServerRead CommittedUses shared locks unless the READ_COMMITTED_SNAPSHOT option switches it to row versions. SNAPSHOT is snapshot isolation; SERIALIZABLE takes key-range locks.
CockroachDBSerializableWeaker levels are opt-in.

Two consequences. Code that is correct on one database's default can be wrong on another's, so set the isolation level explicitly in the code that depends on it. And of these five, only CockroachDB defaults to Serializable: if you have never chosen a level, write skew is probably possible in your system today.

06

Write skew, worked through

Write skew is the anomaly snapshot isolation lets through, and the textbook case is an on-call rota. The rule: at least one doctor stays on call for every shift. Alice and Bob are both on call for shift 1234, both feel ill, and both click "go off call" at the same moment. Each click runs this under PostgreSQL's Repeatable Read:

BEGIN ISOLATION LEVEL REPEATABLE READ;
SELECT count(*) FROM doctors
 WHERE shift_id = 1234 AND on_call;          -- both transactions see 2
-- app: 2 on call, so someone else still covers
UPDATE doctors SET on_call = false
 WHERE shift_id = 1234 AND name = 'alice';   -- Bob's transaction updates 'bob'
COMMIT;                                       -- both commit: nobody is on call

No row was written twice, so snapshot isolation has no conflict to detect. Each transaction read a consistent snapshot and made a sound decision, and together they broke the rule. The shape is always read, decide, then write a different row from the ones the decision depended on. Double-booked meeting rooms and two signups claiming one username are the same bug.

FixWhat happens to Alice and BobCost
SERIALIZABLE on PostgreSQL (SSI)Neither blocks. SSI notices that each read data the other then changed, and aborts one with SQLSTATE 40001. The retry sees one doctor on call and refuses.Aborts under contention; every caller needs a retry loop
SERIALIZABLE on MySQL or SQL Server (two-phase locking)Each SELECT takes shared range locks, so each UPDATE waits for the other: a deadlock, and the database aborts one (MySQL error 1213). The retry refuses.Blocking, plus deadlocks to retry
Lock what the decision readSELECT id FROM doctors WHERE shift_id = 1234 AND on_call FOR UPDATE, then count the rows. At Read Committed, Bob's SELECT waits for Alice's commit, then finds only his own row.Can't lock a row that doesn't exist yet

That last limit matters for "check that nothing exists, then insert" rules such as room bookings: there is no row to lock. Run those transactions as Serializable, or materialize the conflict by pre-creating one row per room and time slot to lock, or let a unique or exclusion constraint enforce the rule. Whatever runs at Serializable needs a retry that re-runs the whole transaction, reads included, because the decision was made on data that changed:

import random, time

RETRY = {"40001", "40P01"}   # serialization_failure, deadlock_detected

def with_retry(run_txn, attempts=5):
    for attempt in range(attempts):
        try:
            return run_txn()          # BEGIN ISOLATION LEVEL SERIALIZABLE ... COMMIT
        except DatabaseError as e:    # e.g. psycopg.Error, which carries .sqlstate
            if e.sqlstate not in RETRY or attempt == attempts - 1:
                raise
            time.sleep(random.uniform(0, 0.02 * 2 ** attempt))   # jittered backoff

Keep side effects such as emails out of the transaction, and send them after the commit succeeds.

07

Four patterns that hold up under load

-- 1. Atomic conditional update: no read-modify-write in the app at all
UPDATE inventory SET stock = stock - 1
 WHERE sku = 'A-42' AND stock > 0;        -- 1 row: reserved; 0 rows: sold out

-- 2. Optimistic: a version column checked in the same statement
UPDATE docs SET body = $1, version = version + 1
 WHERE id = $2 AND version = $3;          -- 0 rows: someone saved first; reload

-- 3. Pessimistic, but fail fast instead of queueing behind the holder
SELECT * FROM seats WHERE id = 17 FOR UPDATE NOWAIT;

-- 4. A job queue: each worker claims different rows and nobody waits
SELECT id, payload FROM jobs
 WHERE status = 'queued'
 ORDER BY id
 LIMIT 10
 FOR UPDATE SKIP LOCKED;

Reach for the atomic update first: the database reads and writes under one row lock, so two buyers of the last item can't both win, and the affected-row count says who did. The version column is optimistic locking in one statement, and HTTP APIs expose the same check as If-Match (see API design). NOWAIT turns a held lock into an immediate error, which suits a UI that should say "someone else is editing" rather than hang.

SKIP LOCKED (PostgreSQL 9.5+, MySQL 8.0+, Oracle; SQL Server's equivalent is the READPAST hint) turns a table into a work queue: each worker skips rows another worker holds, and a crashed worker's rows unlock when its transaction rolls back. For long jobs, don't hold the transaction open. Claim rows by setting a status and a lease expiry, commit, and let a sweeper requeue expired leases.

08

Distributed locks & fencing tokens

When the contended resource lives across services, you reach for a distributed lock (Redis, ZooKeeper, etcd). These are far trickier than in-process locks, because a holder can pause — a long GC, a network stall — past the lock's expiry, then wake up still believing it holds the lock while someone else has taken it.

The fix is a fencing token: a monotonically increasing number issued with each lock grant. The protected resource records the highest token it has seen and rejects any write carrying a lower one — so a stale holder's writes bounce, even if its lock looks valid.

Fencing token rejects the stale holder
Client 1
token 33
→
pauses (GC)
→
Client 2
token 34
→
Storage: 33 < 34
reject Client 1

Two details decide whether fencing works. The token must come from something that never issues the same number twice, even across failovers, and consensus stores provide one: etcd stamps every write with a cluster-wide revision that only grows, and ZooKeeper does the same with its transaction ID (zxid). And the resource must check the token atomically with the write:

UPDATE files SET body = $1, fence = $2
 WHERE name = $3 AND fence <= $2;   -- 0 rows: a newer lock holder exists, so stop

A lock service that can't produce such a token can't fence, which is one of the two main objections in Martin Kleppmann's 2016 critique of Redis's Redlock algorithm. A resource that can't check tokens, such as a third-party API with no version precondition, can't be fenced either; there, rely on idempotency keys.

→ Key insight

A lock with a timeout is a lease, not a guarantee. Without a fencing token, any distributed lock can be held by two clients at once. Idempotency (see consensus & coordination) is the other half of staying correct under retries.

RUN IT YOURSELF

Detect a deadlock, then order it away

A deadlock is a cycle in the wait-for graph — nothing more. This simulates two transactions grabbing locks A and B in opposite orders: each ends up holding one and waiting for the other, and a cycle-detector finds the circular wait. Then it reruns them with a single rule — everyone acquires locks in the same global order — and the cycle vanishes: whoever gets the lower lock finishes, the other simply queues behind it. Same locks, same transactions, no deadlock. Change the lock sets or the order.

CPython · WebAssembly
Trade-offs

Cost and trade-offs of each technique

Here Time is what a caller waits for, and Space is what the technique holds or stores while it works.

CaseTimeSpaceWhy
Atomic conditional UPDATEOne statement; the row lock lasts only that statementNothing extraCheapest correct option, but only when the whole decision fits in one SQL statement.
SELECT … FOR UPDATEWaits behind the holder for its whole transactionA row lock per locked rowA hot row serializes: if each transaction holds it 20 ms, that row tops out at 50 writes a second.
Optimistic version columnNo waiting; a loser redoes its workOne integer per rowWith k writers racing for one row, one wins each round and k−1 retry. Great when rare, wasteful when hot.
Serializable via SSI (PostgreSQL)No blocking; some commits abort with 40001Tracks what each transaction readCorrect by default, paid for in retries that grow with contention.
Serializable via 2PL (MySQL, SQL Server)Blocks on shared range locksRange locks for every readReaders block writers again, and deadlocks become routine.
MVCC snapshotsReads never wait for writesOld row versions until cleanupVersion bloat, and write skew stays possible below Serializable.
Distributed lock + fencingA network round trip to acquire, plus the leaseLock state in a consensus storeUse it only when the resource spans services; the token check must sit in the resource.

The order to try them in: one atomic statement, then a version check, then a row lock, then Serializable with retries. Reach for a distributed lock only when no single database owns the data.

Watch for

Common mistakes

  • Doing the arithmetic in the app

    SELECT stock, subtract one in code, then UPDATE … SET stock = 41 is a lost update waiting to happen at Read Committed, and at MySQL’s Repeatable Read too. Write stock = stock - 1 with a WHERE stock > 0 guard so the database does the read and the write under one lock.

  • A mutex that only one process can see

    A threading.Lock or synchronized block protects memory inside one process. Run four replicas of the service and you have four independent locks around the same row. Shared data needs a lock the database or a lock service owns.

  • A network call inside the transaction

    Calling a payment API or waiting for a user while holding FOR UPDATE stretches a millisecond lock into seconds, and every other writer to that row queues behind it. Do the slow call before or after; inside the transaction, only read and write.

  • Locking reads without an index

    InnoDB locks every index record a locking read or UPDATE scans, not just the rows it returns. With no usable index on the WHERE column, it scans the whole table, and every row ends up locked. Check the plan of any statement that takes locks.

  • Leaving transactions open

    A session that is “idle in transaction” keeps its snapshot alive, so PostgreSQL’s VACUUM cannot remove versions newer than it, and tables bloat. It may also hold locks. Set idle_in_transaction_session_timeout, and make sure every code path commits or rolls back.

In the wild

Where this shows up in production

  • Bank transfers

    A transfer locks two account rows. If one request locks A then B and another locks B then A, you get the deadlock from the runnable example. The standard fix is to always lock the lower account id first, whichever direction the money moves.

  • ORM optimistic locking

    Rails adds a check on a lock_version column automatically and raises StaleObjectError when it fails. JPA does the same with a @Version field and OptimisticLockException. Both run the version-column pattern from this page; you still have to catch the error and retry or tell the user.

  • Compare-and-swap in memory

    Java’s AtomicInteger.compareAndSet and Go’s sync/atomic are optimistic concurrency at CPU level: read, compute, swap only if the value is unchanged, else loop. Lock-free queues and counters are built on exactly that loop.

  • Kubernetes leader election

    Controllers that must run as a single active copy, such as the scheduler, hold a Lease object in the API server and renew it before it expires. That is a lease, not a guarantee, so the same rule applies: work that must never run twice needs fencing or idempotency on top.

Self-check

Check yourself

Balance is 50. Two requests run UPDATE accounts SET balance = balance - 30 WHERE id = 1 AND balance >= 30 at the same moment, at PostgreSQL Read Committed. What happens?

The first takes the row lock and updates it to 20. The second waits. When the first commits, PostgreSQL re-checks the WHERE clause against the new row version: 20 < 30, so the second updates 0 rows. The final balance is 20 and only one withdrawal happened. At Repeatable Read, the second would instead fail with a serialization error and need a retry.

A report runs two SELECTs in one transaction at Read Committed: total of checking, then total of savings. A transfer moves 100 between them in between. Can the grand total be wrong?

Yes, by 100. Read Committed takes a fresh snapshot per statement, so the two SELECTs see different moments: the money can be counted twice or not at all. This is a non-repeatable read across rows. Run the report at Repeatable Read, where the whole transaction reads one snapshot, or compute both totals in a single statement.

A "likes" counter on one viral post gets hundreds of updates a second. Optimistic version check or pessimistic row lock?

Neither is the best choice. With a version check, almost every writer loses the race and retries. A row lock queues them all behind each other. Use an atomic UPDATE posts SET likes = likes + 1, which needs no read in the app. At higher rates, split the count across several rows and sum them when you read it.

Client A holds lock token 41, pauses, and its lease expires. B gets token 42 and writes. A wakes and writes with token 41. What does the fenced UPDATE do, and why not just ask the lock service "am I still the holder?" before writing?

The WHERE fence <= 41 check fails because the row already carries 42, so A’s write updates 0 rows and A stops. Asking first does not help: A can pause again between the check and the write, which is the same race moved a few lines down. Only a check done atomically with the write closes it.

Frequently asked

Quick answers

Optimistic vs pessimistic locking?

Pessimistic locks before touching data (best for high contention). Optimistic takes no lock, checks a version at commit, and retries on conflict (best for read-heavy, low-contention workloads).

What are the four SQL isolation levels?

Read Uncommitted, Read Committed, Repeatable Read, and Serializable — each preventing more anomalies (dirty reads → non-repeatable reads → phantoms) at the cost of concurrency.

What is MVCC?

Multi-Version Concurrency Control keeps multiple row versions so readers see a consistent snapshot without blocking writers. Each transaction reads the version valid as of its start.

Why are distributed locks dangerous?

A holder can pause past the lock's expiry and resume while another process holds it. A fencing token — a monotonic number checked by downstream storage — rejects the stale writer.

What is write skew?

Write skew is when two transactions read the same data, each makes a decision based on it, and each then updates a different row, so neither overwrites the other but together they break an invariant. The classic case is two on-call doctors who both go off call because each saw the other still on. Snapshot isolation allows it; Serializable isolation prevents it, and so does locking the rows the decision read with SELECT ... FOR UPDATE.

What isolation level do PostgreSQL and MySQL use by default?

PostgreSQL defaults to Read Committed, taking a fresh snapshot for every statement. MySQL's InnoDB defaults to Repeatable Read: plain reads use one snapshot for the whole transaction, while locking reads and writes take next-key locks. Neither default prevents write skew.

Finished this one? 0 / 209 Handbooks done
NextConsensus, Transactions & Coordination Next on the “Distributed systems, from first principles” ladder · rung 5 of 7

Explore the topic

See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.

More Handbooks