Engineering~13 min readIntermediate
Deep Dive
Postgres internals: UPDATE never updates, and a janitor keeps it working.
The world's default database has a strange secret at its core: it never modifies data in place. Updates write copies, deletes delete nothing, and a background janitor cleans up the corpses. From that one design choice — MVCC — flows almost everything operators fight with and everything that makes Postgres brilliant. Here's the machine, layer by layer.
01
The heap: tables are bags of 8KB pages
A table is a heap — a file of fixed 8KB pages, each holding row versions (tuples) plus a small item-pointer directory. Rows have no inherent order; a row's physical address (the ctid: page number + slot) is where indexes point. Values too big for a page get sliced into a side table by TOAST — which is why a 5MB JSON blob doesn't wreck the page geometry of its neighbors.
Hold onto the page as the unit of thought: I/O happens in pages, the buffer cache holds pages, and the planner prices queries in page reads. "How many pages will this touch?" is the question behind nearly every performance mystery below.
02
MVCC: every row is a row version
Concurrent readers and writers are the defining database problem. Locking readers against writers murders throughput; Postgres chose Multi-Version Concurrency Control instead: keep multiple versions of each row, let every transaction see its own consistent snapshot.
What DML actually does to the heap
INSERTwrites a tuple stamped xmin = my transaction ID ("born at").
UPDATEwrites a whole NEW tuple, stamps the old one xmax ("expired at"). Two versions now coexist.
DELETEjust stamps xmax. The tuple physically remains — invisible to later snapshots.
SELECTsees a tuple iff its birth committed before my snapshot and its expiry didn’t. Visibility is arithmetic on transaction IDs.
The prize: readers never block writers, writers never block readers — a long analytics query and a burst of updates coexist peacefully, each in its own snapshot. The price: expired versions — dead tuples — accumulate in the heap, and every index entry pointing at them lives on too. Someone has to clean up.
03
VACUUM: the janitor with two jobs
VACUUM walks tables reclaiming dead tuples so their space gets reused, and maintains the visibility map (which pages are all-visible — crucial for index-only scans, next section). Autovacuum triggers it per table when dead tuples cross thresholds. When churn outruns it, you get bloat: tables and indexes physically far larger than their live data, wasting cache, slowing scans — the classic silent Postgres degradation. Watch pg_stat_user_tables.n_dead_tup; tune autovacuum more aggressive, not less, on hot tables.
→ The scary one: wraparound
Transaction IDs are 32-bit and circular. VACUUM's second job is freezing old tuples so their age stops mattering. If autovacuum is disabled or perpetually starved, Postgres will eventually force an aggressive freeze — and in the worst case shut down to protect data. When a DBA looks pale and says "wraparound," this is the story.
04
Indexes: B-trees, heap trips, and the free lunch
The default index is a B-tree mapping key → ctid. But an index hit isn't the end: the tuple's visibility lives in the heap, so a normal index scan does index then heap — two page trips. The visibility map earns its keep here: if the target page is marked all-visible, Postgres can skip the heap trip entirely — an index-only scan, the fastest read Postgres has. (Well-vacuumed tables make them possible; bloated ones don't.)
| Index type | Reach for it when |
| B-tree | equality + ranges + ordering — 95% of indexes |
| GIN | things-inside-things: JSONB containment, arrays, full-text |
| GiST | geometry, ranges, nearest-neighbor |
| BRIN | huge naturally-ordered tables (timestamps) — tiny index, coarse pruning |
| Partial / expression | index only the rows or the transform you actually query |
MVCC's tax collects here too: every UPDATE creates index entries for the new version in every index — unless nothing indexed changed and the new tuple fits the same page (a HOT update). Fewer indexes and non-indexed churn columns = more HOT = less bloat.
05
WAL: one log, three superpowers
Writing dirty 8KB pages to random disk locations on every commit would be slow and torn-write fragile. Postgres instead appends every change to the write-ahead log — sequential, fsynced before commit returns. Data pages are updated lazily in memory and flushed at checkpoints.
One mechanism, three features
WAL stream
every change, in order
→
Crash recovery
replay since checkpoint
→
Replication
ship the stream to replicas
→
PITR
base backup + WAL to any second
Crash? Replay WAL from the last checkpoint — perfect recovery. Replicas? They're just servers replaying your WAL live (streaming replication), or logically decoding it (logical replication, CDC). Point-in-time recovery? A base backup plus archived WAL replayed to any chosen second. Durability, HA and backups: one log.
06
The planner: a cost model, not a mind-reader
SQL says what; the planner picks how: which indexes, which join algorithm (nested loop for tiny inner sides, hash join for big unsorted ones, merge join for pre-sorted inputs), which order. It prices candidate plans with statistics — histograms and distinct-counts per column, refreshed by ANALYZE — and picks the cheapest estimate.
Which is why every slow-query investigation is the same ritual: EXPLAIN ANALYZE, then compare estimated rows vs actual rows. Off by 1000×? The stats are stale or the column correlations exceed what histograms capture — and the planner chose a nested loop for a million rows. Seq scan on a "should be indexed" query? Often correct: past ~5-10% selectivity, reading the whole table sequentially beats a million random heap trips. The planner is rarely wrong given its inputs; fix the inputs.
07
Connections and memory: why the pooler is mandatory
Postgres forks an OS process per connection — robust isolation, expensive arithmetic: each backend carries megabytes, and creation isn't cheap. A serverless app opening 2,000 lambda connections meets reality fast. Hence PgBouncer (or a built-in pooler): thousands of client connections multiplexed onto a few dozen real backends, in transaction-pooling mode. Treat it as part of Postgres, not an optimization.
Memory in one line each: shared_buffers — Postgres's own page cache (classic advice: ~25% of RAM; the OS page cache handles the rest); work_mem — per sort/hash operation, not per connection, so big values multiply frighteningly under concurrency; maintenance_work_mem — feed VACUUM and index builds generously.
08
The machine in one table
| Layer | The one-line truth |
| Heap | 8KB pages of row versions; I/O and caching think in pages |
| MVCC | no in-place writes; snapshots via xmin/xmax arithmetic |
| VACUUM | reclaims dead tuples, feeds the visibility map, prevents wraparound |
| Indexes | B-tree → ctid; index-only scans need a clean visibility map |
| WAL | sequential log = durability + replication + PITR |
| Planner | cost model on statistics — check estimates vs actuals |
| Connections | process per connection → pooler is infrastructure |
Frequently asked
Quick answers
What is MVCC?
No in-place changes: UPDATE writes a new version, DELETE marks. Transactions see snapshot-consistent versions via xmin/xmax — readers and writers never block each other.
Why VACUUM?
To reclaim dead tuples, maintain the visibility map (index-only scans) and freeze old transaction IDs against wraparound. Starve it and you get bloat.
What does the WAL buy?
Crash recovery, streaming replication and point-in-time recovery — all from one sequential, fsynced change log.
Why PgBouncer?
Process-per-connection makes direct high concurrency ruinously expensive; a pooler multiplexes thousands of clients onto dozens of backends.