Handbooks  /  The SQL Handbook
Handbook~16 min readIntermediate
Deep Dive

SQL, joins,
and the index that saves you.

Anyone can write a SELECT. The difference between "it works" and "it works at scale" is understanding what the database does underneath: how joins combine tables, how an index turns a scan into a lookup, and how transactions keep concurrent chaos correct. Here's the mental model.

01

The relational model

A relational database organizes data into tables — rows (records) and columns (fields) with a fixed schema. Two ideas make it "relational." A primary key uniquely identifies each row (a user's id). A foreign key in one table references the primary key of another, encoding a relationship (an orders row has a user_id pointing at users).

This lets you store each fact once and link records instead of duplicating them: a user's name lives in one place, and a thousand orders reference it by id. Relationships come in shapes — one-to-many (a user has many orders), many-to-many (students and courses, via a join table). The whole power of SQL is querying across these relationships, which is what joins are for.

02

Joins — combining tables

A join combines rows from two tables by matching a condition, usually a foreign key equal to a primary key. The join type decides what happens to rows that don't match.

JoinReturnsUse when
INNEROnly rows with a match in both tablesYou want records that have related data on both sides
LEFTAll left rows + matches (NULLs where none)Keep all left rows even without a match (users with no orders)
RIGHTAll right rows + matchesSame, mirrored (rarely used; flip the tables and LEFT)
FULLAll rows from both, matched where possibleEverything from both sides, gaps as NULL

The most common bug is expecting a LEFT JOIN but writing an INNER JOIN and silently dropping rows that have no match — a user with zero orders vanishes from the report. Pick the type by asking: "do I want to keep rows that have no partner on the other side?"

→ Watch the fan-out

Joining a one-to-many relationship multiplies rows: one user with 5 orders becomes 5 rows. That's expected — but it means COUNT and SUM after a join can double-count. Aggregate carefully, and know your cardinalities.

03

Indexes — the single biggest lever

Without an index, finding rows where email = ... means a full table scan: read every row and check. On a million rows, that's slow. An index is a separate, sorted data structure — almost always a B-tree — that maps a column's values to the rows containing them, so the database can jump straight to matches in O(log N) instead of O(N).

Why a B-tree? It keeps keys sorted and balanced, so lookups, range scans (BETWEEN, >), and ordered reads (ORDER BY) are all fast. But indexes aren't free: they take space, and every INSERT/UPDATE/DELETE must also update the index — so indexes speed reads and slow writes. Index the columns you filter, join, and sort on; don't index everything.

No index

  • Full scan: read all N rows
  • O(N) per lookup
  • Fine for tiny tables only

B-tree index

  • Sorted structure → jump to matches
  • O(log N) lookup + fast ranges/ORDER BY
  • Costs storage + slower writes
04

The query planner & EXPLAIN

You write what you want (declarative SQL); the database's query planner decides how to get it — which indexes to use, join order, and algorithm. It estimates the cost of each plan using table statistics and picks the cheapest. The same query can run in milliseconds or minutes depending on the plan.

EXPLAIN (and EXPLAIN ANALYZE) shows you the plan the database chose. The single most important thing to spot: a sequential scan on a big table where you expected an index scan — that usually means a missing index, or a query written so the index can't be used (a function on the column, a leading wildcard LIKE '%x', an implicit type cast). Reading EXPLAIN is how you turn "the query is slow" from a mystery into a fix.

→ First move when a query is slow

Run EXPLAIN ANALYZE. If you see a sequential scan over a large table on a filtered column, you're usually one index (or one query rewrite) away from a huge speedup.

05

Transactions & ACID

A transaction groups several statements so they succeed or fail as a unit. Relational databases guarantee ACID:

AtomicityAll or nothing — every statement commits, or the whole transaction rolls back. No half-done transfers.
ConsistencyValid → valid — constraints (keys, checks) hold before and after; the DB never enters an invalid state.
IsolationNo interference — concurrent transactions don't corrupt each other (tuned by isolation level, next).
DurabilitySurvives crashes — once committed, the data is safe even if the server dies.

The classic example is a money transfer: debit one account and credit another must both happen or neither. Atomicity makes that safe; durability makes it stick. This is why financial and inventory systems live on relational databases — ACID is the guarantee.

06

Isolation levels & the anomalies

Perfect isolation (every transaction as if it ran alone) is expensive, so databases offer isolation levels that trade correctness against concurrency. Each level prevents certain anomalies:

AnomalyWhat happens
Dirty readYou read another transaction's uncommitted change (that may roll back)
Non-repeatable readYou read a row twice and get different values (another txn committed a change between)
Phantom readYou re-run a query and new rows appear (another txn inserted matching rows)

The levels — READ UNCOMMITTED, READ COMMITTED, REPEATABLE READ, SERIALIZABLE — progressively prevent more of these. Most databases default to READ COMMITTED (no dirty reads), and you raise it where you need stronger guarantees. SERIALIZABLE prevents all anomalies but costs the most concurrency. Pick the weakest level that keeps your logic correct.

07

Normalization vs denormalization

Normalization means structuring tables so each fact is stored once, without redundancy — separate users, orders, products tables linked by keys. It keeps data consistent (change a name in one place) and avoids anomalies, and it's the right default. The cost: answering a question often requires joining several tables.

Denormalization deliberately duplicates data (e.g. storing a product name on the order row) to avoid joins and speed reads — at the cost of consistency (now you must update the name in many places) and storage. It's an optimization you apply after measuring, on hot read paths, not a starting point. Normalize first; denormalize surgically when the read cost demands it.

08

The common pitfalls

A handful of mistakes cause most SQL performance pain.

N+1 queriesOne query per item in a loop — fetch a list, then a query per row. Fix with a JOIN or a batched IN (...).
Missing indexFiltering/joining on an unindexed column → full scans. Add the index (check with EXPLAIN).
SELECT *Fetching all columns — wastes I/O and can defeat covering indexes. Select only what you need.
Non-sargableWrapping the column in a function (or leading LIKE '%x') stops the index being used. Rewrite so the raw column is compared.

Underneath all of them is one habit: know what the database is actually doing. Read the plan, index intentionally, fetch in bulk not in loops, and reach for transactions when correctness matters. That's the difference between SQL that works on your laptop and SQL that works in production.

Frequently asked

Quick answers

What is a database index?

A sorted structure (usually a B-tree) that lets the database find rows by a column's value in O(log N) instead of scanning the whole table. It speeds reads and range/ORDER BY queries, at the cost of storage and slower writes.

INNER vs LEFT JOIN?

INNER returns only rows matched in both tables; LEFT returns all left-table rows plus matches, filling NULLs where the right table has none — so it keeps unmatched left rows that INNER would drop.

What does ACID mean?

Atomicity (all-or-nothing), Consistency (valid state to valid state), Isolation (concurrent transactions don't corrupt each other), Durability (committed changes survive crashes) — the guarantees of a transaction.

What is the N+1 problem?

Running one query for a list of N items and then one query per item — N+1 total. It kills performance and is fixed by fetching related data in a single JOIN or a batched query.

Finished this one? 0 / 29 Handbooks done

Explore the topic

See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.