Vibe Engines — Databases & Storage

SQL vs Vector Database

Fri, 24 Jul 2026 00:00:00 GMT

SQL vs vector databases, decided by the kind of question you ask: a relational database answers exact, structured queries over rows (WHERE price < 50); a vector database answers similarity queries over embeddings (find the items most like this). Exact filters vs approximate nearest neighbors, B-tree vs HNSW, pgvector, and why most real systems need both.

Database Index Size Estimator

Fri, 24 Jul 2026 00:00:00 GMT

A database index size estimator. Enter the row count, the size of the indexed key, and a fill factor to estimate the on-disk size of a B-tree index — including the row-pointer overhead and internal nodes — so you can anticipate the storage and memory cost of an index before adding it. Compares the cost of several indexes at once.

Storage Growth Forecaster

Fri, 24 Jul 2026 00:00:00 GMT

A storage growth forecaster. Enter your current data size and either a daily ingest rate or a monthly growth percentage, set a replication factor, and see your projected size at 3, 6, 12 and 24 months — plus when you would cross a capacity threshold. Handy for capacity planning databases, logs, object stores and backups.

The Query Planner

Wed, 22 Jul 2026 00:00:00 GMT

Don't guess why your query is slow — watch the planner choose. A cost-based query planner estimates the cost of every way to run a query — full table scan, index scan, bitmap AND of two indexes — and picks the cheapest. The catch: an index isn't always a win. Change how selective each predicate is, add or drop indexes, and watch the planner flip between a sequential scan and an index scan for reasons you can finally see. Made playable, with theory and a quiz.

B-Tree vs LSM-Tree

Wed, 22 Jul 2026 00:00:00 GMT

Don't read about storage engines — race them. A B-tree updates data in place: reads are cheap and stable, but every write is a random disk write. An LSM-tree appends writes to an in-memory buffer, flushes them as sorted runs, and compacts in the background: writes are cheap and sequential, but a read may have to check several runs. Stream the same writes into both, watch flushes and compaction happen, and see which engine wins for writes and which for reads — made playable, with theory and a quiz.

OLTP vs OLAP

Tue, 21 Jul 2026 00:00:00 GMT

One storage-layout decision explains the whole comparison: row-oriented OLTP makes fetching one record cheap, column-oriented OLAP makes aggregating one column across millions of rows cheap. Why you can’t run heavy analytics well on your production database.

LSM-Tree: Write Fast, Merge Later

Tue, 21 Jul 2026 00:00:00 GMT

Don't memorize the LSM-tree — play it. Writes land instantly in an in-memory memtable; when it fills, it flushes to an immutable sorted file (SSTable) on disk. Reads check the memtable then the SSTables newest-first, and compaction merges files while dropping shadowed old versions. The write-optimized engine behind RocksDB, Cassandra, and LevelDB — made playable, with theory and a quiz.

Design a Logging Pipeline

Mon, 20 Jul 2026 00:00:00 GMT

Build the ingestion pipeline that collects, buffers and delivers logs from thousands of services — the write path, not the query/search side. See why synchronous writes to a central log server are dangerous, local agent buffering, bounded backpressure and overflow policy, batching for throughput, where structured parsing should happen, at-least-once delivery with dedup, sampling at high volume, and why a logging storm can take down the very system it was meant to observe.

Design an Email System

Mon, 20 Jul 2026 00:00:00 GMT

Build an email system like Gmail or Outlook. See why raw inbound mail can't land straight in a mailbox, retry-with-backoff for unreliable delivery across the open internet, layered spam/authentication filtering, per-user mailbox sharding, thread reconstruction via message headers instead of subject lines, delivery dedup and bounce-loop prevention, and IMAP-style incremental multi-device sync.

The PostgreSQL Internals Handbook

Sat, 18 Jul 2026 00:00:00 GMT

UPDATE never updates, and a janitor keeps it working. Heap pages and tuples, MVCC's row versions (xmin/xmax visibility arithmetic), VACUUM, bloat and the wraparound scare, B-trees, HOT updates and index-only scans, the WAL's three superpowers (recovery, replication, PITR), reading the planner honestly, and why PgBouncer is infrastructure.

SQL vs NoSQL

Sat, 18 Jul 2026 00:00:00 GMT

Framed as a war, it’s really a menu. Relational databases give you schema, joins and ACID; NoSQL is an umbrella of document/key-value/wide-column/graph stores that drop some of that for flexibility and horizontal scale. The relationships-vs-scale core difference, the CAP trade-off, and why to start relational.

PostgreSQL vs MySQL

Sat, 18 Jul 2026 00:00:00 GMT

Both are excellent free relational databases you can build a company on, so the choice is about character, not capability: Postgres prizes correctness, rich types and advanced features; MySQL prizes simplicity and read-heavy speed. The real differences, when each wins, and why Postgres became the modern default.

The SQL Handbook

Tue, 14 Jul 2026 00:00:00 GMT

What the database does underneath your SELECT — the relational model and keys, joins (inner/left/right/full), indexes and B-trees, the query planner and reading EXPLAIN, transactions and ACID, isolation levels and the anomalies they prevent, normalization vs denormalization, and the pitfalls (N+1, missing indexes, SELECT *).

Design Object Storage

Tue, 14 Jul 2026 00:00:00 GMT

Build an object store like Amazon S3. See why one disk caps capacity, durability and availability, how a flat bucket/key namespace scales to trillions of immutable objects, how a metadata service maps keys to chunk locations, how replication and erasure coding deliver eleven-nines durability, how read-after-write consistency is anchored in metadata, and how multipart uploads, tiering and background repair work.

Design a Distributed File System

Tue, 14 Jul 2026 00:00:00 GMT

Build a distributed file system like GFS or HDFS. See why one file server caps capacity and throughput, how huge files split into large chunks across chunkservers, how a single master serves only metadata while clients stream bytes directly, how rack-aware replication survives correlated failures, how a leased primary orders pipelined writes, and how the master is made recoverable.

Design a Time-Series Database

Tue, 14 Jul 2026 00:00:00 GMT

Build a time-series database like InfluxDB or Prometheus TSDB. See why a row-per-point SQL table melts, how series = metric + tags models the data, how columnar storage with delta-of-delta timestamps and XOR values shrinks points to ~1–2 bytes, how an LSM write path with a WAL swallows the firehose, how sharding by series and downsampling work, and why cardinality is the real scaling limit.

Design a Log Search System

Tue, 14 Jul 2026 00:00:00 GMT

Build a log aggregation and search system like the ELK stack or Loki. See why grep-over-SSH can't scale, how structured logs and shipper agents get data off ephemeral hosts, how a Kafka buffer absorbs bursts, how a parse/enrich pipeline feeds an inverted index, how time-based sharding and scatter-gather search billions of lines in milliseconds, and how hot-warm-cold retention bounds cost.

Design a Digital Wallet

Tue, 14 Jul 2026 00:00:00 GMT

Build a digital wallet like PayPal or Venmo. See why a mutable balance column is dangerous, how an append-only double-entry ledger makes balance derived and money conserved, how atomic debit+credit transfers survive a mid-transfer crash, how idempotency keys stop double-charges, how per-account locking prevents double-spend, how cross-shard transfers use sagas, and how pending/settled states and reconciliation handle slow bank rails.

Design a Proximity Service

Tue, 14 Jul 2026 00:00:00 GMT

Build a proximity / nearby-search service like Yelp or "find drivers near me". See why SQL distance queries degrade to full scans, the 2D indexing problem, how geohash encodes 2D into a locality-preserving 1D key, how quadtrees and S2 cells adapt to density, the cell-boundary problem, caching hot cells, and how moving objects and geo-sharding are handled.

Data Engineer Roadmap

Tue, 14 Jul 2026 00:00:00 GMT

A step-by-step roadmap to become a data engineer in 2026. From SQL, Python, and data modeling through Spark, Kafka, Airflow, and dbt to data quality, governance, partitioning, and CDC. 18 stations across 3 tracks — Foundations, Core Data, Production.

Concurrency, Locks & Isolation Levels

Sat, 13 Jun 2026 00:00:00 GMT

How overlapping operations stay correct — race conditions, optimistic vs pessimistic locking, the four isolation levels and their anomalies, MVCC, and distributed locks with fencing tokens. Part of System Design Fundamentals.

Partitioning, Sharding & Replication

Sat, 13 Jun 2026 00:00:00 GMT

How one dataset becomes many — range vs hash partitioning, consistent hashing and virtual nodes, hot partitions, replication topologies, replication lag, and quorums. Part of System Design Fundamentals.

Design a Key-Value Store

Wed, 10 Jun 2026 00:00:00 GMT

Build a distributed key-value store like DynamoDB. See how consistent hashing places keys, how replication and tunable quorums (R + W > N) trade consistency for availability, how vector clocks resolve conflicts, and how gossip membership, hinted handoff and read repair keep it alive through failure.

Design a Web Crawler

Wed, 10 Jun 2026 00:00:00 GMT

Build a distributed web crawler. See how a URL frontier drives the fetch–parse–enqueue loop, how a bloom-filter seen-set stops infinite re-crawling, how politeness and robots.txt keep you a good citizen, how DNS caching and content dedup remove bottlenecks, and how to shard the frontier by host and dodge traps.

Design Instagram

Wed, 10 Jun 2026 00:00:00 GMT

Build a photo-sharing app. See how splitting media from metadata, a CDN for immutable images, async image processing, a precomputed feed cache, asynchronous like/view counters, TTL-based ephemeral stories, and sharding fit together to serve billions of photos.

Design Pastebin

Wed, 10 Jun 2026 00:00:00 GMT

Build a text-paste service. See how base62 key generation, separating metadata from blob storage, a cache and CDN for the read-heavy immutable path, TTL expiration, privacy flags and size limits, and async analytics fit together.

Design a Search Engine

Wed, 10 Jun 2026 00:00:00 GMT

Build a web search engine. See how an inverted index turns search into list lookups, how query parsing mirrors indexing, how an offline indexer builds postings, how BM25 and PageRank rank results, and how document-sharded scatter-gather, result caching and a fresh index scale it to billions of pages.

Design Airbnb

Wed, 10 Jun 2026 00:00:00 GMT

Build a lodging marketplace. See how listings and geo search with filters, per-listing availability calendars, atomic booking that never double-books, payment hold-and-capture with idempotency, event-driven reviews and reindexing, and read replicas fit together.

Design a Payment System

Wed, 10 Jun 2026 00:00:00 GMT

Build a payment system like Stripe. See how a payment is modeled as a state machine of authorize and capture, how idempotency keys prevent double-charges, how a double-entry ledger keeps every cent auditable, how tokenization shrinks PCI scope, and how async webhooks, reconciliation and a durable retryable worker handle slow banks and failure.

Design YouTube

Tue, 09 Jun 2026 00:00:00 GMT

Build a planet-scale video platform. See how raw uploads become an adaptive-bitrate ladder through an async transcoding pipeline, how a CDN serves immutable segments from the edge, how tiered blob storage holds exabytes affordably, and how view counts stay async.

Design Dropbox

Tue, 09 Jun 2026 00:00:00 GMT

Build a file sync engine. See how splitting metadata from bytes, content-addressed block storage with deduplication, delta sync, a push-then-pull notification service, and conflict-safe versioning fit together to keep files consistent across every device.

Design Ticketmaster

Tue, 09 Jun 2026 00:00:00 GMT

Build an event-booking system. Tackle the double-booking problem head-on: seat inventory, short-lived holds with TTLs, exactly-once purchases via ACID transactions, payment sagas with idempotency, and a virtual waiting room that tames the on-sale stampede.