Data & Retrieval

Lambda Architecture

A data design running a slow accurate batch layer alongside a fast approximate streaming layer.

Lambda architecture is a big-data pattern that processes data through two parallel paths: a batch layer that recomputes complete, accurate results over all historical data, and a speed layer that handles recent data with low latency but less accuracy. A serving layer merges both so queries see fresh-and-eventually-correct results.

Worked example: a metrics dashboard might show up-to-the-second counts from the streaming speed layer, while a nightly batch job recomputes the exact historical totals and overwrites any drift the fast path introduced. Gotcha: the cost is duplicated logic — the same aggregation is written and maintained twice, once for batch and once for streaming, and keeping them in sync is painful enough that kappa architecture was proposed to collapse both into a single stream path.