Vibe Engines
YouTube
← All decks

Glossary: Data & Retrieval

Loading your review…

All 42 cards in this deck

Embeddings, indexes, chunking, ranking — the retrieval half of the glossary as flashcards. Answer each question before you open it — then run the review above, which schedules every card with spaced repetition so it comes back right before you would forget it.

  1. Embedding

    A vector that places a piece of text in space so that nearby vectors mean similar things.

  2. Retrieval-Augmented Generation (RAG) (RAG)

    Fetching relevant documents and putting them in the prompt so the model answers from real sources.

  3. Vector Database

    A store built to index and search embeddings by nearest-neighbor similarity at scale.

  4. Cosine Similarity

    How aligned two vectors are, ignoring their length — 1 identical, 0 unrelated, −1 opposite.

  5. Bloom Filter

    A tiny probabilistic set that answers “maybe present” or “definitely absent” — no false negatives.

  6. Chunking

    Splitting documents into retrievable pieces — the decision that quietly makes or breaks RAG quality.

  7. BM25 (keyword search, lexical search)

    The classic keyword-ranking function — still the baseline every fancy retriever must beat.

  8. Hybrid Search

    Keyword search and vector search fused — lexical precision plus semantic recall.

  9. Reranking

    A second, smarter pass that reorders retrieved candidates before they reach the prompt.

  10. HNSW (approximate nearest neighbor, ANN index)

    The graph index behind most vector databases — approximate nearest-neighbor search in milliseconds.

  11. Data Foundry

    An operation that manufactures high-quality training and evaluation data at scale, blending human expertise and synthetic generation.

  12. Query Fan-Out (fan-out retrieval, query expansion)

    Expanding one query into many sub-queries issued in parallel, then merging the results.

  13. Semantic Search

    Search that matches on meaning via embeddings, not exact keywords.

  14. Precision and Recall (F1 score)

    Two sides of accuracy: precision is how many of your hits were right; recall is how many of the right ones you caught.

  15. Cross-Encoder

    A reranker that reads the query and a document TOGETHER for an accurate relevance score — slow but precise.

  16. Bi-Encoder

    An encoder that embeds queries and documents separately, so documents can be precomputed and searched fast.

  17. NDCG (normalized discounted cumulative gain)

    A ranking metric that rewards putting the most relevant results highest, discounting relevance further down the list.

  18. MRR (mean reciprocal rank)

    Mean Reciprocal Rank: rewards how high the FIRST correct result appears, averaged over queries.

  19. Recall@k

    Of all the relevant items, what fraction appear in the top-k results — the key metric for the retrieval stage of RAG.

  20. Dense Retrieval

    Retrieval by embedding meaning into vectors and finding nearest neighbors — matches on semantics, not keywords.

  21. Sparse Retrieval

    Classic keyword retrieval (like BM25) scoring documents by term overlap — sparse because most vocabulary entries are zero.

  22. Data Warehouse

    A database optimized for analytics — big aggregate queries over historical data — separate from the transactional systems.

  23. Data Lake (lakehouse)

    A store for raw data in any format at scale, schema applied on read — flexible but easy to turn into a swamp.

  24. ETL (ELT)

    Extract, Transform, Load — the pipeline that moves data from source systems into a warehouse, cleaning it on the way.

  25. OLTP vs OLAP (online transaction processing)

    Online Transaction Processing — the fast, small read/write workload of live apps, versus OLAP analytics.

  26. Columnar Storage

    Storing table data by column instead of by row, so analytics that touch a few columns scan far less data.

  27. Cardinality

    How many distinct values a column or dataset has — high cardinality (many unique) versus low (few).

  28. Star Schema

    A data-warehouse layout with one central fact table linked to descriptive dimension tables — simple and fast for analytics.

  29. DAG (directed acyclic graph)

    A Directed Acyclic Graph: tasks with dependencies but no cycles — the model behind data pipelines and build systems.

  30. Backfill

    Running a pipeline over past data to populate or correct history — after adding a metric or fixing a bug.

  31. Schema Registry

    A central store of the schemas for messages in a streaming system, enforcing compatibility as producers and consumers evolve.

  32. Data Contract

    An explicit, enforced agreement about the schema and semantics of data passed between a producer and its consumers.

  33. Feature Store

    A central system for computing, storing, and serving ML features consistently between training and production.

  34. Embedding Model

    A model whose job is to turn text (or images) into embedding vectors for search, clustering, and RAG.

  35. Dot Product

    A similarity measure summing the products of two vectors’ components — bigger when they point the same way and are large.

  36. Euclidean Distance (L2 distance)

    Straight-line distance between two points/vectors — the intuitive “ruler” metric, sensitive to magnitude.

  37. Lambda Architecture

    A data design running a slow accurate batch layer alongside a fast approximate streaming layer.

  38. Kappa Architecture

    A simplification of lambda that handles both real-time and reprocessing with one streaming path.

  39. Data Mesh

    An org approach treating data as a product owned by domain teams, not one central data team.

  40. Stream Processing

    Computing over data continuously as it arrives, event by event, rather than in scheduled batches.

  41. Batch Processing

    Processing data in large scheduled chunks rather than continuously as it arrives.

  42. OLAP (Online Analytical Processing)

    Databases optimized for heavy analytical queries over large datasets, not many small transactions.