Vibe Engines is free, interactive AI engineering & system design — runs in your browser. Pick a learning path
Finished this one? 0 / 110 Paper Breakdowns done
Explore the topic
See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.
More Paper Breakdowns
- The 95% Number, ExaminedThe most-quoted statistic in enterprise AI, read carefully: roughly 95% of pilots reportedly produced no MEASURABLE profit-and-loss impact. Two words carry the sentence. "Measurable" means attributable financial effect — not accuracy, not satisfaction — and "pilot" means a bounded trial, which selects for projects that end before an accounting period closes. Three separate failures hide inside the one number: MEASUREMENT failure (it worked and no baseline exists, because nobody captured the before-picture in week one and it is unrecoverable afterwards), ATTRIBUTION failure (something improved and three other things changed the same quarter, so finance will not credit yours), and ACTUAL failure (the workflow was wrong or nobody adopted it). The first two are why the forward deployed role exists — both are solved by work before and after the model. Sourced, confidence-labelled (reported via press coverage, not a public methodology), and paired with the a16z argument it mirrors.Read →
- 1,000 FDE Jobs, AnalysedReading a thousand job postings beats reading a thousand opinions. One title covers at least THREE distinct jobs — builder (few customers, deep, milestone-driven), pre-sales (many customers, shallow, quarter-driven, often with quota), and internal/platform — and roughly a third of postings using the FDE title are the pre-sales shape, which the posting rarely makes explicit. A separate census counted 1,206 strict-definition postings across 669 companies at a ~$185K median posted base. What every posting asks for: strong general engineering, LLM app patterns, integration reality, deployment where you do not own the cloud, and increasingly MCP servers as deliverables — with EVALUATION the most-cited hard skill and the least taught anywhere. How to read a posting properly, the two questions that settle which species it is, and the honest caveat that posting counts measure demand signals rather than filled roles.Read →
- GloVeWord vectors from GLOBAL co-occurrence statistics — the count-based cousin of word2vec. Build a co-occurrence matrix X (X_ij = how often word j appears in the context of word i), then learn vectors so a dot product recovers the LOG count: w_i·w̃_j + b_i + b̃_j ≈ log(X_ij). The insight: co-occurrence RATIOS (P(solid|ice)/P(solid|steam)) carry meaning, and taking logs turns multiplicative ratios into differences a vector space represents — which is why king−man+woman≈queen works. Fit with weighted least squares, J = Σ f(X_ij)(…−log X_ij)², where f(x)=(x/xmax)^0.75 caps frequent pairs and f(0)=0 skips the empty matrix. The standard pretrained word embedding for years. Worked math plus runnable code.Read →
- The Lottery Ticket HypothesisThe finding that a dense, randomly-initialized network hides a small "winning ticket" — a sparse subnetwork that, trained in isolation FROM THE SAME INITIAL WEIGHTS, matches the full model's accuracy. You find it by iterative magnitude pruning: train, drop the smallest-magnitude weights (a binary mask), RESET survivors to their original init, repeat. The famous twist: randomly reinitializing the same sparse structure trains worse — structure and lucky initialization are entangled. After k rounds at prune fraction p, only (1−p)^k of the weights remain. A new lens on why over-parameterization helps, and the seed of sparse-training research. Worked math plus runnable code.Read →
- DDIMDenoising Diffusion Implicit Models — the trick that made diffusion sampling deterministic and 10–50× faster, using the SAME trained DDPM network (no retraining). Each step predicts the clean image x̂0 = (x_t − √(1−ᾱ_t)·ε)/√ᾱ_t, then re-projects to an earlier step: x_{t−1} = √ᾱ_{t−1}·x̂0 + √(1−ᾱ_{t−1}−σ²)·ε + σ·z. A single knob σ = η·√((1−ᾱ_{t−1})/(1−ᾱ_t))·√(1−ᾱ_t/ᾱ_{t−1}) unifies the two samplers: η=1 recovers stochastic DDPM, η=0 gives σ=0 and the deterministic DDIM that can skip timesteps. Determinism unlocks reproducibility, latent interpolation, and DDIM inversion for editing — and reframed sampling as an ODE. Worked math plus runnable code.Read →
- Self-InstructHow to bootstrap a large instruction-tuning dataset from a language model's OWN generations, starting from just 175 human-written seed tasks. The generate→filter→add loop: sample existing tasks as examples, prompt the model to write new instructions + input/output instances, filter, add survivors back, repeat — 175 seeds bloom into ~52K diverse tasks. The quantitative heart is a diversity filter: keep a new instruction only if its ROUGE-L similarity (longest-common-subsequence overlap) to every existing one is below 0.7, so the pool never collapses into near-duplicates. The seed of Alpaca and the open instruction-tuning wave, and a landmark in synthetic data. Worked math plus runnable code.Read →