Finished this one? 0 / 111 Paper Breakdowns done
Explore the topic
See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.
More Paper Breakdowns
- The 95% Number, ExaminedThe most-quoted statistic in enterprise AI, read carefully: roughly 95% of pilots reportedly produced no MEASURABLE profit-and-loss impact. Two words carry the sentence. "Measurable" means attributable financial effect — not accuracy, not satisfaction — and "pilot" means a bounded trial, which selects for projects that end before an accounting period closes. Three separate failures hide inside the one number: MEASUREMENT failure (it worked and no baseline exists, because nobody captured the before-picture in week one and it is unrecoverable afterwards), ATTRIBUTION failure (something improved and three other things changed the same quarter, so finance will not credit yours), and ACTUAL failure (the workflow was wrong or nobody adopted it). The first two are why the forward deployed role exists — both are solved by work before and after the model. Sourced, confidence-labelled (reported via press coverage, not a public methodology), and paired with the a16z argument it mirrors.Read →
- 1,000 FDE Jobs, AnalysedReading a thousand job postings beats reading a thousand opinions. One title covers at least THREE distinct jobs — builder (few customers, deep, milestone-driven), pre-sales (many customers, shallow, quarter-driven, often with quota), and internal/platform — and roughly a third of postings using the FDE title are the pre-sales shape, which the posting rarely makes explicit. A separate census counted 1,206 strict-definition postings across 669 companies at a ~$185K median posted base. What every posting asks for: strong general engineering, LLM app patterns, integration reality, deployment where you do not own the cloud, and increasingly MCP servers as deliverables — with EVALUATION the most-cited hard skill and the least taught anywhere. How to read a posting properly, the two questions that settle which species it is, and the honest caveat that posting counts measure demand signals rather than filled roles.Read →
- wav2vec 2.0Self-supervised speech: learn from raw, untranscribed audio, then fine-tune on as little as TEN MINUTES of labels. A CNN encodes the waveform into latent frames; a span is masked; a Transformer context net must identify each masked frame's TRUE quantized latent among K distractors via a contrastive InfoNCE loss (cosine similarity / temperature): L = −log[exp(sim(c,q_true)/κ) / Σ exp(sim(c,q̃)/κ)]. Targets are discretized through a learned codebook (product quantization + Gumbel-softmax) so the model discovers phone-like units. Brought BERT-style masked pretraining and CLIP-style contrastive learning to audio; the basis of HuBERT and multilingual XLS-R. Worked math plus runnable code.Read →
- DeepSeek-V4The efficiency sequel to V3, aimed at the real cost of long context. A ~1.6T-parameter MoE (only ~49B active per token) that serves a 1M-token window at roughly a tenth of V3.2's KV cache. The mechanism is one idea on two faces — HYBRID sparse attention: compress each token's keys/values into a small latent (memory per token: ratio = d_c/width, context-invariant), then SELECT only the top-k blocks of size b to score against, freezing the scored-key count at k·b once n>k·b while dense attention keeps paying O(n²). The two discounts multiply: FLOPs ratio = (k·b/n)·(d_c/width). Reasoning is post-trained with ON-POLICY DISTILLATION — the student generates its own trajectories and a stronger teacher corrects those exact outputs — replacing R1's large-scale RL loop with a denser, more stable signal. Worked math plus runnable code.Read →
- Kimi K3The sequel to Kimi K2, and a different bet: attack decode SPEED, not KV memory. A ~2.8T open MoE with a 1M-token context — reported as the largest open model to date — built on KIMI DELTA ATTENTION (KDA), a linear-attention delta rule. Softmax attention re-reads the whole KV cache every decode step (O(n) per token, so generation slows as context grows); KDA folds the past into ONE fixed-size recurrent state S (~d×d) and reads o=S·q in constant time regardless of length. The DELTA rule (S ← S − β·(S·k − v)·kᵀ) makes each write CORRECTIVE — overwrite a key's old value instead of accumulating it — which is what plain linear attention gets wrong (write (k,a) then (k,b): delta reads b, linear reads a+b, smeared). ATTENTION RESIDUALS keep a minority of full-attention layers for the sharp exact recall a fixed state blurs — a hybrid, not a wholesale swap. Worked math plus runnable code.Read →
- LLMs Get Lost in Multi-Turn ConversationThe ICLR 2026 Outstanding Paper that measured a failure everyone had felt: give a top model a fully-specified task in ONE prompt (concat) and it shines; split the IDENTICAL requirements across several turns (sharded) and 15 leading LLMs fall apart — ~39% average drop. The decomposition is the punchline: APTITUDE (best-case ability) barely moves, but RELIABILITY craters — the spread between a model's best and worst runs roughly DOUBLES. The mechanism is premature commitment: handed partial info, the model guesses a full answer early, locks it in, and when a later turn contradicts it "gets lost and does not recover." A small decay model captures the shape — concat R=p (flat), sharded R=p·(1−q)^(turns−1) (geometric decay), so longer chats get less reliable and the whole loss lives in a reliability term while the ceiling p is untouched. Fixes: front-load the spec, or consolidate/recap a long chat back into one fresh full-spec message. Worked math plus runnable code.Read →