AlphaGo,
explained.

Chess programs win by searching: look ahead many moves, evaluate the resulting positions with a hand-crafted scoring function, and pick the line that leads somewhere good. Go breaks this on two axes. Its branching factor is enormous — around 250 legal moves per turn versus chess's ~35 — so the tree explodes far faster. And no one knew how to write a good evaluation function for a Go position; the value of a board is famously subtle, resisting the tidy piece-counting that works in chess.

So AlphaGo's problem was really two problems: the tree is too wide to search fully, and positions are too subtle to score by rule. Its answer was to learn both from data. A policy network learned which moves are worth considering — narrowing the tree — and a value network learned to score positions — replacing the missing evaluation function. Deep learning supplied the intuition; tree search supplied the calculation. Neither alone was enough; together they were superhuman.

Two deep networks read the board. The policy network outputs a probability over moves — a prior saying which moves look good before any search. This is what tames the branching factor: instead of considering all ~250 moves seriously, the search concentrates on the handful the policy rates highly. The value network outputs a single number — an estimate of the probability that the current player wins from this position. This is the learned evaluation function Go never had.

Both are crucial to making search tractable. Earlier Go programs evaluated a position by playing it out to the end with fast random moves (Monte Carlo rollouts) — noisy and slow. AlphaGo's value network estimates the outcome directly from the position, and its policy network makes any rollouts it does far smarter. The networks turn an impossible search into a guided one.

Monte Carlo Tree Search grows a search tree one simulation at a time. Each simulation: select a path down the tree, expand a new leaf, evaluate it with the value network, and back up the result to update every node on the path. The heart is the selection rule, PUCT, which decides at each node which move to explore by balancing exploitation against exploration:

The first term Q(a) = W/N is the move's average result so far — pure exploitation, pick what's been working. The second rewards moves with high policy prior P(a) and few visits N(a) — exploration, try promising or under-explored moves. Early on, the prior dominates and search follows the policy network's intuition; as a move accumulates visits, its exploration bonus shrinks and the choice increasingly trusts its measured value. Each simulation backs up its leaf value, incrementing visit counts and refining Q. After thousands of simulations, the most-visited move at the root is played. The runnable version below shows PUCT favoring the prior early and the value late.

AlphaGo's networks were trained in stages, each building on the last:

Starting from human games gave the policy a strong initialization; self-play then pushed it past human play by discovering moves no human database contained. The value network, trained on the outcomes of these self-play games, learned to judge positions the way a strong player would. Its successor, AlphaGo Zero, later removed the human data entirely — learning from pure self-play, tabula rasa, and surpassing the original.

In March 2016, AlphaGo beat Lee Sedol, one of the world's top players, 4–1 — a milestone most had expected to be a decade away:

The technical lesson outlasted the match: learned intuition plus deliberate search is a powerful pattern. The networks provided fast, pattern-based judgment; the search provided slow, careful lookahead; each covered the other's weakness. That division — System-1 nets guiding System-2 search — became a template far beyond Go.

AlphaGo's lineage generalized fast. AlphaGo Zero dropped human data and learned Go from scratch by self-play; AlphaZero generalized the same algorithm to chess and shogi with no game-specific knowledge; MuZero learned the rules themselves, planning in a learned model of the environment. The through-line is a single algorithm — search guided by learned policy and value, improved by self-play — applied ever more generally.

Its deepest influence, though, is the idea of scaling test-time search. AlphaGo showed that spending more compute at decision time — running more MCTS simulations — makes a system stronger, separately from training. That principle, learned intuition amplified by deliberate search, is exactly what today's reasoning models revive: generate and evaluate many lines of thought before answering. From a board game to self-improving reasoners, the same bet — think longer, search guided by learned judgment — keeps paying off.