Handbooks / World Models

Model-Based AI~15 min readIntermediate

The idea

A model of the world you can imagine inside.

Q: What are the main types of world models?

Three broad families: latent world models for control (e.g. PlaNet, the Dreamer line) that predict a compact latent state; generative video/interactive models (e.g. Genie, and Sora viewed as a world simulator) that predict future pixels conditioned on actions; and joint-embedding predictive architectures (JEPA) that predict in representation space rather than reconstructing pixels.

Q: Do video-generation models understand physics?

Partly and imperfectly. Models like Sora produce strikingly physical-looking video and clearly capture many regularities, but they also make consistency and physics errors (objects morphing, violated permanence). Whether that constitutes 'understanding' is debated; a true world model needs to be action-conditioned and stay consistent over long horizons, which remains hard.

Humans plan by simulating: we run the next few seconds forward in our heads before we act. A world model gives an AI that same power — a learned predictor of “what happens if I do this” that it can roll forward to plan, to learn cheaply in imagination, or to generate an interactive world. Here’s what world models are, the families that exist, and the fight over whether they really understand physics.

What a world model is

A world model is a learned, predictive model of an environment. In its purest form it answers one question: given the current state and an action, what is the next state (and observation, and reward)? Learn that function well and you have something remarkable — a simulator of the world, inside the agent, that it can query without touching reality.

state_t , action_t → world model → state_t+1 , observation_t+1 , reward_t+1

The classic framing comes from Ha & Schmidhuber’s 2018 “World Models”: an agent that compresses its observations into a latent code, learns to predict how that code evolves, and then trains a tiny controller inside its own dream. That decomposition — see, predict, act — still underlies the field. The payoff is that an agent with a good world model doesn’t just react; it can look ahead.

→ Why “model” is the key word

Model-free RL learns a reflex: state → action, tuned by trial and error. Model-based RL learns the dynamics of the world, then uses them to plan. The first is simpler; the second is what lets you reason about futures you’ve never actually experienced.

Why they matter

Three payoffs make world models one of the most pursued directions in AI — especially for anything embodied.

◷

Sample efficiency

Train a policy inside the model instead of the costly real world. A robot can practice millions of imagined episodes for the price of a few real ones.

⋔

Planning

Roll the model forward over candidate action sequences and pick the best — look ahead instead of acting myopically.

◲

Simulation

A controllable, action-conditioned world is a generator: interactive game worlds, robotics sims, synthetic training data.

The sample-efficiency argument is the sharpest. Real-world interaction — a robot arm, a self-driving car — is slow, expensive, and sometimes dangerous. If the agent can learn a faithful model from limited real data and then do the bulk of its learning “in imagination,” you convert a data problem into a compute problem, which is a far better trade. This is the whole thesis of model-based RL, and why the Dreamer line of agents became a landmark: they learn behaviors almost entirely inside a learned world model.

Three families

“World model” spans research communities that look quite different on the surface. Three families matter most today.

Family	Predicts	Examples	Used for
Latent control models	A compact latent state, forward in time, action-conditioned	PlaNet, Dreamer (v1–v3)	Model-based RL, robotics, control
Generative / interactive video	Future frames (pixels), often conditioned on actions	Genie, GameNGen, Sora (as simulator)	Playable worlds, sims, data generation
Joint-embedding (JEPA)	Future representations, not pixels	I-JEPA, V-JEPA	Self-supervised understanding, planning

Latent control models compress high-dimensional observations into a small latent and predict there — cheap to roll out, ideal for planning and RL. Generative video models predict actual future frames; when they’re action-conditioned and stay consistent, a video generator becomes an interactive world you can steer (Genie learns playable environments from unlabeled video; GameNGen simulated DOOM with a neural network). JEPA, associated with Yann LeCun’s program, argues you should predict in an abstract representation space, not reconstruct every pixel — because forcing a model to render irrelevant detail wastes capacity and courts blurry, uncertain futures. Predict what matters, ignore the noise.

→ Pixels vs representations

The deep split in the field: generative models predict observations (you can watch them); JEPA-style models predict features (you can’t, but they may capture structure more efficiently). It’s the same tension as reconstruction vs. abstraction that runs through all of representation learning.

How a latent world model works

The Dreamer-style recipe is the clearest worked example — the pattern most control world models follow.

Learn the model, then act in imagination

EncodeAn encoder compresses each raw observation (pixels) into a compact latent state.

PredictA dynamics model learns to roll the latent forward given an action — next latent, predicted reward, done flag.

DecodeA decoder reconstructs observations from the latent, providing the training signal that keeps the latent meaningful.

ImagineRoll the dynamics model forward for many steps without the environment — a dream trajectory.

ImproveTrain an actor–critic on those imagined rollouts; the policy learns from predicted futures, then acts for real.

Real interaction feeds the model; the model feeds the policy. Most learning happens in the dream.

Two ways to use the learned model: planning (at decision time, search over action sequences by rolling them out and scoring — model-predictive control), or policy learning in imagination (train a reactive policy on imagined rollouts, so acting is fast at runtime). Dreamer does the latter, which is why it can master hard control tasks from remarkably little real experience. The whole scheme lives or dies on one property: the dynamics model’s predictions have to stay accurate over the horizon you plan across.

The physics-understanding debate

When a model like Sora generates a minute of coherent, physical-looking video, it’s natural to ask: has it learned physics? The honest answer is some, imperfectly, and “understanding” is the wrong yes/no question.

The case for

Generating consistent motion, occlusion, and lighting requires capturing real regularities of the world.
These models clearly learn intuitive-physics priors that transfer — objects fall, fluids flow, faces persist.
Scaling has repeatedly turned “interpolation” into surprising generalization.

The case against

They still break physics: objects morph, appear and vanish, permanence and counting fail.
Looking physical is not being physical — a plausible frame can be dynamically wrong.
A pixel-predictor optimizes visual likelihood, not a causal model of the world.

The productive reframing: a useful world model isn’t judged on how real its output looks, but on three harder properties — it must be action-conditioned (respond correctly to what you do), consistent over long horizons (no drift into nonsense), and controllable (steerable, not just pretty). Today’s video models are strong on appearance and weak on the last three. Closing that gap — from “generates convincing video” to “simulates a world you can act in and trust” — is exactly the open frontier.

Where it’s heading

World models sit at the center of several of AI’s next bets, and share a common set of hard problems.

Embodied AI & robotics. Learn a world model from limited real interaction, then train policies in imagination and transfer to hardware — the sim-to-real and data-efficiency play.
Interactive generative worlds. Action-conditioned video models as game engines and infinite training environments (the Genie / neural-game-engine direction).
Planning agents. Giving agents an explicit model to look ahead in, rather than reacting step to step — closely related to how test-time compute buys better decisions by searching futures.

The recurring obstacles are the same across all of them: compounding error (small prediction mistakes snowball over a long rollout), coverage (the model is only trustworthy where its training data went), and evaluation (measuring whether a world model is right, not just realistic, is genuinely hard). Progress on world models is, in large part, progress on those three.

Related concepts

Model-based RL Latent dynamics Model-predictive control JEPA Sim-to-real Intuitive physics

Frequently asked

Quick answers

What is a world model in AI?

A learned, predictive model of an environment: given the current state and an action, it predicts what happens next. With one, an agent can imagine the consequences of actions and plan without acting in the real world — the basis of model-based reinforcement learning and of controllable simulators.

Why are world models useful?

They enable planning (search over imagined futures), sample-efficient learning (train a policy inside the model instead of the costly real world), and simulation. For robotics and games, learning in imagination can be orders of magnitude cheaper than learning from real interaction.

What are the main types of world models?

Three families: latent control models (PlaNet, Dreamer) that predict a compact latent state for RL and planning; generative video/interactive models (Genie, Sora as a simulator) that predict future frames conditioned on actions; and joint-embedding predictive architectures (JEPA) that predict in representation space rather than reconstructing pixels.

Do video-generation models understand physics?

Partly and imperfectly. They produce strikingly physical-looking video and capture many regularities, but they also make consistency and physics errors. Whether that is “understanding” is debated; a true world model must be action-conditioned and stay consistent over long horizons, which remains hard.

Finished this one? 0 / 208 Handbooks done

Explore the topic

See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.

ML Foundations