What you will learn
Train a small predictive model and identify where rollout errors invalidate its plans.
Before you begin: Reinforcement learning + SNNs; Memory: biological & artificial
START WITH A QUESTION
Picture this.
Before moving a chair through a doorway, you imagine whether it will fit. A world model serves a related computational purpose: predict consequences before committing to an action. The prediction can be useful without being a complete replica of reality.
- Rollout
- A sequence of predicted or simulated future states.
- Latent state
- An internal representation used instead of directly observed variables.
- Model bias
- A systematic mismatch between a model's predictions and the process it models.
Read for the idea first. Work through the numbers next. Try answering the questions before opening the explanations.
Predict a task-relevant future
A model may predict position, an image, a reward or a compact learned representation called a latent state. The right target depends on the decision. Our scalar example predicts x_next from current x and action u using a*x + b*u. The coefficients are chosen, not learned from data. This is enough to demonstrate planning: try candidate actions inside the model, score the predicted outcomes and choose a proposal. Keep the internal prediction distinct from what the environment actually does.
Use a model to choose, then observe
Let x = 1, a = 0.8 and b = 0.5. Candidate actions −0.2, 0 and +0.2 predict states 0.7, 0.8 and 0.9. If the target is 1, the last prediction has the smallest squared distance, so the planner proposes +0.2. If the observed next state is 1.1 instead, prediction error is 0.2. We now have two measurements: how well the model predicted and how close the actual outcome was to the target. They answer different questions.
A little error can become a large story
Rollouts feed predictions back as future inputs. A biased model can drift over many imagined steps, and a planner can choose actions that exploit the model's mistakes. Replanning after fresh observations can limit how long the system relies on an uncorrected forecast. Evaluate the horizon relevant to your decision, including actions the planner actually selects. A low average one-step error is encouraging, but it is not proof of reliable planning in unusual situations.
Connect research without collapsing distinctions
Learned world-model research, including Dreamer, studies how predictions can support behavior learning. Predictive coding and active inference are related topics in theoretical neuroscience, but their assumptions and claims are not interchangeable with every engineering predictor. For this book's learning path, start with the concrete loop: observe, predict candidate consequences, choose, then compare prediction with reality. A fast reflex may run alongside a slower planner when waiting for a rollout would miss the task deadline.
SEE THE RELATIONSHIPS
From structure to behavior.
MAKE THE IDEA PRECISE
The mathematics.
| Symbol | Meaning |
|---|---|
| x | Current scalar state |
| u | Action input |
| a, b | Chosen model coefficients |
| x_hat_next | Predicted next state |
| x_observed | Measured next state on the same scale |
Build the equation, step by step
- Use the current state and action to produce a next-state prediction. A scalar linear model assumes fixed coefficients and a simple state representation.
- Subtract prediction from the observed next state. Squaring removes the sign and emphasizes larger errors.
- For a position state, squared error has units of distance squared. Mixing position, angle and velocity errors in one loss requires explicit scaling. Good one-step error does not guarantee good long-horizon planning.
WORK THROUGH THE NUMBERS
Worked example
With x = 1, u = 0.2, a = 0.8 and b = 0.5, prediction is 0.9. An observed value of 1.1 produces error 0.2 and squared error 0.04. The coefficients are illustrative, not fitted data.
A small Python experiment.
Python 3 · Standard library only · Synthetic educational example. Run this in your own notebook or a local file.
x, target = 1.0, 1.0
actions = [-0.2, 0.0, 0.2]
predictions = [0.8*x + 0.5*u for u in actions]
best = min(range(len(actions)), key=lambda i: (predictions[i]-target)**2)
best_action = actions[best]
prediction = predictions[best]
observed = 1.1 # Separate synthetic observation, not the model output
print(f"action: {best_action:.1f}; predicted state: {prediction:.2f}")
print(f"prediction error squared: {(observed-prediction)**2:.2f}")Read the code as a story
The code evaluates three actions using the same model and target. min selects the smallest predicted squared target error. It then compares the chosen prediction with a separate illustrative observation. That observation is not generated by the model, so the mismatch is visible. This is one-step planning, not a trained world model.
Expected output
action: 0.2; predicted state: 0.90 prediction error squared: 0.04
Change one quantity, predict the result, and then run the code again. Explain any difference before adding complexity.
Test your understanding.
Try it: Roll the model forward for ten steps without corrections. Explain how an incorrect coefficient can accumulate error.
Self-check: Why can a good predictor fail during planning?
The planner may choose actions outside the training distribution or exploit systematic prediction errors. Small errors can compound over many steps. Evaluate rollouts under the policy that will actually use the model, including uncertainty and constraint violations.
Where this model stops: Small average prediction error does not guarantee safe decisions on rare events.
A research question: When should a fast reactive controller override a model-based plan?
Record evidence in your mastery worksheet →PREDICT / TRY / EXPLAIN
Separate prediction quality from task quality
Use paper, a calculator or the Python example. Write your prediction before checking the result.
- Calculate the three candidate predictions and squared distances to target 1.
- For the chosen action, compare prediction 0.9 with observation 1.1. Calculate both prediction error squared and actual target error squared.
- Change b in the model while keeping the observation rule separate. Explain why editing both together would hide model error.
Compare with a worked solution
Predicted target errors are 0.09, 0.04 and 0.01, so +0.2 is selected. Prediction error squared is (1.1−0.9)² = 0.04; actual target error squared is (1.1−1)² = 0.01. A model can be inaccurate on a particular transition while the action happens to end near the target. Repeated matched trials are needed to assess usefulness.
CLOSE YOUR NOTES
Can you explain it back?
These questions test the reasoning, not just a remembered phrase. Say your answer out loud or write a sentence first.
01Why is a generated future not automatically a reliable plan?
Reveal the reasoning for question 1
A plausible-looking future can be wrong about the details that determine success. Evaluate consequences and constraints under actions selected by the planner.
02If x is measured in meters, what unit does squared prediction error have?
Reveal the reasoning for question 2
Square meters. Combining that loss with angle or velocity losses requires explicit scaling rather than adding unlike quantities blindly.
TAKE THE IDEA WITH YOU
Three things to remember.
- A world model predicts selected aspects of the future.
- Planning error and prediction error are related but distinct.
- Fresh observations and horizon-specific evaluation expose rollout drift.
Next we move from one agent predicting its future to several agents coordinating through limited information.
Continue to Part 19 →THE NEXT LAYER
Chapter depth map.
This introduction provides a first model. The following chapters are outlines for deeper future editions; their full treatments have not yet been published.
Representing possible futures
18.1 State-space and latent models · Outline
- 18.1.1 Observed and hidden state
- 18.1.2 Transition and observation models
- 18.1.3 Latent representations and uncertainty
- 18.1.4 Multi-step prediction and distribution shift
- 18.1.5 Calibration and identifiability
18.2 Model-based reinforcement learning · Outline
- 18.2.1 Learned rollouts and imagined trajectories
- 18.2.2 Planning in latent state
- 18.2.3 Model bias and compounding error
- 18.2.4 Dreamer as a reading case study
- 18.2.5 Compare with model-free and classical baselines
Biological theories and engineering claims
18.3 Prediction in neuroscience · Outline
- 18.3.1 Internal models and sensorimotor prediction
- 18.3.2 Predictive processing and predictive coding
- 18.3.3 Active inference: assumptions and objectives
- 18.3.4 Distinguish theory, mechanism and experiment
- 18.3.5 Avoid treating a debated unifying theory as settled fact
18.4 World model meets reflex · Outline
- 18.4.1 Slow forecasts and fast reactive loops
- 18.4.2 Confidence thresholds for intervention
- 18.4.3 Memory and model updates
- 18.4.4 Counterfactual evaluation and ablations
- 18.4.5 Test benefit under unexpected obstacles
Read with the original sources.
These readings support the concepts and deeper investigation. The worked numbers and illustrations above are teaching examples.
Primary research / versioned manuscript
Hafner et al.: Mastering Diverse Domains through World Models ↗
Read after latent dynamics and actor-critic learning. Evaluate imagined rollouts, model errors, and the scope of reported task results.
Theory / review by its proponent
Friston (2010) — The free-energy principle: a unified brain theory? ↗
Read as a proposed theoretical framework, not a settled description of every brain computation.