Prediction, uncertainty and model evidence
The Free Energy Principle
Free energy is a way of measuring how well a model explains sensory data, while also penalising overly flexible or implausible explanations.
1. From prediction error to free energy
Predictive coding focuses on reducing prediction error. The Free Energy Principle generalises this idea: organisms act and perceive in ways that keep their sensory states predictable under their own model of the world.
A simple teaching version is:
$F \approx \text{prediction error} + \text{complexity penalty}$
The first term rewards accurate explanations. The second term prevents the system from explaining everything by making beliefs arbitrarily complicated.
2. Interactive free energy landscape
The dot is the current belief. The target is the sensory input. Moving the belief closer to the target reduces error. But strong prior confidence creates a cost for moving too far from what the model expected.
3. Accuracy and complexity
In variational formulations, free energy is often described as a balance between accuracy and complexity:
$F = \text{complexity} - \text{accuracy}$
Accuracy means the model predicts the data well. Complexity means the posterior beliefs have moved away from the prior. A good explanation should fit the data, but not by inventing an unnecessarily complicated story.
Accuracy
How well does the model predict the observed data?
Complexity
How far did beliefs move from the prior?
Precision
How strongly should errors influence belief updating?
Evidence
How plausible is this model as an explanation of the data?
4. Perception and action
There are two broad ways to reduce free energy. Perception changes beliefs to better explain sensory input. Action changes sensory input so it better matches predictions.
Perception
Update beliefs so predictions better match the world.
beliefs → predictions → lower error
Action
Change the world, or sample the world, so sensations better match predictions.
actions → sensations → lower error
Why this matters for our lab
Free energy gives a shared language for model fitting, predictive coding, active inference and computational psychiatry. In DCM and Variational Laplace, we estimate hidden parameters by optimising a free-energy objective. In active inference, agents choose actions expected to reduce future uncertainty and fulfil preferred states. In psychiatry, altered precision, priors or synaptic gain may change how prediction errors influence perception, action and belief.