The Bellman Principle

The Bellman Principle says:

The optimal cost-to-go from state $x$ is equal to the minimum of the immediate cost plus the optimal cost-to-go from the next state.

In discrete time, with action $ a $ and transition $ x’ = f(x,a) $, this gives us the Bellman equation:

\[\boxed{ V(x) = \min_a \left[ c(x,a) + V(f(x,a)) \right] }\]

This is a recursive version of minimizing $ \mathcal{F}[P] $ from the Minimum Free-Energy Principle in which we compute the total cost step-by-step.