Frame stacking: Using the last 4 frames to make decisions. Can be used to calculate acceleration and velocities. Can be used for more than just positions.

Optimal value function: The optimal cost given x0 is denoted J*(x0) and is defined as

$J*(x_0) =\min J_π(x_0)$, where π = {µ0,...µN-1}