1. The dynamical programming algorithm for finite-horizon control

2. PID control

3 The discrete linear quadratic regulator and iterative LQR

4. Direct methods for optimal control

5. Bandit algorithms

6. Bellman’s equations and their relationship to reinforcement learning

7: Eligibility traces

8: Q**-learning and Value-function approximations**