2.1 Automatic Differentiation

Table of contents

1. Nanograd automatic differentiation framework
2. Finite difference method
3. Data generation
4. Defining and initializing the network
5. Forward pass
6. Training loop
7. Testing your model
8. Further extensions

Exercise a) What is being calculated?

Exercise b) How does the backward function work?

Exercise c) What happens if we run backward again?

Exercise d) Zero gradient

Exercise e) Test correctness of derivatives with the finite difference method