Reading

The mandatory reading material for the week is:

• Chapters 2, 3, and 4 in the Prince book.

Chapter 2 serves as a recap of linear models (covered in 02450), while also establishing the notation used throughout the book. You may also optionally read chapter 1.

Exercises

During the exercise session, we will work on:

• Notebook: 1.1 FNN Pen and Paper.ipynb
• Problems (Prince): 3.5, 3.11, 4.1, 4.11

Slides

• 02456 Week 1.pdf

Notes

Week 1 – Neural Networks (02456 DTU)

📚 Course Structure • Weeks 1–8: Lectures (1h) + Exercises (3h) → hands-on learning with notebooks + problems from Prince: Understanding Deep Learning. • Weeks 9–15: Project-based (groups of 3–4). • Evaluation: • Written exam (25%) → multiple choice, no electronic aids. • Project report (75%) → max 4 pages (+ references & code).

✅ Learning Objectives • Understand core deep learning concepts, terminology, and limitations. • Apply & analyze models in exercises and projects. • Plan and execute a deep learning project. • Use PyTorch for GPU programming. • Summarize & communicate results.

⸻

Part 1: The Deep Learning Revolution • AI hype today: Generative AI (transformers, diffusion models). • AI as umbrella: AI → ML → Neural Networks → Deep Learning. • History: • 1956 Dartmouth workshop → defined AI vision. • Neural nets theorized in 40s–90s. • Breakthrough ~2012: Krizhevsky, Sutskever & Hinton win ImageNet (AlexNet). • 2015–16: Deep reinforcement learning (Atari, AlphaGo). • 2017–20: Transformers & diffusion models → modern generative AI. • Why deep learning took off: • Faster computers (GPUs/TPUs). • Bigger datasets (ImageNet etc.). • Better software (PyTorch, TensorFlow). • Key idea: Representation learning. • Traditional ML: manual feature engineering (PCA, SIFT, etc.). • Deep learning: end-to-end feature learning from raw data. • “Deep” = multiple nonlinear transformations → hierarchical representations.

⸻

Part 2: Neural Networks Basics

Supervised Learning • Data D = \{(x_i, y_i)\}, want model y = f_\phi(x). • Learn parameters \phi by minimizing loss L(\phi). • Example: Linear regression, MSE loss.