QTM 447: Optimization Module

Welcome to the Optimization Module

This website contains asynchronous notes and interactive materials for the Optimization module of QTM 447.

Overview

In this module, we will dive deep into the mathematics and algorithms behind training logistic regression models (binary and multinomial). We will move beyond “calling fit()” to understanding exactly how gradients and Hessians are computed and used to update parameters.

Key Topics

1. Matrix Calculus: Derivatives of vector and matrix functions.
2. Computational Graphs: Visualizing and computing gradients via the chain rule (backpropagation).
3. Logistic Regression: Deriving gradients and Hessians from scratch.
4. Optimization Algorithms: Implementing Gradient Descent and Newton’s Method.
5. Multinomial Logistic Regression: Extending to multi-class classification with “tensor” gradients.

Getting Started

Navigate to the Overview to begin.