This is a *theory* course. There will not be any programming assignment. The students will have to read and write rigorous mathematical proofs. This course introduces optimization algorithms for machine learning, in particular first-order convex optimization algorithms, for their scalability with respect to the parameter dimension and sample size. The algorithms this course will cover include gradient descent, mirror descent, proximal gradient methods, the Frank-Wolfe method, and if time allows, stochastic mirror descent. The focus will be non-asymptotic error analysis of these algorithms.

After taking this course, the students are expected to - be familiar with basic concepts in the black-box approach to convex optimization, - be able to read literature on optimization theory, and - be able to choose an appropriate optimization algorithm given a problem.

- The students are expected to be motivated enough to take this course. - The students are expected to be familiar with multivariate calculus, linear algebra, and probability. Knowledge in machine learning or statistics may be helpful but are not necessary.

- Yu. Nesterov. Lectures on Convex Optimization. 2018. - S. Bubeck. Convex Optimization: Algorithms and Complexity. 2015. - A. Beck. First-Order Methods in Optimization. 2017. - G. Lan. First-order and Stochastic Optimization Methods for Machine Learning. 2020. - Lecture notes by A. Nemirovski: https://www2.isye.gatech.edu/~nemirovs/