Machine learning is a fast-growing field and plays a central role in the era of AI and big data with many applications in prediction and data analysis. In particular, optimization formulations and algorithms are key to analysis and design of efficient methods in machine learning problems. This course will focus on the optimization models arising from various machine learning problems and introduce efficient algorithms for solving these models. The course will cover basic convex analysis, model formulation, and fundamental (stochastic) first-order methods, supplemented with recent developments in the related literature.

The students are expected to achieve the following after taking this course: 1. Students will be able to identify the underlying optimization model in different machine learning problems. 2. Students will be able to identify different structures and properties of optimization models in machine learning problems, and how to choose appropriate algorithms based on the problem structure. 3. Students will be able to construct analysis for basic optimization algorithms such as gradient descent, stochastic gradient descent. In addition, students will be able to understand and follow the logic in the analysis for more complicated algorithms. 4. Students will be able to read scientific paper in the related field on their own and summarize the findings and their own understanding, and verify the results in the paper if necessary.

Linear Algebra Multivariate Calculus Probability and basic statistics knowledge. Basic knowledge for optimization Basic knowledge for machine learning (not required but recommended)

1. “Optimization for Machine Learning” (Neural Information Processing series) edited by Suvrit Sra, Sebastian Nowozin, Stephen Wright et al., MIT Press 2012 2. “First-order and Stochastic Optimization Methods for Machine Learning” by Guanghui Lan, Springer 2019 3. “Lectures on Convex Optimization” by Yurii Nesterov, 2018. 4. “Convex Optimization: Algorithms and Complexity” by Sebastien Bubeck. 2015.