First-order optimization and stationarity
Seminar Données et Aléatoire Théorie & Applications
14/11/2024 - 14:00 Guillaume Olikier Salle 106
This talk is a broadly accessible presentation of recent results in first-order optimization; the basic definitions and the results are illustrated with pictures to help the audience gain intuition. Specifically, the talk considers a problem ubiquitous in data science, machine learning, and image processing: minimizing a smooth function over a closed subset. Finding a local minimizer of such a nonconvex problem is generally intractable, and algorithms are only expected to find a stationary point. The first result of the talk is that the simplest algorithm, projected gradient descent, enjoys the strongest stationarity property for the considered problem: it generates a sequence in the set whose accumulation points are “Bouligand stationary”. This desirable property holds even more broadly in presence of favorable geometry: if the closed set can be partitioned into finitely many smooth submanifolds that fit nicely together, cheaper first-order methods also have the same strong stationarity property, up to a suitable modification in the iteration.