Stochastic Online Optimization using Kalman Recursion

We present an analysis of the Extended Kalman Filter (EKF) in a degenerate setting called static. It has been remarked that in this setting the EKF can be seen as a gradient algorithm. Therefore, we study the static EKF as an online optimization algorithm to enrich the link between bayesian statistics and optimization.
We propose a two-phase analysis. First, for Generalized Linear Models, we obtain high probability bounds on the cumulative excess risk, under the assumption that after some time the algorithm is trapped in a small region around the optimum. Second, we prove that « local » assumption for linear and logistic regressions, slightly modifying the algorithm in the logistic setting.
This is a joint work with Olivier Wintenberger.

Vidéo : https://cloud-ljk.imag.fr/index.php/s/eqaKqnxYWtrJitx