Non-geodesically-convex optimization in the Wasserstein space
Authors:
Hoang Phuc Hau Luu,
Hanlin Yu,
Bernardo Williams,
Petrus Mikkola,
Marcelo Hartmann,
Kai Puolamäki,
Arto Klami
Abstract:
We study a class of optimization problems in the Wasserstein space (the space of probability measures) where the objective function is \emph{nonconvex} along generalized geodesics. When the regularization term is the negative entropy, the optimization problem becomes a sampling problem where it minimizes the Kullback-Leibler divergence between a probability measure (optimization variable) and a ta…
▽ More
We study a class of optimization problems in the Wasserstein space (the space of probability measures) where the objective function is \emph{nonconvex} along generalized geodesics. When the regularization term is the negative entropy, the optimization problem becomes a sampling problem where it minimizes the Kullback-Leibler divergence between a probability measure (optimization variable) and a target probability measure whose logarithmic probability density is a nonconvex function. We derive multiple convergence insights for a novel {\em semi Forward-Backward Euler scheme} under several nonconvex (and possibly nonsmooth) regimes. Notably, the semi Forward-Backward Euler is just a slight modification of the Forward-Backward Euler whose convergence is -- to our knowledge -- still unknown in our very general non-geodesically-convex setting.
△ Less
Submitted 1 June, 2024;
originally announced June 2024.
Online Stochastic DCA with applications to Principal Component Analysis
Authors:
Hoai An Le Thi,
Hoang Phuc Hau Luu,
Tao Pham Dinh
Abstract:
Stochastic algorithms are well-known for their performance in the era of big data. In convex optimization, stochastic algorithms have been studied in depth and breadth. However, the current body of research on stochastic algorithms for nonsmooth, nonconvex optimization is relatively limited. In this paper, we propose new stochastic algorithms based on DC (Difference of Convex functions) programmin…
▽ More
Stochastic algorithms are well-known for their performance in the era of big data. In convex optimization, stochastic algorithms have been studied in depth and breadth. However, the current body of research on stochastic algorithms for nonsmooth, nonconvex optimization is relatively limited. In this paper, we propose new stochastic algorithms based on DC (Difference of Convex functions) programming and DCA (DC Algorithm) - the backbone of nonconvex, nonsmooth optimization. Since most real-world nonconvex programs fall into the framework of DC programming, our proposed methods can be employed in various situations, in which they confront stochastic nature and nonconvexity simultaneously. The convergence analysis of the proposed algorithms is studied intensively with the help of tools from modern convex analysis and martingale theory. Finally, we study several aspects of the proposed algorithms on an important problem in machine learning: the expected problem in Principal Component Analysis.
△ Less
Submitted 4 August, 2021;
originally announced August 2021.
