-
Frank-Wolfe and friends: a journey into projection-free first-order optimization methods
Authors:
Immanuel. M. Bomze,
Francesco Rinaldi,
Damiano Zeffiro
Abstract:
Invented some 65 years ago in a seminal paper by Marguerite Straus-Frank and Philip Wolfe, the Frank-Wolfe method recently enjoys a remarkable revival, fuelled by the need of fast and reliable first-order optimization methods in Data Science and other relevant application areas. This review tries to explain the success of this approach by illustrating versatility and applicability in a wide range…
▽ More
Invented some 65 years ago in a seminal paper by Marguerite Straus-Frank and Philip Wolfe, the Frank-Wolfe method recently enjoys a remarkable revival, fuelled by the need of fast and reliable first-order optimization methods in Data Science and other relevant application areas. This review tries to explain the success of this approach by illustrating versatility and applicability in a wide range of contexts, combined with an account on recent progress in variants, both improving on the speed and efficiency of this surprisingly simple principle of first-order optimization.
△ Less
Submitted 18 June, 2021;
originally announced June 2021.
-
Fast cluster detection in networks by first-order optimization
Authors:
Immanuel M. Bomze,
Francesco Rinaldi,
Damiano Zeffiro
Abstract:
Cluster detection plays a fundamental role in the analysis of data. In this paper, we focus on the use of s-defective clique models for network-based cluster detection and propose a nonlinear optimization approach that efficiently handles those models in practice. In particular, we introduce an equivalent continuous formulation for the problem under analysis, and we analyze some tailored variants…
▽ More
Cluster detection plays a fundamental role in the analysis of data. In this paper, we focus on the use of s-defective clique models for network-based cluster detection and propose a nonlinear optimization approach that efficiently handles those models in practice. In particular, we introduce an equivalent continuous formulation for the problem under analysis, and we analyze some tailored variants of the Frank-Wolfe algorithm that enable us to quickly find maximal s-defective cliques. The good practical behavior of those algorithmic tools, which is closely connected to their support identification properties, makes them very appealing in practical applications. The reported numerical results clearly show the effectiveness of the proposed approach.
△ Less
Submitted 29 March, 2021;
originally announced March 2021.
-
The $ρ$ Oph region revisited with Gaia EDR3
Authors:
Natalie Grasser,
Sebastian Ratzenböck,
João Alves,
Josefa Großschedl,
Stefan Meingast,
Catherine Zucker,
Alvaro Hacar,
Charles Lada,
Alyssa Goodman,
Marco Lombardi,
John C. Forbes,
Immanuel M. Bomze,
Torsten Möller
Abstract:
Context. Young and embedded stellar populations are important probes of the star formation process. Paradoxically, we have a better census of nearby embedded young populations than the slightly more evolved optically visible young populations. The high accuracy measurements and all-sky coverage of Gaia data are about to change this situation. Aims. This work aims to construct the most complete sam…
▽ More
Context. Young and embedded stellar populations are important probes of the star formation process. Paradoxically, we have a better census of nearby embedded young populations than the slightly more evolved optically visible young populations. The high accuracy measurements and all-sky coverage of Gaia data are about to change this situation. Aims. This work aims to construct the most complete sample to date of YSOs in the $ρ$ Oph region. Methods. We compile a catalog of 1114 Ophiuchus YSOs from the literature and crossmatch it with the Gaia EDR3, Gaia-ESO and APOGEE-2 surveys. We apply a multivariate classification algorithm to this catalog to identify new, co-moving population candidates. Results. We find 191 new high-fidelity YSO candidates in the Gaia EDR3 catalog belonging to the $ρ$ Oph region. The new sources appear to be mainly Class III M-stars and substellar objects and are less extincted than the known members. We find 28 previously unknown sources with disks. The analysis of the proper motion distribution of the entire sample reveals a well-defined bimodality, implying two distinct populations sharing a similar 3D volume. The first population comprises young stars' clusters around the $ρ$ Ophiuchi star and the main Ophiuchus clouds (L1688, L1689, L1709). In contrast, the second population is older ($\sim$ 10 Myr), dispersed, has a distinct proper motion, and is possibly from the Upper Sco group. The two populations are moving away from each other at about 4.1 km/s, and will no longer overlap in about 4 Myr. Finally, we flag 17 sources in the literature as impostors, which are sources that exhibit large deviations from the average distance and proper motion properties of the $ρ$ Oph population. Our results show the importance of accurate 3D space and motion information for improved stellar population analysis. (Abridged)
△ Less
Submitted 8 June, 2021; v1 submitted 28 January, 2021;
originally announced January 2021.
-
Active set complexity of the Away-step Frank-Wolfe Algorithm
Authors:
Immanuel M. Bomze,
Francesco Rinaldi,
Damiano Zeffiro
Abstract:
In this paper, we study active set identification results for the away-step Frank-Wolfe algorithm in different settings. We first prove a local identification property that we apply, in combination with a convergence hypothesis, to get an active set identification result. We then prove, in the nonconvex case, a novel $O(1/\sqrt{k})$ convergence rate result and active set identification for differe…
▽ More
In this paper, we study active set identification results for the away-step Frank-Wolfe algorithm in different settings. We first prove a local identification property that we apply, in combination with a convergence hypothesis, to get an active set identification result. We then prove, in the nonconvex case, a novel $O(1/\sqrt{k})$ convergence rate result and active set identification for different stepsizes (under suitable assumptions on the set of stationary points). By exploiting those results, we also give explicit active set complexity bounds for both strongly convex and nonconvex objectives. While we initially consider the probability simplex as feasible set, in the appendix we show how to adapt some of our results to generic polytopes.
△ Less
Submitted 24 December, 2019;
originally announced December 2019.
-
Hessian barrier algorithms for linearly constrained optimization problems
Authors:
Immanuel M. Bomze,
Panayotis Mertikopoulos,
Werner Schachinger,
Mathias Staudigl
Abstract:
In this paper, we propose an interior-point method for linearly constrained optimization problems (possibly nonconvex). The method - which we call the Hessian barrier algorithm (HBA) - combines a forward Euler discretization of Hessian Riemannian gradient flows with an Armijo backtracking step-size policy. In this way, HBA can be seen as an alternative to mirror descent (MD), and contains as speci…
▽ More
In this paper, we propose an interior-point method for linearly constrained optimization problems (possibly nonconvex). The method - which we call the Hessian barrier algorithm (HBA) - combines a forward Euler discretization of Hessian Riemannian gradient flows with an Armijo backtracking step-size policy. In this way, HBA can be seen as an alternative to mirror descent (MD), and contains as special cases the affine scaling algorithm, regularized Newton processes, and several other iterative solution methods. Our main result is that, modulo a non-degeneracy condition, the algorithm converges to the problem's set of critical points; hence, in the convex case, the algorithm converges globally to the problem's minimum set. In the case of linearly constrained quadratic programs (not necessarily convex), we also show that the method's convergence rate is $\mathcal{O}(1/k^ρ)$ for some $ρ\in(0,1]$ that depends only on the choice of kernel function (i.e., not on the problem's primitives). These theoretical results are validated by numerical experiments in standard non-convex test functions and large-scale traffic assignment problems.
△ Less
Submitted 8 May, 2019; v1 submitted 25 September, 2018;
originally announced September 2018.
-
Extended Trust-Region Problems with One or Two Balls: Exact Copositive and Lagrangian Relaxations
Authors:
I. M. Bomze,
V. Jeyakumar,
G. Li
Abstract:
We establish a geometric condition guaranteeing exact copositive relaxation for the nonconvex quadratic optimization problem under two quadratic and several linear constraints, and present sufficient conditions for global optimality in terms of generalized Karush-Kuhn-Tucker multipliers. The copositive relaxation is tighter than the usual Lagrangian relaxation. We illustrate this by providing a wh…
▽ More
We establish a geometric condition guaranteeing exact copositive relaxation for the nonconvex quadratic optimization problem under two quadratic and several linear constraints, and present sufficient conditions for global optimality in terms of generalized Karush-Kuhn-Tucker multipliers. The copositive relaxation is tighter than the usual Lagrangian relaxation. We illustrate this by providing a whole class of quadratic optimization problems that enjoys exactness of copositive relaxation while the usual Lagrangian duality gap is infinite. Finally, we also provide verifiable conditions under which both the usual Lagrangian relaxation and the copositive relaxation are exact for an extended CDT (two-ball trust-region) problem. Importantly, the sufficient conditions can be verified by solving linear optimization problems.
△ Less
Submitted 2 October, 2017; v1 submitted 26 February, 2017;
originally announced February 2017.
-
New results on the cp rank and related properties of co(mpletely)positive matrices
Authors:
Naomi Shaked-Monderer,
Abraham Berman,
Immanuel M. Bomze,
Florian Jarre,
Werner Schachinger
Abstract:
Copositive and completely positive matrices play an increasingly important role in Applied Mathematics, namely as a key concept for approximating NP-hard optimization problems. The cone of copositive matrices of a given order and the cone of completely positive matrices of the same order are dual to each other with respect to the standard scalar product on the space of symmetric matrices. This pap…
▽ More
Copositive and completely positive matrices play an increasingly important role in Applied Mathematics, namely as a key concept for approximating NP-hard optimization problems. The cone of copositive matrices of a given order and the cone of completely positive matrices of the same order are dual to each other with respect to the standard scalar product on the space of symmetric matrices. This paper establishes some new relations between orthogonal pairs of such matrices lying on the boundary of either cone. As a consequence, we can establish an improvement on the upper bound of the cp-rank of completely positive matrices of general order, and a further improvement for such matrices of order six.
△ Less
Submitted 23 November, 2013; v1 submitted 27 April, 2013;
originally announced May 2013.
-
Improving SDP bounds for minimizing quadratic functions over the l1-ball
Authors:
Immanuel M. Bomze,
Florian Frommlet,
Martin Rubey
Abstract:
In this note, we establish superiority of the so-called copositive bound over a bound suggested by Nesterov for the quadratic problem to minimize a quadratic form over the l1-ball. We illustrate the improvement by simulation results. The copositive bound has the additional advantage that it can be easily extended to the inhomogeneous case of quadratic objectives including a linear term. We also…
▽ More
In this note, we establish superiority of the so-called copositive bound over a bound suggested by Nesterov for the quadratic problem to minimize a quadratic form over the l1-ball. We illustrate the improvement by simulation results. The copositive bound has the additional advantage that it can be easily extended to the inhomogeneous case of quadratic objectives including a linear term. We also indicate some improvements of the eigenvalue bound for the quadratic optimization over the lp-ball with 1<p<2, at least for p close to one.
△ Less
Submitted 22 March, 2005; v1 submitted 9 March, 2005;
originally announced March 2005.
