-
Geometric characterizations of Lipschitz stability for convex optimization problems
Authors:
Tran T. A. Nghia
Abstract:
In this paper, we mainly study tilt stability and Lipschitz stability of convex optimization problems. Our characterizations are geometric and fully computable in many important cases. As a result, we apply our theory to the group Lasso problem and the nuclear norm minimization problem and reveal that the Lipschitz stability of the solution map** in these problems is automatic whenever the solut…
▽ More
In this paper, we mainly study tilt stability and Lipschitz stability of convex optimization problems. Our characterizations are geometric and fully computable in many important cases. As a result, we apply our theory to the group Lasso problem and the nuclear norm minimization problem and reveal that the Lipschitz stability of the solution map** in these problems is automatic whenever the solution map** is single-valued.
△ Less
Submitted 7 February, 2024;
originally announced February 2024.
-
Solution uniqueness of convex optimization problems via the radial cone
Authors:
Jalal Fadili,
Tran T. A. Nghia,
Duy Nhat Phan
Abstract:
In this paper, we mainly study solution uniqueness of some convex optimization problems. Our characterizations of solution uniqueness are in terms of the radial cone. This approach allows us to know when a unique solution is a strong solution or even a tilt-stable one without checking second-order information. Consequently, we apply our theory to low-rank optimization problems. The radial cone is…
▽ More
In this paper, we mainly study solution uniqueness of some convex optimization problems. Our characterizations of solution uniqueness are in terms of the radial cone. This approach allows us to know when a unique solution is a strong solution or even a tilt-stable one without checking second-order information. Consequently, we apply our theory to low-rank optimization problems. The radial cone is fully calculated in this case and numerical experiments show that our characterizations are sharp.
△ Less
Submitted 18 January, 2024;
originally announced January 2024.
-
Geometric characterizations for strong minima with applications to nuclear norm minimization problems
Authors:
Jalal Fadili,
Tran T. A. Nghia,
Duy Nhat Phan
Abstract:
In this paper, we introduce several geometric characterizations for strong minima of optimization problems. Applying these results to nuclear norm minimization problems allows us to obtain new necessary and sufficient quantitative conditions for this important property. Our characterizations for strong minima are weaker than the Restricted Injectivity and Nondegenerate Source Condition, which are…
▽ More
In this paper, we introduce several geometric characterizations for strong minima of optimization problems. Applying these results to nuclear norm minimization problems allows us to obtain new necessary and sufficient quantitative conditions for this important property. Our characterizations for strong minima are weaker than the Restricted Injectivity and Nondegenerate Source Condition, which are usually used to identify solution uniqueness of nuclear norm minimization problems. Consequently, we obtain the minimum (tight) bound on the number of measurements for (strong) exact recovery of low-rank matrices.
△ Less
Submitted 17 August, 2023;
originally announced August 2023.
-
Sharp, strong and unique minimizers for low complexity robust recovery
Authors:
Jalal Fadili,
Tran T. A. Nghia,
Trinh T. T. Tran
Abstract:
In this paper, we show the important roles of sharp minima and strong minima for robust recovery. We also obtain several characterizations of sharp minima for convex regularized optimization problems. Our characterizations are quantitative and verifiable especially for the case of decomposable norm regularized problems including sparsity, group-sparsity, and low-rank convex problems. For group-spa…
▽ More
In this paper, we show the important roles of sharp minima and strong minima for robust recovery. We also obtain several characterizations of sharp minima for convex regularized optimization problems. Our characterizations are quantitative and verifiable especially for the case of decomposable norm regularized problems including sparsity, group-sparsity, and low-rank convex problems. For group-sparsity optimization problems, we show that a unique solution is a strong solution and obtain quantitative characterizations for solution uniqueness.
△ Less
Submitted 9 November, 2021;
originally announced November 2021.
-
Second order optimality conditions for strong local minimizers via subgradient graphical derivative
Authors:
Nguyen Huy Chieu,
Le Van Hien,
Tran T. A. Nghia,
Ha Anh Tuan
Abstract:
This paper is devoted to the study of second order optimality conditions for strong local minimizers in the frameworks of unconstrained and constrained optimization problems in finite dimensions via subgradient graphical derivative. We prove that the positive definiteness of the subgradient graphical derivative of an extended-real-valued lower semicontinuous proper function at a proximal stationar…
▽ More
This paper is devoted to the study of second order optimality conditions for strong local minimizers in the frameworks of unconstrained and constrained optimization problems in finite dimensions via subgradient graphical derivative. We prove that the positive definiteness of the subgradient graphical derivative of an extended-real-valued lower semicontinuous proper function at a proximal stationary point is sufficient for the quadratic growth condition. It is also a necessary condition for the latter property when the function is either subdifferentially continuous, prox-regular, twice epi-differentiable or variationally convex. By applying our results to the $\mathcal{C}^2$-cone reducible constrained programs, we establish no-gap second order optimality conditions for (strong) local minimizers under the metric subregularity constraint qualification. These results extend the classical second order optimality conditions by surpassing the well-known Robinson's constraint qualification. Our approach also highlights the interconnection between the strong metric subregularity of subdifferential and quadratic growth condition in optimization problems.
△ Less
Submitted 13 March, 2019;
originally announced March 2019.
-
On the Q-linear convergence of forward-backward splitting method and uniqueness of optimal solution to Lasso
Authors:
Yunier Bello-Cruz,
G. Li,
T. T. A. Nghia
Abstract:
In this paper, by using tools of second-order variational analysis, we study the popular forward-backward splitting method with Beck-Teboulle's line-search for solving convex optimization problem where the objective function can be split into the sum of a differentiable function and a possible nonsmooth function. We first establish that this method exhibits global convergence to an optimal solutio…
▽ More
In this paper, by using tools of second-order variational analysis, we study the popular forward-backward splitting method with Beck-Teboulle's line-search for solving convex optimization problem where the objective function can be split into the sum of a differentiable function and a possible nonsmooth function. We first establish that this method exhibits global convergence to an optimal solution of the problem (if it exists) without the usual assumption that the gradient of the differentiable function involved is globally Lipschitz continuous. We also obtain the $o(k^{-1})$ complexity for the functional value sequence when this usual assumption is weaken from global Lipschitz continuity to local Lipschitz continuity, improving the existing $\mathcal{O}(k^{-1})$ complexity result. We then derive the local and global Q-linear convergence of the method in terms of both the function value sequence and the iterative sequence, under a general metric subregularity assumption which is automatically satisfied for convex piecewise-linear-quadratic optimization problems. In particular, we provide verifiable sufficient conditions for metric subregularity assumptions, and so, local and global Q-linear convergence of the proposed method for broad structured optimization problems arise in machine learning and signal processing including Poisson linear inverse problem, the partly smooth optimization problems, as well as the $\ell_1$-regularized optimization problems. Our results complement the current literature by providing $Q$-linear convergence result to the forward-backward splitting method under weaker assumptions. Moreover, via this approach, we obtain several full characterizations for the uniqueness of optimal solution to Lasso problem, which covers some recent results in this direction.
△ Less
Submitted 17 June, 2018;
originally announced June 2018.
-
Full stability of general parametric variational systems
Authors:
Boris S. Mordukhovich,
Tran T. A. Nghia,
Dat T. Pham
Abstract:
The paper introduces and studies the notions of Lipschitzian and Hölderian full stability of solutions to three-parametric variational systems described in the generalized equation formalism involving nonsmooth base map**s and partial subgradients of prox-regular functions acting in Hilbert spaces. Employing advanced tools and techniques of second-order variational analysis allows us to establis…
▽ More
The paper introduces and studies the notions of Lipschitzian and Hölderian full stability of solutions to three-parametric variational systems described in the generalized equation formalism involving nonsmooth base map**s and partial subgradients of prox-regular functions acting in Hilbert spaces. Employing advanced tools and techniques of second-order variational analysis allows us to establish complete characterizations of, as well as directly verifiable sufficient conditions for, such full stability notions under mild assumptions. Furthermore, we derive exact formulas and effective quantitative estimates for the corresponding moduli. The obtained results are specified for important classes of variational inequalities and variational conditions in both finite and infinite dimensions.
△ Less
Submitted 20 August, 2017;
originally announced August 2017.
-
Characterization of tilt stability via subgradient graphical derivative with applications to nonlinear programming
Authors:
Nguyen Huy Chieu,
Le Van Hien,
Tran T. A. Nghia
Abstract:
This paper is devoted to the study of tilt stability in finite dimensional optimization via the approach of using the subgradient graphical derivative. We establish a new characterization of tilt-stable local minimizers for a broad class of unconstrained optimization problems in terms of a uniform positive definiteness of the subgradient graphical derivative of the objective function around the po…
▽ More
This paper is devoted to the study of tilt stability in finite dimensional optimization via the approach of using the subgradient graphical derivative. We establish a new characterization of tilt-stable local minimizers for a broad class of unconstrained optimization problems in terms of a uniform positive definiteness of the subgradient graphical derivative of the objective function around the point in question. By applying this result to nonlinear programming under the metric subregularity constraint qualification, we derive a second-order characterization and several new sufficient conditions for tilt stability. In particular, we show that each stationary point of a nonlinear programming problem satisfying the metric subregularity constraint qualification is a tilt-stable local minimizer if the classical strong second-order sufficient condition holds.
△ Less
Submitted 26 May, 2017;
originally announced May 2017.
-
Error Bounds for Parametric Polynomial Systems with Applications to Higher-Order Stability Analysis and Convergence Rates
Authors:
G. Li,
B. S. Mordukhovich,
T. T. A. Nghia,
T. S. Pham
Abstract:
The paper addresses parametric inequality systems described by polynomial functions in finite dimensions, where state-dependent infinite parameter sets are given by finitely many polynomial inequalities and equalities. Such systems can be viewed, in particular, as solution sets to problems of generalized semi-infinite programming with polynomial data. Exploiting the imposed polynomial structure to…
▽ More
The paper addresses parametric inequality systems described by polynomial functions in finite dimensions, where state-dependent infinite parameter sets are given by finitely many polynomial inequalities and equalities. Such systems can be viewed, in particular, as solution sets to problems of generalized semi-infinite programming with polynomial data. Exploiting the imposed polynomial structure together with powerful tools of variational analysis and semialgebraic geometry, we establish a far-going extension of the Łojasiewicz gradient inequality to the general nonsmooth class of supremum marginal functions as well as higher-order (Hölder type) local error bounds results with explicitly calculated exponents. The obtained results are applied to higher-order quantitative stability analysis for various classes of optimization problems including generalized semi-infinite programming with polynomial data, optimization of real polynomials under polynomial matrix inequality constraints, and polynomial second-order cone programming. Other applications provide explicit convergence rate estimates for the cyclic projection algorithm to find common points of convex sets described by matrix polynomial inequalities and for the asymptotic convergence of trajectories of subgradient dynamical systems in semialgebraic settings.
△ Less
Submitted 12 September, 2015;
originally announced September 2015.
-
On the convergence of the forward-backward splitting method with linesearches
Authors:
J. Y. Bello Cruz,
T. T. A. Nghia
Abstract:
In this paper we focus on the convergence analysis of the forward-backward splitting method for solving nonsmooth optimization problems in Hilbert spaces when the objective function is the sum of two convex functions. Assuming that one of the functions is Fréchet differentiable and using two new linesearches, the weak convergence is established without any Lipschitz continuity assumption on the gr…
▽ More
In this paper we focus on the convergence analysis of the forward-backward splitting method for solving nonsmooth optimization problems in Hilbert spaces when the objective function is the sum of two convex functions. Assuming that one of the functions is Fréchet differentiable and using two new linesearches, the weak convergence is established without any Lipschitz continuity assumption on the gradient. Furthermore, we obtain many complexity results of cost values at the iterates when the stepsizes are bounded below by a positive constant.
△ Less
Submitted 14 October, 2016; v1 submitted 11 January, 2015;
originally announced January 2015.
-
Coderivative characterizations of maximal monotonicity for set-valued map**s
Authors:
N. H. Chieu,
G. M. Lee,
B. S. Mordukhovich,
T. T. A. Nghia
Abstract:
This paper concerns generalized differential characterizations of maximal monotone set-valued map**s. Using advanced tools of variational analysis, we establish coderivative criteria for maximal monotonicity of set-valued map**s, which seem to be the first infinitesimal characterizations of maximal monotonicity outside the single-valued case. We also present second-order necessary and sufficie…
▽ More
This paper concerns generalized differential characterizations of maximal monotone set-valued map**s. Using advanced tools of variational analysis, we establish coderivative criteria for maximal monotonicity of set-valued map**s, which seem to be the first infinitesimal characterizations of maximal monotonicity outside the single-valued case. We also present second-order necessary and sufficient conditions for lower-${\mathcal C}^2$ functions to be convex and strongly convex. Examples are provided to illustrate the obtained results and the imposed assumptions.
△ Less
Submitted 1 January, 2015;
originally announced January 2015.
-
Local strong maximal monotonicity and full stability for parametric variational systems
Authors:
B. S. Mordukhovich,
T. T. A. Nghia
Abstract:
The paper introduces and characterizes new notions of Lipschitzian and Hölderian full stability of solutions to general parametric variational systems described via partial subdifferential and normal cone map**s acting in Hilbert spaces. These notions, postulated certain quantitative properties of single-valued localizations of solution maps, are closely related to local strong maximal monotonic…
▽ More
The paper introduces and characterizes new notions of Lipschitzian and Hölderian full stability of solutions to general parametric variational systems described via partial subdifferential and normal cone map**s acting in Hilbert spaces. These notions, postulated certain quantitative properties of single-valued localizations of solution maps, are closely related to local strong maximal monotonicity of associated set-valued map**s. Based on advanced tools of variational analysis and generalized differentiation, we derive verifiable characterizations of the local strong maximal monotonicity and full stability notions under consideration via some positive-definiteness conditions involving second-order constructions of variational analysis. The general results obtained are specified for important classes of variational inequalities and variational conditions in both finite and infinite dimensions.
△ Less
Submitted 6 September, 2014;
originally announced September 2014.
-
Optimal rates of convergence of matrices with applications
Authors:
Heinz H. Bauschke,
J. Y. Bello Cruz,
Tran T. A. Nghia,
Hung M. Phan,
Xianfu Wang
Abstract:
We present a systematic study on the linear convergence rates of the powers of (real or complex) matrices. We derive a characterization when the optimal convergence rate is attained. This characterization is given in terms of semi-simpleness of all eigenvalues having the second-largest modulus after 1. We also provide applications of our general results to analyze the optimal convergence rates for…
▽ More
We present a systematic study on the linear convergence rates of the powers of (real or complex) matrices. We derive a characterization when the optimal convergence rate is attained. This characterization is given in terms of semi-simpleness of all eigenvalues having the second-largest modulus after 1. We also provide applications of our general results to analyze the optimal convergence rates for several relaxed alternating projection methods and the generalized Douglas-Rachford splitting methods for finding the projection on the intersection of two subspaces. Numerical experiments confirm our convergence analysis.
△ Less
Submitted 2 July, 2014;
originally announced July 2014.
-
The rate of linear convergence of the Douglas-Rachford algorithm for subspaces is the cosine of the Friedrichs angle
Authors:
Heinz H. Bauschke,
J. Y. Bello Cruz,
Tran T. A. Nghia,
Hung M. Phan,
Xianfu Wang
Abstract:
The Douglas-Rachford splitting algorithm is a classical optimization method that has found many applications. When specialized to two normal cone operators, it yields an algorithm for finding a point in the intersection of two convex sets. This method for solving feasibility problems has attracted a lot of attention due to its good performance even in nonconvex settings.
In this paper, we consid…
▽ More
The Douglas-Rachford splitting algorithm is a classical optimization method that has found many applications. When specialized to two normal cone operators, it yields an algorithm for finding a point in the intersection of two convex sets. This method for solving feasibility problems has attracted a lot of attention due to its good performance even in nonconvex settings.
In this paper, we consider the Douglas-Rachford algorithm for finding a point in the intersection of two subspaces. We prove that the method converges strongly to the projection of the starting point onto the intersection. Moreover, if the sum of the two subspaces is closed, then the convergence is linear with the rate being the cosine of the Friedrichs angle between the subspaces. Our results improve upon existing results in three ways: First, we identify the location of the limit and thus reveal the method as a best approximation algorithm; second, we quantify the rate of convergence, and third, we carry out our analysis in general (possibly infinite-dimensional) Hilbert space. We also provide various examples as well as a comparison with the classical method of alternating projections.
△ Less
Submitted 21 December, 2013; v1 submitted 18 September, 2013;
originally announced September 2013.
-
Second-order growth, tilt stability, and metric regularity of the subdifferential
Authors:
D. Drusvyatskiy,
B. S. Mordukhovich,
T. T. A. Nghia
Abstract:
This paper sheds new light on several interrelated topics of second-order variational analysis, both in finite and infinite-dimensional settings. We establish new relationships between second-order growth conditions on functions, the basic properties of metric regularity and subregularity of the limiting subdifferential, tilt-stability of local minimizers, and positive-definiteness/semidefinitenes…
▽ More
This paper sheds new light on several interrelated topics of second-order variational analysis, both in finite and infinite-dimensional settings. We establish new relationships between second-order growth conditions on functions, the basic properties of metric regularity and subregularity of the limiting subdifferential, tilt-stability of local minimizers, and positive-definiteness/semidefiniteness properties of the second-order subdifferential (or generalized Hessian).
△ Less
Submitted 27 April, 2013;
originally announced April 2013.
-
Constraint Qualifications and Optimality Conditions for Nonconvex Semi-Infinite and Infinite Programs
Authors:
B. S. Mordukhovich,
T. T. A. Nghia
Abstract:
The paper concerns the study of new classes of nonlinear and nonconvex optimization problems of the so-called infinite programming that are generally defined on infinite-dimensional spaces of decision variables and contain infinitely many of equality and inequality constraints with arbitrary (may not be compact) index sets. These problems reduce to semi-infinite programs in the case of finite-dime…
▽ More
The paper concerns the study of new classes of nonlinear and nonconvex optimization problems of the so-called infinite programming that are generally defined on infinite-dimensional spaces of decision variables and contain infinitely many of equality and inequality constraints with arbitrary (may not be compact) index sets. These problems reduce to semi-infinite programs in the case of finite-dimensional spaces of decision variables. We extend the classical Mangasarian-Fromovitz and Farkas-Minkowski constraint qualifications to such infinite and semi-infinite programs. The new qualification conditions are used for efficient computing the appropriate normal cones to sets of feasible solutions for these programs by employing advanced tools of variational analysis and generalized differentiation. In the further development we derive first-order necessary optimality conditions for infinite and semi-infinite programs, which are new in both finite-dimensional and infinite-dimensional settings.
△ Less
Submitted 22 March, 2011; v1 submitted 20 March, 2011;
originally announced March 2011.
