-
Exploiting Structure in Quantum Relative Entropy Programs
Authors:
Kerry He,
James Saunderson,
Hamza Fawzi
Abstract:
Quantum relative entropy programs are convex optimization problems which minimize a linear functional over an affine section of the epigraph of the quantum relative entropy function. Recently, the self-concordance of a natural barrier function was proved for this set. This has opened up the opportunity to use interior-point methods for nonsymmetric cone programs to solve these optimization problem…
▽ More
Quantum relative entropy programs are convex optimization problems which minimize a linear functional over an affine section of the epigraph of the quantum relative entropy function. Recently, the self-concordance of a natural barrier function was proved for this set. This has opened up the opportunity to use interior-point methods for nonsymmetric cone programs to solve these optimization problems. In this paper, we show how common structures arising from applications in quantum information theory can be exploited to improve the efficiency of solving quantum relative entropy programs using interior-point methods. First, we show that the natural barrier function for the epigraph of the quantum relative entropy composed with positive linear operators is optimally self-concordant, even when these linear operators map to singular matrices. Second, we show how we can exploit a catalogue of common structures in these linear operators to compute the inverse Hessian products of the barrier function more efficiently. This step is typically the bottleneck when solving quantum relative entropy programs using interior-point methods, and therefore improving the efficiency of this step can significantly improve the computational performance of the algorithm. We demonstrate how these methods can be applied to important applications in quantum information theory, including quantum key distribution, quantum rate-distortion, quantum channel capacities, and estimating the ground state energy of Hamiltonians. Our numerical results show that these techniques improve computation times by up to several orders of magnitude, and allow previously intractable problems to be solved.
△ Less
Submitted 28 June, 2024;
originally announced July 2024.
-
An Inexact Frank-Wolfe Algorithm for Composite Convex Optimization Involving a Self-Concordant Function
Authors:
Nimita Shinde,
Vishnu Narayanan,
James Saunderson
Abstract:
In this paper, we consider Frank-Wolfe-based algorithms for composite convex optimization problems with objective involving a logarithmically-homogeneous, self-concordant functions. Recent Frank-Wolfe-based methods for this class of problems assume an oracle that returns exact solutions of a linearized subproblem. We relax this assumption and propose a variant of the Frank-Wolfe method with inexac…
▽ More
In this paper, we consider Frank-Wolfe-based algorithms for composite convex optimization problems with objective involving a logarithmically-homogeneous, self-concordant functions. Recent Frank-Wolfe-based methods for this class of problems assume an oracle that returns exact solutions of a linearized subproblem. We relax this assumption and propose a variant of the Frank-Wolfe method with inexact oracle for this class of problems. We show that our inexact variant enjoys similar convergence guarantees to the exact case, while allowing considerably more flexibility in approximately solving the linearized subproblem. In particular, our approach can be applied if the subproblem can be solved prespecified additive error or to prespecified relative error (even though the optimal value of the subproblem may not be uniformly bounded). Furthermore, our approach can also handle the situation where the subproblem is solved via a randomized algorithm that fails with positive probability. Our inexact oracle model is motivated by certain large-scale semidefinite programs where the subproblem reduces to computing an extreme eigenvalue-eigenvector pair, and we demonstrate the practical performance of our algorithm with numerical experiments on problems of this form.
△ Less
Submitted 22 October, 2023;
originally announced October 2023.
-
Efficient Computation of the Quantum Rate-Distortion Function
Authors:
Kerry He,
James Saunderson,
Hamza Fawzi
Abstract:
The quantum rate-distortion function plays a fundamental role in quantum information theory, however there is currently no practical algorithm which can efficiently compute this function to high accuracy for moderate channel dimensions. In this paper, we show how symmetry reduction can significantly simplify common instances of the entanglement-assisted quantum rate-distortion problems. This allow…
▽ More
The quantum rate-distortion function plays a fundamental role in quantum information theory, however there is currently no practical algorithm which can efficiently compute this function to high accuracy for moderate channel dimensions. In this paper, we show how symmetry reduction can significantly simplify common instances of the entanglement-assisted quantum rate-distortion problems. This allows us to better understand the properties of the quantum channels which obtain the optimal rate-distortion trade-off, while also allowing for more efficient computation of the quantum rate-distortion function regardless of the numerical algorithm being used. Additionally, we propose an inexact variant of the mirror descent algorithm to compute the quantum rate-distortion function with provable sublinear convergence rates. We show how this mirror descent algorithm is related to Blahut-Arimoto and expectation-maximization methods previously used to solve similar problems in information theory. Using these techniques, we present the first numerical experiments to compute a multi-qubit quantum rate-distortion function, and show that our proposed algorithm solves faster and to higher accuracy when compared to existing methods.
△ Less
Submitted 2 April, 2024; v1 submitted 27 September, 2023;
originally announced September 2023.
-
A Bregman Proximal Perspective on Classical and Quantum Blahut-Arimoto Algorithms
Authors:
Kerry He,
James Saunderson,
Hamza Fawzi
Abstract:
The Blahut-Arimoto algorithm is a well-known method to compute classical channel capacities and rate-distortion functions. Recent works have extended this algorithm to compute various quantum analogs of these quantities. In this paper, we show how these Blahut-Arimoto algorithms are special instances of mirror descent, which is a type of Bregman proximal method, and a well-studied generalization o…
▽ More
The Blahut-Arimoto algorithm is a well-known method to compute classical channel capacities and rate-distortion functions. Recent works have extended this algorithm to compute various quantum analogs of these quantities. In this paper, we show how these Blahut-Arimoto algorithms are special instances of mirror descent, which is a type of Bregman proximal method, and a well-studied generalization of gradient descent for constrained convex optimization. Using recently developed convex analysis tools, we show how analysis based on relative smoothness and strong convexity recovers known sublinear and linear convergence rates for Blahut-Arimoto algorithms. This Bregman proximal viewpoint allows us to derive related algorithms with similar convergence guarantees to solve problems in information theory for which Blahut-Arimoto-type algorithms are not directly applicable. We apply this framework to compute energy-constrained classical and quantum channel capacities, classical and quantum rate-distortion functions, and approximations of the relative entropy of entanglement, all with provable convergence guarantees.
△ Less
Submitted 7 June, 2024; v1 submitted 7 June, 2023;
originally announced June 2023.
-
Optimal self-concordant barriers for quantum relative entropies
Authors:
Hamza Fawzi,
James Saunderson
Abstract:
Quantum relative entropies are jointly convex functions of two positive definite matrices that generalize the Kullback-Leibler divergence and arise naturally in quantum information theory. In this paper, we prove self-concordance of natural barrier functions for the epigraphs of various quantum relative entropies and divergences. Furthermore we show that these barriers have optimal barrier paramet…
▽ More
Quantum relative entropies are jointly convex functions of two positive definite matrices that generalize the Kullback-Leibler divergence and arise naturally in quantum information theory. In this paper, we prove self-concordance of natural barrier functions for the epigraphs of various quantum relative entropies and divergences. Furthermore we show that these barriers have optimal barrier parameter. These barriers allow convex optimization problems involving quantum relative entropies to be directly solved using interior point methods for non-symmetric cones, avoiding the approximations and lifting techniques used in previous approaches. More generally, we establish the self-concordance of natural barriers for various closed convex cones related to the noncommutative perspectives of operator concave functions, and show that the resulting barrier parameters are optimal.
△ Less
Submitted 19 February, 2023; v1 submitted 9 May, 2022;
originally announced May 2022.
-
Memory-Efficient Approximation Algorithms for Max-k-Cut and Correlation Clustering
Authors:
Nimita Shinde,
Vishnu Narayanan,
James Saunderson
Abstract:
Max-k-Cut and correlation clustering are fundamental graph partitioning problems. For a graph with G=(V,E) with n vertices, the methods with the best approximation guarantees for Max-k-Cut and the Max-Agree variant of correlation clustering involve solving SDPs with $O(n^2)$ variables and constraints. Large-scale instances of SDPs, thus, present a memory bottleneck. In this paper, we develop simpl…
▽ More
Max-k-Cut and correlation clustering are fundamental graph partitioning problems. For a graph with G=(V,E) with n vertices, the methods with the best approximation guarantees for Max-k-Cut and the Max-Agree variant of correlation clustering involve solving SDPs with $O(n^2)$ variables and constraints. Large-scale instances of SDPs, thus, present a memory bottleneck. In this paper, we develop simple polynomial-time Gaussian sampling-based algorithms for these two problems that use $O(n+|E|)$ memory and nearly achieve the best existing approximation guarantees. For dense graphs arriving in a stream, we eliminate the dependence on $|E|$ in the storage complexity at the cost of a slightly worse approximation ratio by combining our approach with sparsification.
△ Less
Submitted 27 October, 2021; v1 submitted 2 October, 2021;
originally announced October 2021.
-
A convex form that is not a sum of squares
Authors:
James Saunderson
Abstract:
Every convex homogeneous polynomial (or form) is nonnegative. Blekherman has shown that there exist convex forms that are not sums of squares via a nonconstructive argument. We provide an explicit example of a convex form of degree four in 272 variables that is not a sum of squares. The form is related to the Cauchy-Schwarz inequality over the octonions. The proof uses symmetry reduction together…
▽ More
Every convex homogeneous polynomial (or form) is nonnegative. Blekherman has shown that there exist convex forms that are not sums of squares via a nonconstructive argument. We provide an explicit example of a convex form of degree four in 272 variables that is not a sum of squares. The form is related to the Cauchy-Schwarz inequality over the octonions. The proof uses symmetry reduction together with the fact (due to Blekherman) that forms of even degree, that are near-constant on the unit sphere, are convex. Using this same connection, we obtain improved bounds on the approximation quality achieved by the basic sum-of-squares relaxation for optimizing quaternary quartic forms on the sphere.
△ Less
Submitted 11 June, 2021; v1 submitted 18 May, 2021;
originally announced May 2021.
-
Hyperbolicity cones are amenable
Authors:
Bruno F. Lourenço,
Vera Roshchina,
James Saunderson
Abstract:
Amenability is a notion of facial exposedness for convex cones that is stronger than being facially dual complete (or "nice") which is, in turn, stronger than merely being facially exposed. Hyperbolicity cones are a family of algebraically structured closed convex cones that contain all spectrahedral cones (linear sections of positive semidefinite cones) as special cases. It is known that all spec…
▽ More
Amenability is a notion of facial exposedness for convex cones that is stronger than being facially dual complete (or "nice") which is, in turn, stronger than merely being facially exposed. Hyperbolicity cones are a family of algebraically structured closed convex cones that contain all spectrahedral cones (linear sections of positive semidefinite cones) as special cases. It is known that all spectrahedral cones are amenable. We establish that all hyperbolicity cones are amenable. As part of the argument, we show that any face of a hyperbolicity cone is a hyperbolicity cone. As a corollary, we show that the intersection of two hyperbolicity cones, not necessarily sharing a common relative interior point, is a hyperbolicity cone.
△ Less
Submitted 2 April, 2023; v1 submitted 12 February, 2021;
originally announced February 2021.
-
Amenable cones are particularly nice
Authors:
Bruno F. Lourenço,
Vera Roshchina,
James Saunderson
Abstract:
Amenability is a geometric property of convex cones that is stronger than facial exposedness and assists in the study of error bounds for conic feasibility problems. In this paper we establish numerous properties of amenable cones, and investigate the relationships between amenability and other properties of convex cones, such as niceness and projectional exposure.
We show that the amenability o…
▽ More
Amenability is a geometric property of convex cones that is stronger than facial exposedness and assists in the study of error bounds for conic feasibility problems. In this paper we establish numerous properties of amenable cones, and investigate the relationships between amenability and other properties of convex cones, such as niceness and projectional exposure.
We show that the amenability of a compact slice of a closed convex cone is equivalent to the amenability of the cone, and prove several results on the preservation of amenability under intersections and other convex operations. It then follows that homogeneous, doubly nonnegative and other cones that can be represented as slices of the cone of positive semidefinite matrices, are amenable.
It is known that projectionally exposed cones are amenable and that amenable cones are nice, however the converse statements have been open questions. We construct an example of a four-dimensional cone that is nice but not amenable. We also show that amenable cones are projectionally exposed in dimensions up to and including four.
We conclude with a discussion on open problems related to facial structure of convex sets that we came across in the course of this work, but were not able to fully resolve.
△ Less
Submitted 21 April, 2022; v1 submitted 16 November, 2020;
originally announced November 2020.
-
Terracini Convexity
Authors:
James Saunderson,
Venkat Chandrasekaran
Abstract:
We present a generalization of the notion of neighborliness to non-polyhedral convex cones. Although a definition of neighborliness is available in the non-polyhedral case in the literature, it is fairly restrictive as it requires all the low-dimensional faces to be polyhedral. Our approach is more flexible and includes, for example, the cone of positive-semidefinite matrices as a special case (th…
▽ More
We present a generalization of the notion of neighborliness to non-polyhedral convex cones. Although a definition of neighborliness is available in the non-polyhedral case in the literature, it is fairly restrictive as it requires all the low-dimensional faces to be polyhedral. Our approach is more flexible and includes, for example, the cone of positive-semidefinite matrices as a special case (this cone is not neighborly in general). We term our generalization Terracini convexity due to its conceptual similarity with the conclusion of Terracini's lemma from algebraic geometry. Polyhedral cones are Terracini convex if and only if they are neighborly. More broadly, we derive many families of non-polyhedral Terracini convex cones based on neighborly cones, linear images of cones of positive semidefinite matrices, and derivative relaxations of Terracini convex hyperbolicity cones. As a demonstration of the utility of our framework in the non-polyhedral case, we give a characterization based on Terracini convexity of the tightness of semidefinite relaxations for certain inverse problems.
△ Less
Submitted 11 January, 2022; v1 submitted 2 October, 2020;
originally announced October 2020.
-
Memory-efficient structured convex optimization via extreme point sampling
Authors:
Nimita Shinde,
Vishnu Narayanan,
James Saunderson
Abstract:
Memory is a key computational bottleneck when solving large-scale convex optimization problems such as semidefinite programs (SDPs). In this paper, we focus on the regime in which storing an $n\times n$ matrix decision variable is prohibitive. To solve SDPs in this regime, we develop a randomized algorithm that returns a random vector whose covariance matrix is near-feasible and near-optimal for t…
▽ More
Memory is a key computational bottleneck when solving large-scale convex optimization problems such as semidefinite programs (SDPs). In this paper, we focus on the regime in which storing an $n\times n$ matrix decision variable is prohibitive. To solve SDPs in this regime, we develop a randomized algorithm that returns a random vector whose covariance matrix is near-feasible and near-optimal for the SDP. We show how to develop such an algorithm by modifying the Frank-Wolfe algorithm to systematically replace the matrix iterates with random vectors. As an application of this approach, we show how to implement the Goemans-Williamson approximation algorithm for \textsc{MaxCut} using $\mathcal{O}(n)$ memory in addition to the memory required to store the problem instance. We then extend our approach to deal with a broader range of structured convex optimization problems, replacing decision variables with random extreme points of the feasible region.
△ Less
Submitted 24 August, 2021; v1 submitted 18 June, 2020;
originally announced June 2020.
-
Lifting for Simplicity: Concise Descriptions of Convex Sets
Authors:
Hamza Fawzi,
João Gouveia,
Pablo A. Parrilo,
James Saunderson,
Rekha R. Thomas
Abstract:
This paper presents a selected tour through the theory and applications of lifts of convex sets. A lift of a convex set is a higher-dimensional convex set that projects onto the original set. Many convex sets have lifts that are dramatically simpler to describe than the original set. Finding such simple lifts has significant algorithmic implications, particularly for optimization problems. We cons…
▽ More
This paper presents a selected tour through the theory and applications of lifts of convex sets. A lift of a convex set is a higher-dimensional convex set that projects onto the original set. Many convex sets have lifts that are dramatically simpler to describe than the original set. Finding such simple lifts has significant algorithmic implications, particularly for optimization problems. We consider both the classical case of polyhedral lifts, described by linear inequalities, as well as spectrahedral lifts, defined by linear matrix inequalities, with a focus on recent developments related to spectrahedral lifts.
Given a convex set, ideally we would either like to find a (low-complexity) polyhedral or spectrahedral lift, or find an obstruction proving that no such lift is possible. To this end, we explain the connection between the existence of lifts of a convex set and certain structured factorizations of its associated slack operator. Based on this characterization, we describe a uniform approach, via sums of squares, to the construction of spectrahedral lifts of convex sets and illustrate the method on several families of examples. Finally, we discuss two flavors of obstruction to the existence of lifts: one related to facial structure, and the other related to algebraic properties of the set in question.
Rather than being exhaustive, our aim is to illustrate the richness of the area. We touch on a range of different topics related to the existence of lifts, and present many examples of lifts from different areas of mathematics and its applications.
△ Less
Submitted 18 November, 2021; v1 submitted 22 February, 2020;
originally announced February 2020.
-
Spectral Polyhedra
Authors:
Raman Sanyal,
James Saunderson
Abstract:
A "spectral convex set" is a collection of symmetric matrices whose range of eigenvalues form a symmetric convex set. Spectral convex sets generalize the Schur-Horn orbitopes studied by Sanyal-Sottile-Sturmfels (2011). We study this class of convex bodies, which is closed under intersections, polarity, and Minkowski sums. We describe orbits of faces and give a formula for their Steiner polynomials…
▽ More
A "spectral convex set" is a collection of symmetric matrices whose range of eigenvalues form a symmetric convex set. Spectral convex sets generalize the Schur-Horn orbitopes studied by Sanyal-Sottile-Sturmfels (2011). We study this class of convex bodies, which is closed under intersections, polarity, and Minkowski sums. We describe orbits of faces and give a formula for their Steiner polynomials. We then focus on spectral polyhedra. We prove that spectral polyhedra are spectrahedra and give small representations as spectrahedral shadows. We close with observations and questions regarding hyperbolicity cones, polar convex bodies, and spectral zonotopes.
△ Less
Submitted 25 April, 2022; v1 submitted 13 January, 2020;
originally announced January 2020.
-
Certifying polynomial nonnegativity via hyperbolic optimization
Authors:
James Saunderson
Abstract:
We describe a new approach to certifying the global nonnegativity of multivariate polynomials by solving hyperbolic optimization problems---a class of convex optimization problems that generalize semidefinite programs. We show how to produce families of nonnegative polynomials (which we call hyperbolic certificates of nonnegativity) from any hyperbolic polynomial. We investigate the pairs $(n,d)$…
▽ More
We describe a new approach to certifying the global nonnegativity of multivariate polynomials by solving hyperbolic optimization problems---a class of convex optimization problems that generalize semidefinite programs. We show how to produce families of nonnegative polynomials (which we call hyperbolic certificates of nonnegativity) from any hyperbolic polynomial. We investigate the pairs $(n,d)$ for which there is a hyperbolic polynomial of degree $d$ in $n$ variables such that an associated hyperbolic certificate of nonnegativity is not a sum of squares. If $d\geq 4$ we show that this occurs whenever $n\geq 4$. In the degree three case, we find an explicit hyperbolic cubic in $43$ variables that gives hyperbolic certificates that are not sums of squares. As a corollary, we obtain the first known hyperbolic cubic no power of which has a definite determinantal representation. Our approach also allows us to show that, given a cubic $p$, and a direction $e$, the decision problem "Is $p$ hyperbolic with respect to $e$?" is co-NP hard.
△ Less
Submitted 4 October, 2019; v1 submitted 31 March, 2019;
originally announced April 2019.
-
Limitations on the expressive power of convex cones without long chains of faces
Authors:
James Saunderson
Abstract:
A convex optimization problem in conic form involves minimizing a linear functional over the intersection of a convex cone and an affine subspace. In some cases, it is possible to replace a conic formulation using a certain cone, with a 'lifted' conic formulation using another cone that is higher-dimensional, but simpler, in some sense. One situation in which this can be computationally advantageo…
▽ More
A convex optimization problem in conic form involves minimizing a linear functional over the intersection of a convex cone and an affine subspace. In some cases, it is possible to replace a conic formulation using a certain cone, with a 'lifted' conic formulation using another cone that is higher-dimensional, but simpler, in some sense. One situation in which this can be computationally advantageous is when the higher-dimensional cone is a Cartesian product of many 'low-complexity' cones, such as second-order cones, or small positive semidefinite cones.
This paper studies obstructions to a convex cone having a lifted representation with such a product structure. The main result says that whenever a convex cone has a certain neighborliness property, then it does not have a lifted representation using a finite product of cones, each of which has only short chains of faces. This is a generalization of recent work of Averkov ('Optimal size of linear matrix inequalities in semidefinite approaches to polynomial optimization', SIAM J. Appl. Alg. Geom., Vol. 3, No. 1, 128--151, 2019) which considers only lifted representations using products of positive semidefinite cones of bounded size. Among the consequences of the main result is that various cones related to nonnegative polynomials do not have lifted representations using products of 'low-complexity' cones, such as smooth cones, the exponential cone, and cones defined by hyperbolic polynomials of low degree.
△ Less
Submitted 5 August, 2019; v1 submitted 17 February, 2019;
originally announced February 2019.
-
A Projection Method for Metric-Constrained Optimization
Authors:
Nate Veldt,
David Gleich,
Anthony Wirth,
James Saunderson
Abstract:
We outline a new approach for solving optimization problems which enforce triangle inequalities on output variables. We refer to this as metric-constrained optimization, and give several examples where problems of this form arise in machine learning applications and theoretical approximation algorithms for graph clustering. Although these problem are interesting from a theoretical perspective, the…
▽ More
We outline a new approach for solving optimization problems which enforce triangle inequalities on output variables. We refer to this as metric-constrained optimization, and give several examples where problems of this form arise in machine learning applications and theoretical approximation algorithms for graph clustering. Although these problem are interesting from a theoretical perspective, they are challenging to solve in practice due to the high memory requirement of black-box solvers. In order to address this challenge we first prove that the metric-constrained linear program relaxation of correlation clustering is equivalent to a special case of the metric nearness problem. We then developed a general solver for metric-constrained linear and quadratic programs by generalizing and improving a simple projection algorithm originally developed for metric nearness. We give several novel approximation guarantees for using our framework to find lower bounds for optimal solutions to several challenging graph clustering problems. We also demonstrate the power of our framework by solving optimizing problems involving up to 10^{8} variables and 10^{11} constraints.
△ Less
Submitted 5 June, 2018;
originally announced June 2018.
-
Competitive Online Algorithms for Resource Allocation over the Positive Semidefinite Cone
Authors:
Reza Eghbali,
James Saunderson,
Maryam Fazel
Abstract:
We consider a new and general online resource allocation problem, where the goal is to maximize a function of a positive semidefinite (PSD) matrix with a scalar budget constraint. The problem data arrives online, and the algorithm needs to make an irrevocable decision at each step. Of particular interest are classic experiment design problems in the online setting, with the algorithm deciding whet…
▽ More
We consider a new and general online resource allocation problem, where the goal is to maximize a function of a positive semidefinite (PSD) matrix with a scalar budget constraint. The problem data arrives online, and the algorithm needs to make an irrevocable decision at each step. Of particular interest are classic experiment design problems in the online setting, with the algorithm deciding whether to allocate budget to each experiment as new experiments become available sequentially.
We analyze two greedy primal-dual algorithms and provide bounds on their competitive ratios. Our analysis relies on a smooth surrogate of the objective function that needs to satisfy a new diminishing returns (PSD-DR) property (that its gradient is order-reversing with respect to the PSD cone). Using the representation for monotone maps on the PSD cone given by Löwner's theorem, we obtain a convex parametrization of the family of functions satisfying PSD-DR. We then formulate a convex optimization problem to directly optimize our competitive ratio bound over this set. This design problem can be solved offline before the data start arriving. The online algorithm that uses the designed smoothing is tailored to the given cost function, and enjoys a competitive ratio at least as good as our optimized bound. We provide examples of computing the smooth surrogate for D-optimal and A-optimal experiment design, and demonstrate the performance of the custom-designed algorithm.
△ Less
Submitted 12 June, 2018; v1 submitted 5 February, 2018;
originally announced February 2018.
-
Improving Efficiency and Scalability of Sum of Squares Optimization: Recent Advances and Limitations
Authors:
Amir Ali Ahmadi,
Georgina Hall,
Antonis Papachristodoulou,
James Saunderson,
Yang Zheng
Abstract:
It is well-known that any sum of squares (SOS) program can be cast as a semidefinite program (SDP) of a particular structure and that therein lies the computational bottleneck for SOS programs, as the SDPs generated by this procedure are large and costly to solve when the polynomials involved in the SOS programs have a large number of variables and degree. In this paper, we review SOS optimization…
▽ More
It is well-known that any sum of squares (SOS) program can be cast as a semidefinite program (SDP) of a particular structure and that therein lies the computational bottleneck for SOS programs, as the SDPs generated by this procedure are large and costly to solve when the polynomials involved in the SOS programs have a large number of variables and degree. In this paper, we review SOS optimization techniques and present two new methods for improving their computational efficiency. The first method leverages the sparsity of the underlying SDP to obtain computational speed-ups. Further improvements can be obtained if the coefficients of the polynomials that describe the problem have a particular sparsity pattern, called chordal sparsity. The second method bypasses semidefinite programming altogether and relies instead on solving a sequence of more tractable convex programs, namely linear and second order cone programs. This opens up the question as to how well one can approximate the cone of SOS polynomials by second order representable cones. In the last part of the paper, we present some recent negative results related to this question.
△ Less
Submitted 3 October, 2017;
originally announced October 2017.
-
A spectrahedral representation of the first derivative relaxation of the positive semidefinite cone
Authors:
James Saunderson
Abstract:
If $X$ is an $n\times n$ symmetric matrix, then the directional derivative of $X \mapsto \det(X)$ in the direction $I$ is the elementary symmetric polynomial of degree $n-1$ in the eigenvalues of $X$. This is a polynomial in the entries of $X$ with the property that it is hyperbolic with respect to the direction $I$. The corresponding hyperbolicity cone is a relaxation of the positive semidefinite…
▽ More
If $X$ is an $n\times n$ symmetric matrix, then the directional derivative of $X \mapsto \det(X)$ in the direction $I$ is the elementary symmetric polynomial of degree $n-1$ in the eigenvalues of $X$. This is a polynomial in the entries of $X$ with the property that it is hyperbolic with respect to the direction $I$. The corresponding hyperbolicity cone is a relaxation of the positive semidefinite (PSD) cone known as the first derivative relaxation (or Renegar derivative) of the PSD cone. A spectrahedal cone is a convex cone that has a representation as the intersection of a subspace with the cone of PSD matrices in some dimension. We show that the first derivative relaxation of the PSD cone is a spectrahedral cone, and give an explicit spectrahedral description of size $\binom{n+1}{2}-1$. The construction provides a new explicit example of a hyperbolicity cone that is also a spectrahedron. This is consistent with the generalized Lax conjecture, which conjectures that every hyperbolicity cone is a spectrahedron.
△ Less
Submitted 7 January, 2018; v1 submitted 28 July, 2017;
originally announced July 2017.
-
Semidefinite approximations of the matrix logarithm
Authors:
Hamza Fawzi,
James Saunderson,
Pablo A. Parrilo
Abstract:
The matrix logarithm, when applied to Hermitian positive definite matrices, is concave with respect to the positive semidefinite order. This operator concavity property leads to numerous concavity and convexity results for other matrix functions, many of which are of importance in quantum information theory. In this paper we show how to approximate the matrix logarithm with functions that preserve…
▽ More
The matrix logarithm, when applied to Hermitian positive definite matrices, is concave with respect to the positive semidefinite order. This operator concavity property leads to numerous concavity and convexity results for other matrix functions, many of which are of importance in quantum information theory. In this paper we show how to approximate the matrix logarithm with functions that preserve operator concavity and can be described using the feasible regions of semidefinite optimization problems of fairly small size. Such approximations allow us to use off-the-shelf semidefinite optimization solvers for convex optimization problems involving the matrix logarithm and related functions, such as the quantum relative entropy. The basic ingredients of our approach apply, beyond the matrix logarithm, to functions that are operator concave and operator monotone. As such, we introduce strategies for constructing semidefinite approximations that we expect will be useful, more generally, for studying the approximation power of functions with small semidefinite representations.
△ Less
Submitted 15 March, 2018; v1 submitted 2 May, 2017;
originally announced May 2017.
-
Symmetric Sums of Squares over $k$-Subset Hypercubes
Authors:
Annie Raymond,
James Saunderson,
Mohit Singh,
Rekha R. Thomas
Abstract:
We consider the problem of finding sum of squares (sos) expressions to establish the non-negativity of a symmetric polynomial over a discrete hypercube whose coordinates are indexed by the $k$-element subsets of $[n]$. For simplicity, we focus on the case $k=2$, but our results extend naturally to all values of $k \geq 2$. We develop a variant of the Gatermann-Parrilo symmetry-reduction method tai…
▽ More
We consider the problem of finding sum of squares (sos) expressions to establish the non-negativity of a symmetric polynomial over a discrete hypercube whose coordinates are indexed by the $k$-element subsets of $[n]$. For simplicity, we focus on the case $k=2$, but our results extend naturally to all values of $k \geq 2$. We develop a variant of the Gatermann-Parrilo symmetry-reduction method tailored to our setting that allows for several simplifications and a connection to flag algebras.
We show that every symmetric polynomial that has a sos expression of a fixed degree also has a succinct sos expression whose size depends only on the degree and not on the number of variables. Our method bypasses much of the technical difficulties needed to apply the Gatermann-Parrilo method, and offers flexibility in obtaining succinct sos expressions that are combinatorially meaningful. As a byproduct of our results, we arrive at a natural representation-theoretic justification for the concept of flags as introduced by Razborov in his flag algebra calculus. Furthermore, this connection exposes a family of non-negative polynomials that cannot be certified with any fixed set of flags, answering a question of Razborov in the context of our finite setting.
△ Less
Submitted 6 August, 2016; v1 submitted 17 June, 2016;
originally announced June 2016.
-
Lieb's concavity theorem, matrix geometric means, and semidefinite optimization
Authors:
Hamza Fawzi,
James Saunderson
Abstract:
A famous result of Lieb establishes that the map $(A,B) \mapsto \text{tr}\left[K^* A^{1-t} K B^t\right]$ is jointly concave in the pair $(A,B)$ of positive definite matrices, where $K$ is a fixed matrix and $t \in [0,1]$. In this paper we show that Lieb's function admits an explicit semidefinite programming formulation for any rational $t \in [0,1]$. Our construction makes use of a semidefinite fo…
▽ More
A famous result of Lieb establishes that the map $(A,B) \mapsto \text{tr}\left[K^* A^{1-t} K B^t\right]$ is jointly concave in the pair $(A,B)$ of positive definite matrices, where $K$ is a fixed matrix and $t \in [0,1]$. In this paper we show that Lieb's function admits an explicit semidefinite programming formulation for any rational $t \in [0,1]$. Our construction makes use of a semidefinite formulation of weighted matrix geometric means. We provide an implementation of our constructions in Matlab.
△ Less
Submitted 13 April, 2020; v1 submitted 10 December, 2015;
originally announced December 2015.
-
Sparse sum-of-squares certificates on finite abelian groups
Authors:
Hamza Fawzi,
James Saunderson,
Pablo A. Parrilo
Abstract:
Let G be a finite abelian group. This paper is concerned with nonnegative functions on G that are sparse with respect to the Fourier basis. We establish combinatorial conditions on subsets S and T of Fourier basis elements under which nonnegative functions with Fourier support S are sums of squares of functions with Fourier support T. Our combinatorial condition involves constructing a chordal cov…
▽ More
Let G be a finite abelian group. This paper is concerned with nonnegative functions on G that are sparse with respect to the Fourier basis. We establish combinatorial conditions on subsets S and T of Fourier basis elements under which nonnegative functions with Fourier support S are sums of squares of functions with Fourier support T. Our combinatorial condition involves constructing a chordal cover of a graph related to G and S (the Cayley graph Cay($\hat{G}$,S)) with maximal cliques related to T. Our result relies on two main ingredients: the decomposition of sparse positive semidefinite matrices with a chordal sparsity pattern, as well as a simple but key observation exploiting the structure of the Fourier basis elements of G.
We apply our general result to two examples. First, in the case where $G = \mathbb{Z}_2^n$, by constructing a particular chordal cover of the half-cube graph, we prove that any nonnegative quadratic form in n binary variables is a sum of squares of functions of degree at most $\lceil n/2 \rceil$, establishing a conjecture of Laurent. Second, we consider nonnegative functions of degree d on $\mathbb{Z}_N$ (when d divides N). By constructing a particular chordal cover of the d'th power of the N-cycle, we prove that any such function is a sum of squares of functions with at most $3d\log(N/d)$ nonzero Fourier coefficients. Dually this shows that a certain cyclic polytope in $\mathbb{R}^{2d}$ with N vertices can be expressed as a projection of a section of the cone of psd matrices of size $3d\log(N/d)$. Putting $N=d^2$ gives a family of polytopes $P_d \subset \mathbb{R}^{2d}$ with LP extension complexity $\text{xc}_{LP}(P_d) = Ω(d^2)$ and SDP extension complexity $\text{xc}_{PSD}(P_d) = O(d\log(d))$. To the best of our knowledge, this is the first explicit family of polytopes in increasing dimensions where $\text{xc}_{PSD}(P_d) = o(\text{xc}_{LP}(P_d))$.
△ Less
Submitted 3 March, 2015;
originally announced March 2015.
-
A convex solution to Psiaki's first joint attitude and spin-rate estimation problem
Authors:
James Saunderson,
Pablo A. Parrilo,
Alan S. Willsky
Abstract:
We consider the problem of jointly estimating the attitude and spin-rate of a spinning spacecraft. Psiaki (J. Astronautical Sci., 57(1-2):73--92, 2009) has formulated a family of optimization problems that generalize the classical least-squares attitude estimation problem, known as Wahba's problem, to the case of a spinning spacecraft. If the rotation axis is fixed and known, but the spin-rate is…
▽ More
We consider the problem of jointly estimating the attitude and spin-rate of a spinning spacecraft. Psiaki (J. Astronautical Sci., 57(1-2):73--92, 2009) has formulated a family of optimization problems that generalize the classical least-squares attitude estimation problem, known as Wahba's problem, to the case of a spinning spacecraft. If the rotation axis is fixed and known, but the spin-rate is unknown (such as for nutation-damped spin-stabilized spacecraft) we show that Psiaki's problem can be reformulated exactly as a type of tractable convex optimization problem called a semidefinite optimization problem. This reformulation allows us to globally solve the problem using standard numerical routines for semidefinite optimization. It also provides a natural semidefinite relaxation-based approach to more complicated variations on the problem.
△ Less
Submitted 10 October, 2014;
originally announced October 2014.
-
Equivariant semidefinite lifts of regular polygons
Authors:
Hamza Fawzi,
James Saunderson,
Pablo A. Parrilo
Abstract:
Given a polytope P in $\mathbb{R}^n$, we say that P has a positive semidefinite lift (psd lift) of size d if one can express P as the linear projection of an affine slice of the positive semidefinite cone $\mathbf{S}^d_+$. If a polytope P has symmetry, we can consider equivariant psd lifts, i.e. those psd lifts that respect the symmetry of P. One of the simplest families of polytopes with interest…
▽ More
Given a polytope P in $\mathbb{R}^n$, we say that P has a positive semidefinite lift (psd lift) of size d if one can express P as the linear projection of an affine slice of the positive semidefinite cone $\mathbf{S}^d_+$. If a polytope P has symmetry, we can consider equivariant psd lifts, i.e. those psd lifts that respect the symmetry of P. One of the simplest families of polytopes with interesting symmetries are regular polygons in the plane, which have played an important role in the study of linear programming lifts (or extended formulations). In this paper we study equivariant psd lifts of regular polygons. We first show that the standard Lasserre/sum-of-squares hierarchy for the regular N-gon requires exactly ceil(N/4) iterations and thus yields an equivariant psd lift of size linear in N. In contrast we show that one can construct an equivariant psd lift of the regular 2^n-gon of size 2n-1, which is exponentially smaller than the psd lift of the sum-of-squares hierarchy. Our construction relies on finding a sparse sum-of-squares certificate for the facet-defining inequalities of the regular 2^n-gon, i.e., one that only uses a small (logarithmic) number of monomials. Since any equivariant LP lift of the regular 2^n-gon must have size 2^n, this gives the first example of a polytope with an exponential gap between sizes of equivariant LP lifts and equivariant psd lifts. Finally we prove that our construction is essentially optimal by showing that any equivariant psd lift of the regular N-gon must have size at least logarithmic in N.
△ Less
Submitted 15 September, 2014;
originally announced September 2014.
-
Semidefinite descriptions of the convex hull of rotation matrices
Authors:
James Saunderson,
Pablo A. Parrilo,
Alan S. Willsky
Abstract:
We study the convex hull of $SO(n)$, thought of as the set of $n\times n$ orthogonal matrices with unit determinant, from the point of view of semidefinite programming. We show that the convex hull of $SO(n)$ is doubly spectrahedral, i.e. both it and its polar have a description as the intersection of a cone of positive semidefinite matrices with an affine subspace. Our spectrahedral representatio…
▽ More
We study the convex hull of $SO(n)$, thought of as the set of $n\times n$ orthogonal matrices with unit determinant, from the point of view of semidefinite programming. We show that the convex hull of $SO(n)$ is doubly spectrahedral, i.e. both it and its polar have a description as the intersection of a cone of positive semidefinite matrices with an affine subspace. Our spectrahedral representations are explicit, and are of minimum size, in the sense that there are no smaller spectrahedral representations of these convex bodies.
△ Less
Submitted 19 March, 2014;
originally announced March 2014.
-
Equivariant semidefinite lifts and sum-of-squares hierarchies
Authors:
Hamza Fawzi,
James Saunderson,
Pablo A. Parrilo
Abstract:
A central question in optimization is to maximize (or minimize) a linear function over a given polytope P. To solve such a problem in practice one needs a concise description of the polytope P. In this paper we are interested in representations of P using the positive semidefinite cone: a positive semidefinite lift (psd lift) of a polytope P is a representation of P as the projection of an affine…
▽ More
A central question in optimization is to maximize (or minimize) a linear function over a given polytope P. To solve such a problem in practice one needs a concise description of the polytope P. In this paper we are interested in representations of P using the positive semidefinite cone: a positive semidefinite lift (psd lift) of a polytope P is a representation of P as the projection of an affine slice of the positive semidefinite cone $\mathbf{S}^d_+$. Such a representation allows linear optimization problems over P to be written as semidefinite programs of size d. Such representations can be beneficial in practice when d is much smaller than the number of facets of the polytope P. In this paper we are concerned with so-called equivariant psd lifts (also known as symmetric psd lifts) which respect the symmetries of the polytope P. We present a representation-theoretic framework to study equivariant psd lifts of a certain class of symmetric polytopes known as orbitopes. Our main result is a structure theorem where we show that any equivariant psd lift of size d of an orbitope is of sum-of-squares type where the functions in the sum-of-squares decomposition come from an invariant subspace of dimension smaller than d^3. We use this framework to study two well-known families of polytopes, namely the parity polytope and the cut polytope, and we prove exponential lower bounds for equivariant psd lifts of these polytopes.
△ Less
Submitted 15 April, 2015; v1 submitted 23 December, 2013;
originally announced December 2013.
-
Polynomial-sized Semidefinite Representations of Derivative Relaxations of Spectrahedral Cones
Authors:
James Saunderson,
Pablo A. Parrilo
Abstract:
We give explicit polynomial-sized (in $n$ and $k$) semidefinite representations of the hyperbolicity cones associated with the elementary symmetric polynomials of degree $k$ in $n$ variables. These convex cones form a family of non-polyhedral outer approximations of the non-negative orthant that preserve low-dimensional faces while successively discarding high-dimensional faces. More generally we…
▽ More
We give explicit polynomial-sized (in $n$ and $k$) semidefinite representations of the hyperbolicity cones associated with the elementary symmetric polynomials of degree $k$ in $n$ variables. These convex cones form a family of non-polyhedral outer approximations of the non-negative orthant that preserve low-dimensional faces while successively discarding high-dimensional faces. More generally we construct explicit semidefinite representations (polynomial-sized in $k,m$, and $n$) of the hyperbolicity cones associated with $k$th directional derivatives of polynomials of the form $p(x) = \det(\sum_{i=1}^{n}A_i x_i)$ where the $A_i$ are $m\times m$ symmetric matrices. These convex cones form an analogous family of outer approximations to any spectrahedral cone. Our representations allow us to use semidefinite programming to solve the linear cone programs associated with these convex cones as well as their (less well understood) dual cones.
△ Less
Submitted 11 August, 2014; v1 submitted 7 August, 2012;
originally announced August 2012.
-
Diagonal and Low-Rank Matrix Decompositions, Correlation Matrices, and Ellipsoid Fitting
Authors:
James Saunderson,
Venkat Chandrasekaran,
Pablo A. Parrilo,
Alan S. Willsky
Abstract:
In this paper we establish links between, and new results for, three problems that are not usually considered together. The first is a matrix decomposition problem that arises in areas such as statistical modeling and signal processing: given a matrix $X$ formed as the sum of an unknown diagonal matrix and an unknown low rank positive semidefinite matrix, decompose $X$ into these constituents. The…
▽ More
In this paper we establish links between, and new results for, three problems that are not usually considered together. The first is a matrix decomposition problem that arises in areas such as statistical modeling and signal processing: given a matrix $X$ formed as the sum of an unknown diagonal matrix and an unknown low rank positive semidefinite matrix, decompose $X$ into these constituents. The second problem we consider is to determine the facial structure of the set of correlation matrices, a convex set also known as the elliptope. This convex body, and particularly its facial structure, plays a role in applications from combinatorial optimization to mathematical finance. The third problem is a basic geometric question: given points $v_1,v_2,...,v_n\in \R^k$ (where $n > k$) determine whether there is a centered ellipsoid passing \emph{exactly} through all of the points.
We show that in a precise sense these three problems are equivalent. Furthermore we establish a simple sufficient condition on a subspace $U$ that ensures any positive semidefinite matrix $L$ with column space $U$ can be recovered from $D+L$ for any diagonal matrix $D$ using a convex optimization-based heuristic known as minimum trace factor analysis. This result leads to a new understanding of the structure of rank-deficient correlation matrices and a simple condition on a set of points that ensures there is a centered ellipsoid passing through them.
△ Less
Submitted 5 April, 2012;
originally announced April 2012.
