-
A Unified Linear Programming Framework for Offline Reward Learning from Human Demonstrations and Feedback
Authors:
Kihyun Kim,
Jiawei Zhang,
Asuman Ozdaglar,
Pablo A. Parrilo
Abstract:
Inverse Reinforcement Learning (IRL) and Reinforcement Learning from Human Feedback (RLHF) are pivotal methodologies in reward learning, which involve inferring and sha** the underlying reward function of sequential decision-making problems based on observed human demonstrations and feedback. Most prior work in reward learning has relied on prior knowledge or assumptions about decision or prefer…
▽ More
Inverse Reinforcement Learning (IRL) and Reinforcement Learning from Human Feedback (RLHF) are pivotal methodologies in reward learning, which involve inferring and sha** the underlying reward function of sequential decision-making problems based on observed human demonstrations and feedback. Most prior work in reward learning has relied on prior knowledge or assumptions about decision or preference models, potentially leading to robustness issues. In response, this paper introduces a novel linear programming (LP) framework tailored for offline reward learning. Utilizing pre-collected trajectories without online exploration, this framework estimates a feasible reward set from the primal-dual optimality conditions of a suitably designed LP, and offers an optimality guarantee with provable sample efficiency. Our LP framework also enables aligning the reward functions with human feedback, such as pairwise trajectory comparison data, while maintaining computational tractability and sample efficiency. We demonstrate that our framework potentially achieves better performance compared to the conventional maximum likelihood estimation (MLE) approach through analytical examples and numerical experiments.
△ Less
Submitted 3 June, 2024; v1 submitted 20 May, 2024;
originally announced May 2024.
-
Towards Tight Convex Relaxations for Contact-Rich Manipulation
Authors:
Bernhard P. Graesdal,
Shao Y. C. Chia,
Tobia Marcucci,
Savva Morozov,
Alexandre Amice,
Pablo A. Parrilo,
Russ Tedrake
Abstract:
We present a method for global motion planning of robotic systems that interact with the environment through contacts. Our method directly handles the hybrid nature of such tasks using tools from convex optimization. We formulate the motion-planning problem as a shortest-path problem in a graph of convex sets, where a path in the graph corresponds to a contact sequence and a convex set models the…
▽ More
We present a method for global motion planning of robotic systems that interact with the environment through contacts. Our method directly handles the hybrid nature of such tasks using tools from convex optimization. We formulate the motion-planning problem as a shortest-path problem in a graph of convex sets, where a path in the graph corresponds to a contact sequence and a convex set models the quasi-static dynamics within a fixed contact mode. For each contact mode, we use semidefinite programming to relax the nonconvex dynamics that results from the simultaneous optimization of the object's pose, contact locations, and contact forces. The result is a tight convex relaxation of the overall planning problem, that can be efficiently solved and quickly rounded to find a feasible contact-rich trajectory. As a first application of this technique, we focus on the task of planar pushing. Exhaustive experiments show that our convex-optimization method generates plans that are consistently within a small percentage of the global optimum. We demonstrate the quality of these plans on a real robotic system.
△ Less
Submitted 15 February, 2024;
originally announced February 2024.
-
Acceleration by Stepsize Hedging I: Multi-Step Descent and the Silver Stepsize Schedule
Authors:
Jason M. Altschuler,
Pablo A. Parrilo
Abstract:
Can we accelerate convergence of gradient descent without changing the algorithm -- just by carefully choosing stepsizes? Surprisingly, we show that the answer is yes. Our proposed Silver Stepsize Schedule optimizes strongly convex functions in $k^{\log_ρ 2} \approx k^{0.7864}$ iterations, where $ρ=1+\sqrt{2}$ is the silver ratio and $k$ is the condition number. This is intermediate between the te…
▽ More
Can we accelerate convergence of gradient descent without changing the algorithm -- just by carefully choosing stepsizes? Surprisingly, we show that the answer is yes. Our proposed Silver Stepsize Schedule optimizes strongly convex functions in $k^{\log_ρ 2} \approx k^{0.7864}$ iterations, where $ρ=1+\sqrt{2}$ is the silver ratio and $k$ is the condition number. This is intermediate between the textbook unaccelerated rate $k$ and the accelerated rate $\sqrt{k}$ due to Nesterov in 1983. The non-strongly convex setting is conceptually identical, and standard black-box reductions imply an analogous accelerated rate $\varepsilon^{-\log_ρ 2} \approx \varepsilon^{-0.7864}$. We conjecture and provide partial evidence that these rates are optimal among all possible stepsize schedules.
The Silver Stepsize Schedule is constructed recursively in a fully explicit way. It is non-monotonic, fractal-like, and approximately periodic of period $k^{\log_ρ 2}$. This leads to a phase transition in the convergence rate: initially super-exponential (acceleration regime), then exponential (saturation regime).
△ Less
Submitted 14 September, 2023;
originally announced September 2023.
-
Kernel approximation on algebraic varieties
Authors:
Jason M. Altschuler,
Pablo A. Parrilo
Abstract:
Low-rank approximation of kernels is a fundamental mathematical problem with widespread algorithmic applications. Often the kernel is restricted to an algebraic variety, e.g., in problems involving sparse or low-rank data. We show that significantly better approximations are obtainable in this setting: the rank required to achieve a given error depends on the variety's dimension rather than the am…
▽ More
Low-rank approximation of kernels is a fundamental mathematical problem with widespread algorithmic applications. Often the kernel is restricted to an algebraic variety, e.g., in problems involving sparse or low-rank data. We show that significantly better approximations are obtainable in this setting: the rank required to achieve a given error depends on the variety's dimension rather than the ambient dimension, which is typically much larger. This is true in both high-precision and high-dimensional regimes. Our results are presented for smooth isotropic kernels, the predominant class of kernels used in applications. Our main technical insight is to approximate smooth kernels by polynomial kernels, and leverage two key properties of polynomial kernels that hold when they are restricted to a variety. First, their ranks decrease exponentially in the variety's co-dimension. Second, their maximum values are governed by their values over a small set of points. Together, our results provide a general approach for exploiting (approximate) "algebraic structure" in datasets in order to efficiently solve large-scale data science problems.
△ Less
Submitted 4 June, 2021;
originally announced June 2021.
-
Semidefinite Relaxations of Products of Nonnegative Forms on the Sphere
Authors:
Chenyang Yuan,
Pablo A. Parrilo
Abstract:
We study the problem of maximizing the geometric mean of $d$ low-degree non-negative forms on the real or complex sphere in $n$ variables. We show that this highly non-convex problem is NP-hard even when the forms are quadratic and is equivalent to optimizing a homogeneous polynomial of degree $O(d)$ on the sphere. The standard Sum-of-Squares based convex relaxation for this polynomial optimizatio…
▽ More
We study the problem of maximizing the geometric mean of $d$ low-degree non-negative forms on the real or complex sphere in $n$ variables. We show that this highly non-convex problem is NP-hard even when the forms are quadratic and is equivalent to optimizing a homogeneous polynomial of degree $O(d)$ on the sphere. The standard Sum-of-Squares based convex relaxation for this polynomial optimization problem requires solving a semidefinite program (SDP) of size $n^{O(d)}$, with multiplicative approximation guarantees of $Ω(\frac{1}{n})$. We exploit the compact representation of this polynomial to introduce a SDP relaxation of size polynomial in $n$ and $d$, and prove that it achieves a constant factor multiplicative approximation when maximizing the geometric mean of non-negative quadratic forms. We also show that this analysis is asymptotically tight, with a sequence of instances where the gap between the relaxation and true optimum approaches this constant factor as $d \rightarrow \infty$. Next we propose a series of intermediate relaxations of increasing complexity that interpolate to the full Sum-of-Squares relaxation, as well as a rounding algorithm that finds an approximate solution from the solution of any intermediate relaxation. Finally we show that this approach can be generalized for relaxations of products of non-negative forms of any degree.
△ Less
Submitted 20 March, 2021; v1 submitted 25 February, 2021;
originally announced February 2021.
-
Shortest Paths in Graphs of Convex Sets
Authors:
Tobia Marcucci,
Jack Umenberger,
Pablo A. Parrilo,
Russ Tedrake
Abstract:
Given a graph, the shortest-path problem requires finding a sequence of edges with minimum cumulative length that connects a source vertex to a target vertex. We consider a variant of this classical problem in which the position of each vertex in the graph is a continuous decision variable constrained in a convex set, and the length of an edge is a convex function of the position of its endpoints.…
▽ More
Given a graph, the shortest-path problem requires finding a sequence of edges with minimum cumulative length that connects a source vertex to a target vertex. We consider a variant of this classical problem in which the position of each vertex in the graph is a continuous decision variable constrained in a convex set, and the length of an edge is a convex function of the position of its endpoints. Problems of this form arise naturally in many areas, from motion planning of autonomous vehicles to optimal control of hybrid systems. The price for such a wide applicability is the complexity of this problem, which is easily seen to be NP-hard. Our main contribution is a strong and lightweight mixed-integer convex formulation based on perspective operators, that makes it possible to efficiently find globally optimal paths in large graphs and in high-dimensional spaces.
△ Less
Submitted 3 July, 2023; v1 submitted 27 January, 2021;
originally announced January 2021.
-
Approximating Min-Mean-Cycle for low-diameter graphs in near-optimal time and memory
Authors:
Jason M. Altschuler,
Pablo A. Parrilo
Abstract:
We revisit Min-Mean-Cycle, the classical problem of finding a cycle in a weighted directed graph with minimum mean weight. Despite an extensive algorithmic literature, previous work falls short of a near-linear runtime in the number of edges $m$. We propose an approximation algorithm that, for graphs with polylogarithmic diameter, achieves a near-linear runtime. In particular, this is the first al…
▽ More
We revisit Min-Mean-Cycle, the classical problem of finding a cycle in a weighted directed graph with minimum mean weight. Despite an extensive algorithmic literature, previous work falls short of a near-linear runtime in the number of edges $m$. We propose an approximation algorithm that, for graphs with polylogarithmic diameter, achieves a near-linear runtime. In particular, this is the first algorithm whose runtime scales in the number of vertices $n$ as $\tilde{O}(n^2)$ for the complete graph. Moreover, unconditionally on the diameter, the algorithm uses only $O(n)$ memory beyond reading the input, making it "memory-optimal". Our approach is based on solving a linear programming relaxation using entropic regularization, which reduces the problem to Matrix Balancing -- á la the popular reduction of Optimal Transport to Matrix Scaling. The algorithm is practical and simple to implement.
△ Less
Submitted 14 November, 2021; v1 submitted 7 April, 2020;
originally announced April 2020.
-
Near-linear convergence of the Random Osborne algorithm for Matrix Balancing
Authors:
Jason M. Altschuler,
Pablo A. Parrilo
Abstract:
We revisit Matrix Balancing, a pre-conditioning task used ubiquitously for computing eigenvalues and matrix exponentials. Since 1960, Osborne's algorithm has been the practitioners' algorithm of choice and is now implemented in most numerical software packages. However, its theoretical properties are not well understood. Here, we show that a simple random variant of Osborne's algorithm converges i…
▽ More
We revisit Matrix Balancing, a pre-conditioning task used ubiquitously for computing eigenvalues and matrix exponentials. Since 1960, Osborne's algorithm has been the practitioners' algorithm of choice and is now implemented in most numerical software packages. However, its theoretical properties are not well understood. Here, we show that a simple random variant of Osborne's algorithm converges in near-linear time in the input sparsity. Specifically, it balances $K\in\mathbb{R}_{\geq 0}^{n\times n}$ after $O(mε^{-2}\logκ)$ arithmetic operations, where $m$ is the number of nonzeros in $K$, $ε$ is the $\ell_1$ accuracy, and $κ=\sum_{ij}K_{ij}/(\min_{ij:K_{ij}\neq 0}K_{ij})$ measures the conditioning of $K$. Previous work had established near-linear runtimes either only for $\ell_2$ accuracy (a weaker criterion which is less relevant for applications), or through an entirely different algorithm based on (currently) impractical Laplacian solvers.
We further show that if the graph with adjacency matrix $K$ is moderately connected--e.g., if $K$ has at least one positive row/column pair--then Osborne's algorithm initially converges exponentially fast, yielding an improved runtime $O(mε^{-1}\logκ)$. We also address numerical precision by showing that these runtime bounds still hold when using $O(\log(nκ/ε))$-bit numbers.
Our results are established through an intuitive potential argument that leverages a convex optimization perspective of Osborne's algorithm, and relates the per-iteration progress to the current imbalance as measured in Hellinger distance. Unlike previous analyses, we critically exploit log-convexity of the potential. Our analysis extends to other variants of Osborne's algorithm: along the way, we establish significantly improved runtime bounds for cyclic, greedy, and parallelized variants.
△ Less
Submitted 2 July, 2021; v1 submitted 6 April, 2020;
originally announced April 2020.
-
Maximizing Products of Linear Forms, and The Permanent of Positive Semidefinite Matrices
Authors:
Chenyang Yuan,
Pablo A. Parrilo
Abstract:
We study the convex relaxation of a polynomial optimization problem, maximizing a product of linear forms over the complex sphere. We show that this convex program is also a relaxation of the permanent of Hermitian positive semidefinite (HPSD) matrices. By analyzing a constructive randomized rounding algorithm, we obtain an improved multiplicative approximation factor to the permanent of HPSD matr…
▽ More
We study the convex relaxation of a polynomial optimization problem, maximizing a product of linear forms over the complex sphere. We show that this convex program is also a relaxation of the permanent of Hermitian positive semidefinite (HPSD) matrices. By analyzing a constructive randomized rounding algorithm, we obtain an improved multiplicative approximation factor to the permanent of HPSD matrices, as well as computationally efficient certificates for this approximation. We also propose an analog of van der Waerden's conjecture for HPSD matrices, where the polynomial optimization problem is interpreted as a relaxation of the permanent.
△ Less
Submitted 13 January, 2021; v1 submitted 10 February, 2020;
originally announced February 2020.
-
Convergence Rate of Block-Coordinate Maximization Burer-Monteiro Method for Solving Large SDPs
Authors:
Murat A. Erdogdu,
Asuman Ozdaglar,
Pablo A. Parrilo,
Nuri Denizcan Vanli
Abstract:
Semidefinite programming (SDP) with diagonal constraints arise in many optimization problems, such as Max-Cut, community detection and group synchronization. Although SDPs can be solved to arbitrary precision in polynomial time, generic convex solvers do not scale well with the dimension of the problem. In order to address this issue, Burer and Monteiro proposed to reduce the dimension of the prob…
▽ More
Semidefinite programming (SDP) with diagonal constraints arise in many optimization problems, such as Max-Cut, community detection and group synchronization. Although SDPs can be solved to arbitrary precision in polynomial time, generic convex solvers do not scale well with the dimension of the problem. In order to address this issue, Burer and Monteiro proposed to reduce the dimension of the problem by appealing to a low-rank factorization and solve the subsequent non-convex problem instead. In this paper, we present coordinate ascent based methods to solve this non-convex problem with provable convergence guarantees. More specifically, we prove that the block-coordinate maximization algorithm applied to the non-convex Burer-Monteiro method globally converges to a first-order stationary point with a sublinear rate without any assumptions on the problem. We further show that this algorithm converges linearly around a local maximum provided that the objective function exhibits quadratic decay. We establish that this condition generically holds when the rank of the factorization is sufficiently large. Furthermore, incorporating Lanczos method to the block-coordinate maximization, we propose an algorithm that is guaranteed to return a solution that provides $1-O(1/r)$ approximation to the original SDP without any assumptions, where $r$ is the rank of the factorization. This approximation ratio is known to be optimal (up to constants) under the unique games conjecture, and we can explicitly quantify the number of iterations to obtain such a solution.
△ Less
Submitted 26 November, 2019; v1 submitted 12 July, 2018;
originally announced July 2018.
-
Chordal networks of polynomial ideals
Authors:
Diego Cifuentes,
Pablo A. Parrilo
Abstract:
We introduce a novel representation of structured polynomial ideals, which we refer to as chordal networks. The sparsity structure of a polynomial system is often described by a graph that captures the interactions among the variables. Chordal networks provide a computationally convenient decomposition into simpler (triangular) polynomial sets, while preserving the underlying graphical structure.…
▽ More
We introduce a novel representation of structured polynomial ideals, which we refer to as chordal networks. The sparsity structure of a polynomial system is often described by a graph that captures the interactions among the variables. Chordal networks provide a computationally convenient decomposition into simpler (triangular) polynomial sets, while preserving the underlying graphical structure. We show that many interesting families of polynomial ideals admit compact chordal network representations (of size linear in the number of variables), even though the number of components is exponentially large. Chordal networks can be computed for arbitrary polynomial systems using a refinement of the chordal elimination algorithm from [Cifuentes-Parrilo-2016]. Furthermore, they can be effectively used to obtain several properties of the variety, such as its dimension, cardinality, and equidimensional components, as well as an efficient probabilistic test for radical ideal membership. We apply our methods to examples from algebraic statistics and vector addition systems; for these instances, algorithms based on chordal networks outperform existing techniques by orders of magnitude.
△ Less
Submitted 22 November, 2016; v1 submitted 9 April, 2016;
originally announced April 2016.
-
An efficient tree decomposition method for permanents and mixed discriminants
Authors:
Diego Cifuentes,
Pablo A. Parrilo
Abstract:
We present an efficient algorithm to compute permanents, mixed discriminants and hyperdeterminants of structured matrices and multidimensional arrays (tensors). We describe the sparsity structure of an array in terms of a graph, and we assume that its treewidth, denoted as $ω$, is small. Our algorithm requires $O(n 2^ω)$ arithmetic operations to compute permanents, and $O(n^2 + n 3^ω)$ for mixed d…
▽ More
We present an efficient algorithm to compute permanents, mixed discriminants and hyperdeterminants of structured matrices and multidimensional arrays (tensors). We describe the sparsity structure of an array in terms of a graph, and we assume that its treewidth, denoted as $ω$, is small. Our algorithm requires $O(n 2^ω)$ arithmetic operations to compute permanents, and $O(n^2 + n 3^ω)$ for mixed discriminants and hyperdeterminants. We finally show that mixed volume computation continues to be hard under bounded treewidth assumptions.
△ Less
Submitted 10 July, 2015;
originally announced July 2015.
-
Semidefinite Programming Approach to Gaussian Sequential Rate-Distortion Trade-offs
Authors:
Takashi Tanaka,
Kwang-Ki K. Kim,
Pablo A. Parrilo,
Sanjoy K. Mitter
Abstract:
Sequential rate-distortion (SRD) theory provides a framework for studying the fundamental trade-off between data-rate and data-quality in real-time communication systems. In this paper, we consider the SRD problem for multi-dimensional time-varying Gauss-Markov processes under mean-square distortion criteria. We first revisit the sensor-estimator separation principle, which asserts that considered…
▽ More
Sequential rate-distortion (SRD) theory provides a framework for studying the fundamental trade-off between data-rate and data-quality in real-time communication systems. In this paper, we consider the SRD problem for multi-dimensional time-varying Gauss-Markov processes under mean-square distortion criteria. We first revisit the sensor-estimator separation principle, which asserts that considered SRD problem is equivalent to a joint sensor and estimator design problem in which data-rate of the sensor output is minimized while the estimator's performance satisfies the distortion criteria. We then show that the optimal joint design can be performed by semidefinite programming. A semidefinite representation of the corresponding SRD function is obtained. Implications of the obtained result in the context of zero-delay source coding theory and applications to networked control theory are also discussed.
△ Less
Submitted 9 August, 2016; v1 submitted 27 November, 2014;
originally announced November 2014.
-
Equivariant semidefinite lifts of regular polygons
Authors:
Hamza Fawzi,
James Saunderson,
Pablo A. Parrilo
Abstract:
Given a polytope P in $\mathbb{R}^n$, we say that P has a positive semidefinite lift (psd lift) of size d if one can express P as the linear projection of an affine slice of the positive semidefinite cone $\mathbf{S}^d_+$. If a polytope P has symmetry, we can consider equivariant psd lifts, i.e. those psd lifts that respect the symmetry of P. One of the simplest families of polytopes with interest…
▽ More
Given a polytope P in $\mathbb{R}^n$, we say that P has a positive semidefinite lift (psd lift) of size d if one can express P as the linear projection of an affine slice of the positive semidefinite cone $\mathbf{S}^d_+$. If a polytope P has symmetry, we can consider equivariant psd lifts, i.e. those psd lifts that respect the symmetry of P. One of the simplest families of polytopes with interesting symmetries are regular polygons in the plane, which have played an important role in the study of linear programming lifts (or extended formulations). In this paper we study equivariant psd lifts of regular polygons. We first show that the standard Lasserre/sum-of-squares hierarchy for the regular N-gon requires exactly ceil(N/4) iterations and thus yields an equivariant psd lift of size linear in N. In contrast we show that one can construct an equivariant psd lift of the regular 2^n-gon of size 2n-1, which is exponentially smaller than the psd lift of the sum-of-squares hierarchy. Our construction relies on finding a sparse sum-of-squares certificate for the facet-defining inequalities of the regular 2^n-gon, i.e., one that only uses a small (logarithmic) number of monomials. Since any equivariant LP lift of the regular 2^n-gon must have size 2^n, this gives the first example of a polytope with an exponential gap between sizes of equivariant LP lifts and equivariant psd lifts. Finally we prove that our construction is essentially optimal by showing that any equivariant psd lift of the regular N-gon must have size at least logarithmic in N.
△ Less
Submitted 15 September, 2014;
originally announced September 2014.
-
Positive semidefinite rank
Authors:
Hamza Fawzi,
João Gouveia,
Pablo A. Parrilo,
Richard Z. Robinson,
Rekha R. Thomas
Abstract:
Let M be a p-by-q matrix with nonnegative entries. The positive semidefinite rank (psd rank) of M is the smallest integer k for which there exist positive semidefinite matrices $A_i, B_j$ of size $k \times k$ such that $M_{ij} = \text{trace}(A_i B_j)$. The psd rank has many appealing geometric interpretations, including semidefinite representations of polyhedra and information-theoretic applicatio…
▽ More
Let M be a p-by-q matrix with nonnegative entries. The positive semidefinite rank (psd rank) of M is the smallest integer k for which there exist positive semidefinite matrices $A_i, B_j$ of size $k \times k$ such that $M_{ij} = \text{trace}(A_i B_j)$. The psd rank has many appealing geometric interpretations, including semidefinite representations of polyhedra and information-theoretic applications. In this paper we develop and survey the main mathematical properties of psd rank, including its geometry, relationships with other rank notions, and computational and algorithmic aspects.
△ Less
Submitted 15 July, 2014;
originally announced July 2014.
-
Exponential lower bounds on fixed-size psd rank and semidefinite extension complexity
Authors:
Hamza Fawzi,
Pablo A. Parrilo
Abstract:
There has been a lot of interest recently in proving lower bounds on the size of linear programs needed to represent a given polytope P. In a breakthrough paper Fiorini et al. [Proceedings of 44th ACM Symposium on Theory of Computing 2012, pages 95-106] showed that any linear programming formulation of maximum-cut must have exponential size. A natural question to ask is whether one can prove such…
▽ More
There has been a lot of interest recently in proving lower bounds on the size of linear programs needed to represent a given polytope P. In a breakthrough paper Fiorini et al. [Proceedings of 44th ACM Symposium on Theory of Computing 2012, pages 95-106] showed that any linear programming formulation of maximum-cut must have exponential size. A natural question to ask is whether one can prove such strong lower bounds for semidefinite programming formulations. In this paper we take a step towards this goal and we prove strong lower bounds for a certain class of SDP formulations, namely SDPs over the Cartesian product of fixed-size positive semidefinite cones. In practice this corresponds to semidefinite programs with a block-diagonal structure and where blocks have constant size d. We show that any such extended formulation of the cut polytope must have exponential size (when d is fixed). The result of Fiorini et al. for LP formulations is obtained as a special case when d=1. For blocks of size d=2 the result rules out any small formulations using second-order cone programming. Our study of SDP lifts over Cartesian product of fixed-size positive semidefinite cones is motivated mainly from practical considerations where it is well known that such SDPs can be solved more efficiently than general SDPs. The proof of our lower bound relies on new results about the sparsity pattern of certain matrices with small psd rank, combined with an induction argument inspired from the recent paper by Kaibel and Weltge [arXiv:1307.3543] on the LP extension complexity of the correlation polytope.
△ Less
Submitted 11 November, 2013;
originally announced November 2013.
-
Stability of Polynomial Differential Equations: Complexity and Converse Lyapunov Questions
Authors:
Amir Ali Ahmadi,
Pablo A. Parrilo
Abstract:
We consider polynomial differential equations and make a number of contributions to the questions of (i) complexity of deciding stability, (ii) existence of polynomial Lyapunov functions, and (iii) existence of sum of squares (sos) Lyapunov functions.
(i) We show that deciding local or global asymptotic stability of cubic vector fields is strongly NP-hard. Simple variations of our proof are show…
▽ More
We consider polynomial differential equations and make a number of contributions to the questions of (i) complexity of deciding stability, (ii) existence of polynomial Lyapunov functions, and (iii) existence of sum of squares (sos) Lyapunov functions.
(i) We show that deciding local or global asymptotic stability of cubic vector fields is strongly NP-hard. Simple variations of our proof are shown to imply strong NP-hardness of several other decision problems: testing local attractivity of an equilibrium point, stability of an equilibrium point in the sense of Lyapunov, invariance of the unit ball, boundedness of trajectories, convergence of all trajectories in a ball to a given equilibrium point, existence of a quadratic Lyapunov function, local collision avoidance, and existence of a stabilizing control law.
(ii) We present a simple, explicit example of a globally asymptotically stable quadratic vector field on the plane which does not admit a polynomial Lyapunov function (joint work with M. Krstic). For the subclass of homogeneous vector fields, we conjecture that asymptotic stability implies existence of a polynomial Lyapunov function, but show that the minimum degree of such a Lyapunov function can be arbitrarily large even for vector fields in fixed dimension and degree. For the same class of vector fields, we further establish that there is no monotonicity in the degree of polynomial Lyapunov functions.
(iii) We show via an explicit counterexample that if the degree of the polynomial Lyapunov function is fixed, then sos programming may fail to find a valid Lyapunov function even though one exists. On the other hand, if the degree is allowed to increase, we prove that existence of a polynomial Lyapunov function for a planar or a homogeneous vector field implies existence of a polynomial Lyapunov function that is sos and that the negative of its derivative is also sos.
△ Less
Submitted 30 August, 2013;
originally announced August 2013.
-
Exchangeable Equilibria, Part I: Symmetric Bimatrix Games
Authors:
Noah D. Stein,
Asuman Ozdaglar,
Pablo A. Parrilo
Abstract:
We introduce the notion of exchangeable equilibria of a symmetric bimatrix game, defined as those correlated equilibria in which players' strategy choices are conditionally independently and identically distributed given some hidden variable. We give several game-theoretic interpretations and a version of the "revelation principle". Geometrically, the set of exchangeable equilibria is convex and l…
▽ More
We introduce the notion of exchangeable equilibria of a symmetric bimatrix game, defined as those correlated equilibria in which players' strategy choices are conditionally independently and identically distributed given some hidden variable. We give several game-theoretic interpretations and a version of the "revelation principle". Geometrically, the set of exchangeable equilibria is convex and lies between the symmetric Nash equilibria and the symmetric correlated equilibria. Exchangeable equilibria can achieve higher expected utility than symmetric Nash equilibria.
△ Less
Submitted 20 January, 2014; v1 submitted 12 July, 2013;
originally announced July 2013.
-
Weighted $\ell_1$-minimization for generalized non-uniform sparse model
Authors:
Sidhant Misra,
Pablo A. Parrilo
Abstract:
Model-based compressed sensing refers to compressed sensing with extra structure about the underlying sparse signal known a priori. Recent work has demonstrated that both for deterministic and probabilistic models imposed on the signal, this extra information can be successfully exploited to enhance recovery performance. In particular, weighted $\ell_1$-minimization with suitable choice of weights…
▽ More
Model-based compressed sensing refers to compressed sensing with extra structure about the underlying sparse signal known a priori. Recent work has demonstrated that both for deterministic and probabilistic models imposed on the signal, this extra information can be successfully exploited to enhance recovery performance. In particular, weighted $\ell_1$-minimization with suitable choice of weights has been shown to improve performance in the so called non-uniform sparse model of signals. In this paper, we consider a full generalization of the non-uniform sparse model with very mild assumptions. We prove that when the measurements are obtained using a matrix with i.i.d Gaussian entries, weighted $\ell_1$-minimization successfully recovers the sparse signal from its measurements with overwhelming probability. We also provide a method to choose these weights for any general signal model from the non-uniform sparse class of signal models.
△ Less
Submitted 16 September, 2014; v1 submitted 7 January, 2013;
originally announced January 2013.
-
Rejoinder: Latent variable graphical model selection via convex optimization
Authors:
Venkat Chandrasekaran,
Pablo A. Parrilo,
Alan S. Willsky
Abstract:
Rejoinder to "Latent variable graphical model selection via convex optimization" by Venkat Chandrasekaran, Pablo A. Parrilo and Alan S. Willsky [arXiv:1008.1290].
Rejoinder to "Latent variable graphical model selection via convex optimization" by Venkat Chandrasekaran, Pablo A. Parrilo and Alan S. Willsky [arXiv:1008.1290].
△ Less
Submitted 5 November, 2012;
originally announced November 2012.
-
Dynamics in Near-Potential Games
Authors:
Ozan Candogan,
Asuman Ozdaglar,
Pablo A. Parrilo
Abstract:
Except for special classes of games, there is no systematic framework for analyzing the dynamical properties of multi-agent strategic interactions. Potential games are one such special but restrictive class of games that allow for tractable dynamic analysis. Intuitively, games that are "close" to a potential game should share similar properties. In this paper, we formalize and develop this idea by…
▽ More
Except for special classes of games, there is no systematic framework for analyzing the dynamical properties of multi-agent strategic interactions. Potential games are one such special but restrictive class of games that allow for tractable dynamic analysis. Intuitively, games that are "close" to a potential game should share similar properties. In this paper, we formalize and develop this idea by quantifying to what extent the dynamic features of potential games extend to "near-potential" games. We study convergence of three commonly studied classes of adaptive dynamics: discrete-time better/best response, logit response, and discrete-time fictitious play dynamics. For better/best response dynamics, we focus on the evolution of the sequence of pure strategy profiles and show that this sequence converges to a (pure) approximate equilibrium set, whose size is a function of the "distance" from a close potential game. We then study logit response dynamics and provide a characterization of the stationary distribution of this update rule in terms of the distance of the game from a close potential game and the corresponding potential function. We further show that the stochastically stable strategy profiles are pure approximate equilibria. Finally, we turn attention to fictitious play, and establish that the sequence of empirical frequencies of player actions converges to a neighborhood of (mixed) equilibria of the game, where the size of the neighborhood increases with distance of the game to a potential game. Thus, our results suggest that games that are close to a potential game inherit the dynamical properties of potential games. Since a close potential game to a given game can be found by solving a convex optimization problem, our approach also provides a systematic framework for studying convergence behavior of adaptive learning dynamics in arbitrary finite strategic form games.
△ Less
Submitted 21 July, 2011;
originally announced July 2011.
-
NP-hardness of Deciding Convexity of Quartic Polynomials and Related Problems
Authors:
Amir Ali Ahmadi,
Alex Olshevsky,
Pablo A. Parrilo,
John N. Tsitsiklis
Abstract:
We show that unless P=NP, there exists no polynomial time (or even pseudo-polynomial time) algorithm that can decide whether a multivariate polynomial of degree four (or higher even degree) is globally convex. This solves a problem that has been open since 1992 when N. Z. Shor asked for the complexity of deciding convexity for quartic polynomials. We also prove that deciding strict convexity, stro…
▽ More
We show that unless P=NP, there exists no polynomial time (or even pseudo-polynomial time) algorithm that can decide whether a multivariate polynomial of degree four (or higher even degree) is globally convex. This solves a problem that has been open since 1992 when N. Z. Shor asked for the complexity of deciding convexity for quartic polynomials. We also prove that deciding strict convexity, strong convexity, quasiconvexity, and pseudoconvexity of polynomials of even degree four or higher is strongly NP-hard. By contrast, we show that quasiconvexity and pseudoconvexity of odd degree polynomials can be decided in polynomial time.
△ Less
Submitted 8 December, 2010;
originally announced December 2010.
-
Convex Graph Invariants
Authors:
Venkat Chandrasekaran,
Pablo A. Parrilo,
Alan S. Willsky
Abstract:
The structural properties of graphs are usually characterized in terms of invariants, which are functions of graphs that do not depend on the labeling of the nodes. In this paper we study convex graph invariants, which are graph invariants that are convex functions of the adjacency matrix of a graph. Some examples include functions of a graph such as the maximum degree, the MAXCUT value (and its s…
▽ More
The structural properties of graphs are usually characterized in terms of invariants, which are functions of graphs that do not depend on the labeling of the nodes. In this paper we study convex graph invariants, which are graph invariants that are convex functions of the adjacency matrix of a graph. Some examples include functions of a graph such as the maximum degree, the MAXCUT value (and its semidefinite relaxation), and spectral invariants such as the sum of the $k$ largest eigenvalues. Such functions can be used to construct convex sets that impose various structural constraints on graphs, and thus provide a unified framework for solving a number of interesting graph problems via convex optimization. We give a representation of all convex graph invariants in terms of certain elementary invariants, and describe methods to compute or approximate convex graph invariants tractably. We also compare convex and non-convex invariants, and discuss connections to robust optimization. Finally we use convex graph invariants to provide efficient convex programming solutions to graph problems such as the deconvolution of the composition of two graphs into the individual components, hypothesis testing between graph families, and the generation of graphs with certain desired structural properties.
△ Less
Submitted 2 December, 2010;
originally announced December 2010.
-
Exchangeable equilibria contradict exactness of the Papadimitriou-Roughgarden algorithm
Authors:
Noah D. Stein,
Pablo A. Parrilo,
Asuman Ozdaglar
Abstract:
The authors have decided to withdraw this submission. Clarifications/corrections, if any, may follow at a later date.
The authors have decided to withdraw this submission. Clarifications/corrections, if any, may follow at a later date.
△ Less
Submitted 25 October, 2010; v1 submitted 14 October, 2010;
originally announced October 2010.
-
A partial proof of Nash's Theorem via exchangeable equilibria
Authors:
Noah D. Stein,
Pablo A. Parrilo,
Asuman Ozdaglar
Abstract:
This document consists of two parts: the second part was submitted earlier as a new proof of Nash's theorem, and the first part is a note explaining a problem found in that proof. We are indebted to Sergiu Hart and Eran Shmaya for their careful study which led to their simultaneous discovery of this error. So far the error has not been fixed, but many of the results and techniques of the paper rem…
▽ More
This document consists of two parts: the second part was submitted earlier as a new proof of Nash's theorem, and the first part is a note explaining a problem found in that proof. We are indebted to Sergiu Hart and Eran Shmaya for their careful study which led to their simultaneous discovery of this error. So far the error has not been fixed, but many of the results and techniques of the paper remain valid, so we will continue to make it available online.
Abstract for the original paper:
We give a novel proof of the existence of Nash equilibria in all finite games without using fixed point theorems or path following arguments. Our approach relies on a new notion intermediate between Nash and correlated equilibria called exchangeable equilibria, which are correlated equilibria with certain symmetry and factorization properties. We prove these exist by a duality argument, using Hart and Schmeidler's proof of correlated equilibrium existence as a first step.
In an appropriate limit exchangeable equilibria converge to the convex hull of Nash equilibria, proving that these exist as well. Exchangeable equilibria are defined in terms of symmetries of the game, so this method automatically proves the stronger statement that a symmetric game has a symmetric Nash equilibrium. The case without symmetries follows by a symmetrization argument.
△ Less
Submitted 13 September, 2010; v1 submitted 17 May, 2010;
originally announced May 2010.
-
Flows and Decompositions of Games: Harmonic and Potential Games
Authors:
Ozan Candogan,
Ishai Menache,
Asuman Ozdaglar,
Pablo A. Parrilo
Abstract:
In this paper we introduce a novel flow representation for finite games in strategic form. This representation allows us to develop a canonical direct sum decomposition of an arbitrary game into three components, which we refer to as the potential, harmonic and nonstrategic components. We analyze natural classes of games that are induced by this decomposition, and in particular, focus on games wit…
▽ More
In this paper we introduce a novel flow representation for finite games in strategic form. This representation allows us to develop a canonical direct sum decomposition of an arbitrary game into three components, which we refer to as the potential, harmonic and nonstrategic components. We analyze natural classes of games that are induced by this decomposition, and in particular, focus on games with no harmonic component and games with no potential component. We show that the first class corresponds to the well-known potential games. We refer to the second class of games as harmonic games, and study the structural and equilibrium properties of this new class of games. Intuitively, the potential component of a game captures interactions that can equivalently be represented as a common interest game, while the harmonic part represents the conflicts between the interests of the players. We make this intuition precise, by studying the properties of these two classes, and show that indeed they have quite distinct and remarkable characteristics. For instance, while finite potential games always have pure Nash equilibria, harmonic games generically never do. Moreover, we show that the nonstrategic component does not affect the equilibria of a game, but plays a fundamental role in their efficiency properties, thus decoupling the location of equilibria and their payoff-related properties. Exploiting the properties of the decomposition framework, we obtain explicit expressions for the projections of games onto the subspaces of potential and harmonic games. This enables an extension of the properties of potential and harmonic games to "nearby" games. We exemplify this point by showing that the set of approximate equilibria of an arbitrary game can be characterized through the equilibria of its projection onto the set of potential games.
△ Less
Submitted 24 June, 2010; v1 submitted 13 May, 2010;
originally announced May 2010.
-
Structure of Extreme Correlated Equilibria: a Zero-Sum Example and its Implications
Authors:
Noah D. Stein,
Asuman Ozdaglar,
Pablo A. Parrilo
Abstract:
We exhibit the rich structure of the set of correlated equilibria by analyzing the simplest of polynomial games: the mixed extension of matching pennies. We show that while the correlated equilibrium set is convex and compact, the structure of its extreme points can be quite complicated. In finite games the ratio of extreme correlated to extreme Nash equilibria can be greater than exponential in t…
▽ More
We exhibit the rich structure of the set of correlated equilibria by analyzing the simplest of polynomial games: the mixed extension of matching pennies. We show that while the correlated equilibrium set is convex and compact, the structure of its extreme points can be quite complicated. In finite games the ratio of extreme correlated to extreme Nash equilibria can be greater than exponential in the size of the strategy spaces. In polynomial games there can exist extreme correlated equilibria which are not finitely supported; we construct a large family of examples using techniques from ergodic theory. We show that in general the set of correlated equilibrium distributions of a polynomial game cannot be described by conditions on finitely many moments (means, covariances, etc.), in marked contrast to the set of Nash equilibria which is always expressible in terms of finitely many moments.
△ Less
Submitted 25 January, 2011; v1 submitted 29 January, 2010;
originally announced February 2010.
-
Correlated Equilibria in Continuous Games: Characterization and Computation
Authors:
Noah D. Stein,
Pablo A. Parrilo,
Asuman Ozdaglar
Abstract:
We present several new characterizations of correlated equilibria in games with continuous utility functions. These have the advantage of being more computationally and analytically tractable than the standard definition in terms of departure functions. We use these characterizations to construct effective algorithms for approximating a single correlated equilibrium or the entire set of correlat…
▽ More
We present several new characterizations of correlated equilibria in games with continuous utility functions. These have the advantage of being more computationally and analytically tractable than the standard definition in terms of departure functions. We use these characterizations to construct effective algorithms for approximating a single correlated equilibrium or the entire set of correlated equilibria of a game with polynomial utility functions.
△ Less
Submitted 22 April, 2010; v1 submitted 22 December, 2008;
originally announced December 2008.
-
Polynomial stochastic games via sum of squares optimization
Authors:
Parikshit Shah,
Pablo A. Parrilo
Abstract:
Stochastic games are an important class of problems that generalize Markov decision processes to game theoretic scenarios. We consider finite state two-player zero-sum stochastic games over an infinite time horizon with discounted rewards. The players are assumed to have infinite strategy spaces and the payoffs are assumed to be polynomials. In this paper we restrict our attention to a special c…
▽ More
Stochastic games are an important class of problems that generalize Markov decision processes to game theoretic scenarios. We consider finite state two-player zero-sum stochastic games over an infinite time horizon with discounted rewards. The players are assumed to have infinite strategy spaces and the payoffs are assumed to be polynomials. In this paper we restrict our attention to a special class of games for which the single-controller assumption holds. It is shown that minimax equilibria and optimal strategies for such games may be obtained via semidefinite programming.
△ Less
Submitted 15 June, 2008;
originally announced June 2008.
-
Separable and Low-Rank Continuous Games
Authors:
Noah D. Stein,
Asuman Ozdaglar,
Pablo A. Parrilo
Abstract:
In this paper, we study nonzero-sum separable games, which are continuous games whose payoffs take a sum-of-products form. Included in this subclass are all finite games and polynomial games. We investigate the structure of equilibria in separable games. We show that these games admit finitely supported Nash equilibria. Motivated by the bounds on the supports of mixed equilibria in two-player fi…
▽ More
In this paper, we study nonzero-sum separable games, which are continuous games whose payoffs take a sum-of-products form. Included in this subclass are all finite games and polynomial games. We investigate the structure of equilibria in separable games. We show that these games admit finitely supported Nash equilibria. Motivated by the bounds on the supports of mixed equilibria in two-player finite games in terms of the ranks of the payoff matrices, we define the notion of the rank of an n-player continuous game and use this to provide bounds on the cardinality of the support of equilibrium strategies. We present a general characterization theorem that states that a continuous game has finite rank if and only if it is separable. Using our rank results, we present an efficient algorithm for computing approximate equilibria of two-player separable games with fixed strategy spaces in time polynomial in the rank of the game.
△ Less
Submitted 24 July, 2007;
originally announced July 2007.
