Showing 1–50 of 85 results for author: Magron, V

Verifying Properties of Binary Neural Networks Using Sparse Polynomial Optimization

Authors: Jianting Yang, Srećko Ðurašinović, Jean-Bernard Lasserre, Victor Magron, Jun Zhao

Abstract: This paper explores methods for verifying the properties of Binary Neural Networks (BNNs), focusing on robustness against adversarial attacks. Despite their lower computational and memory needs, BNNs, like their full-precision counterparts, are also sensitive to input perturbations. Established methods for solving this problem are predominantly based on Satisfiability Modulo Theories and Mixed-Int… ▽ More This paper explores methods for verifying the properties of Binary Neural Networks (BNNs), focusing on robustness against adversarial attacks. Despite their lower computational and memory needs, BNNs, like their full-precision counterparts, are also sensitive to input perturbations. Established methods for solving this problem are predominantly based on Satisfiability Modulo Theories and Mixed-Integer Linear Programming techniques, which are characterized by NP complexity and often face scalability issues. We introduce an alternative approach using Semidefinite Programming relaxations derived from sparse Polynomial Optimization. Our approach, compatible with continuous input space, not only mitigates numerical issues associated with floating-point calculations but also enhances verification scalability through the strategic use of tighter first-order semidefinite relaxations. We demonstrate the effectiveness of our method in verifying robustness against both $\|.\|_\infty$ and $\|.\|_2$-based adversarial attacks. △ Less

Submitted 27 May, 2024; originally announced May 2024.

Comments: 22 pages, 2 figures, 7 tables

Slow convergence of the moment-SOS hierarchy for an elementary polynomial optimization problem

Authors: Didier Henrion, Adrien Le Franc, Victor Magron

Abstract: We describe a parametric univariate quadratic optimization problem for which the moment-SOS hierarchy has finite but increasingly slow convergence when the parameter tends to its limit value. We estimate the order of finite convergence as a function of the parameter. We describe a parametric univariate quadratic optimization problem for which the moment-SOS hierarchy has finite but increasingly slow convergence when the parameter tends to its limit value. We estimate the order of finite convergence as a function of the parameter. △ Less

Submitted 13 March, 2024; originally announced March 2024.

Stability Analysis of Feedback Systems with ReLU Nonlinearities via Semialgebraic Set Representation

Authors: Shingo Nishinaka, Rin Saeki, Tsuyoshi Yuno, Yoshio Ebihara, Victor Magron, Dimitri Peaucelle, Samuele Zoboli, Sophie Tarbouriech

Abstract: This paper is concerned with the stability analysis problem of feedback systems with rectified linear unit (ReLU) nonlinearities. Such feedback systems arise when we model dynamical (recurrent) neural networks (NNs) and NN-driven control systems where all the activation functions of NNs are ReLUs. In this study, we focus on the semialgebraic set representation characterizing the input-output prope… ▽ More This paper is concerned with the stability analysis problem of feedback systems with rectified linear unit (ReLU) nonlinearities. Such feedback systems arise when we model dynamical (recurrent) neural networks (NNs) and NN-driven control systems where all the activation functions of NNs are ReLUs. In this study, we focus on the semialgebraic set representation characterizing the input-output properties of ReLUs. This allows us to employ a novel copositive multiplier in the framework of the integral quadratic constraint and, thus, to derive a linear matrix inequality (LMI) condition for the stability analysis of the feedback systems. However, the infeasibility of this LMI does not allow us to obtain any conclusion on the system's stability due to its conservativeness. This motivates us to consider its dual LMI. By investigating the structure of the dual solution, we derive a rank condition on the dual variable certificating that the system at hand is never stable. In addition, we construct a hierarchy of dual LMIs allowing for improved instability detection. We illustrate the effectiveness of the proposed approach by several numerical examples. △ Less

Submitted 6 March, 2024; originally announced March 2024.

Comments: 8 pages, 5 figures

Upper bound hierarchies for noncommutative polynomial optimization

Authors: Igor Klep, Victor Magron, Gaël Massé, Jurij Volčič

Abstract: This work focuses on minimizing the eigenvalue of a noncommutative polynomial subject to a finite number of noncommutative polynomial inequality constraints. Based on the Helton-McCullough Positivstellensatz, the noncommutative analog of Lasserre's moment-sum of squares hierarchy provides a sequence of lower bounds converging to the minimal eigenvalue, under mild assumptions on the constraint se… ▽ More This work focuses on minimizing the eigenvalue of a noncommutative polynomial subject to a finite number of noncommutative polynomial inequality constraints. Based on the Helton-McCullough Positivstellensatz, the noncommutative analog of Lasserre's moment-sum of squares hierarchy provides a sequence of lower bounds converging to the minimal eigenvalue, under mild assumptions on the constraint set. Each lower bound can be obtained by solving a semidefinite program. We derive complementary converging hierarchies of upper bounds. They are noncommutative analogues of the upper bound hierarchies due to Lasserre for minimizing polynomials over compact sets. Each upper bound can be obtained by solving a generalized eigenvalue problem. △ Less

Submitted 3 February, 2024; originally announced February 2024.

Comments: 4 pages

MSC Class: 90C22; 90C26

Induced Norm Analysis of Linear Systems for Nonnegative Input Signals

Authors: Yoshio Ebihara, Noboru Sebe, Hayato Waki, Dimitri Peaucelle, Sophie Tarbouriech, Victor Magron, Tomomichi Hagiwara

Abstract: This paper is concerned with the analysis of the $L_p\ (p\in[1,\infty), p=\infty)$ induced norms of continuous-time linear systems where input signals are restricted to be nonnegative. This norm is referred to as the $L_{p+}$ induced norm in this paper. It has been shown recently that the $L_{2+}$ induced norm is effective for the stability analysis of nonlinear feedback systems where the nonlinea… ▽ More This paper is concerned with the analysis of the $L_p\ (p\in[1,\infty), p=\infty)$ induced norms of continuous-time linear systems where input signals are restricted to be nonnegative. This norm is referred to as the $L_{p+}$ induced norm in this paper. It has been shown recently that the $L_{2+}$ induced norm is effective for the stability analysis of nonlinear feedback systems where the nonlinearity returns only nonnegative signals. However, the exact computation of the $L_{2+}$ induced norm is essentially difficult. To get around this difficulty, in the first part of this paper, we provide a copositive-programming-based method for the upper bound computation by capturing the nonnegativity of the input signals by copositive multipliers. Then, in the second part of the paper, we derive uniform lower bounds of the $L_{p+}\ (p\in[1,\infty), p=\infty)$ induced norms with respect to the standard $L_{p}$ induced norms that are valid for all linear systems including infinite-dimensional ones. For each linear system, we finally derive a computation method of the lower bounds of the $L_{2+}$ induced norm that are larger than (or equal to) the uniform one. The effectiveness of the upper/lower bound computation methods are fully illustrated by numerical examples. △ Less

Submitted 5 February, 2024; v1 submitted 2 February, 2024; originally announced February 2024.

Comments: 12 pages, 3 figures. A preliminary version of this paper was presented at ECC 2022 (arXiv:2401.03242) and IFAC WC 2023

$L_{2+}$ Induced Norm Analysis of Continuous-Time LTI Systems Using Positive Filters and Copositive Programming

Authors: Yoshio Ebihara, Hayato Waki, Noboru Sebe, Victor Magron, Dimitri Peaucelle, Sophie Tarbouriech

Abstract: This paper is concerned with the analysis of the $L_{2}$ induced norm of continuous-time LTI systems where the input signals are restricted to be nonnegative. This induced norm is referred to as the $L_{2+}$ induced norm in this paper. It has been shown very recently that the $L_{2+}$ induced norm is particularly useful for the stability analysis of nonlinear feedback systems constructed from line… ▽ More This paper is concerned with the analysis of the $L_{2}$ induced norm of continuous-time LTI systems where the input signals are restricted to be nonnegative. This induced norm is referred to as the $L_{2+}$ induced norm in this paper. It has been shown very recently that the $L_{2+}$ induced norm is particularly useful for the stability analysis of nonlinear feedback systems constructed from linear systems and static nonlinearities where the nonlinear elements only provide nonnegative signals. For the upper bound computation of the $L_{2+}$ induced norm, an approach with copositive programming has also been proposed. It is nonetheless true that this approach becomes effective only for multi-input systems, and for single-input systems this approach does not bring any improvement over the trivial upper bound, the standard $L_2$ norm. To overcome this difficulty, we newly introduce positive filters to increase the number of positive signals. This enables us to enlarge the size of the copositive multipliers so that we can obtain better (smaller) upper bounds with copositive programming. △ Less

Submitted 6 January, 2024; originally announced January 2024.

Comments: 9 pages, 3 figures, Proceedings of the European Control Conference (ECC) 2022

Local Lipschitz Constant Computation of ReLU-FNNs: Upper Bound Computation with Exactness Verification

Authors: Yoshio Ebihara, Xin Dai, Victor Magron, Dimitri Peaucelle, Sophie Tarbouriech

Abstract: This paper is concerned with the computation of the local Lipschitz constant of feedforward neural networks (FNNs) with activation functions being rectified linear units (ReLUs). The local Lipschitz constant of an FNN for a target input is a reasonable measure for its quantitative evaluation of the reliability. By following a standard procedure using multipliers that capture the behavior of ReLUs,… ▽ More This paper is concerned with the computation of the local Lipschitz constant of feedforward neural networks (FNNs) with activation functions being rectified linear units (ReLUs). The local Lipschitz constant of an FNN for a target input is a reasonable measure for its quantitative evaluation of the reliability. By following a standard procedure using multipliers that capture the behavior of ReLUs,we first reduce the upper bound computation problem of the local Lipschitz constant into a semidefinite programming problem (SDP). Here we newly introduce copositive multipliers to capture the ReLU behavior accurately. Then, by considering the dual of the SDP for the upper bound computation, we second derive a viable test to conclude the exactness of the computed upper bound. However, these SDPs are intractable for practical FNNs with hundreds of ReLUs. To address this issue, we further propose a method to construct a reduced order model whose input-output property is identical to the original FNN over a neighborhood of the target input. We finally illustrate the effectiveness of the model reduction and exactness verification methods with numerical examples of practical FNNs. △ Less

Submitted 7 April, 2024; v1 submitted 17 October, 2023; originally announced October 2023.

Comments: 8 pages, 3 figures, Fixed authors' display in the bibliography

Certifying ground-state properties of quantum many-body systems

Authors: Jie Wang, Jacopo Surace, Irénée Frérot, Benoît Legat, Marc-Olivier Renou, Victor Magron, Antonio Acín

Abstract: A ubiquitous problem in quantum physics is to understand the ground-state properties of many-body systems. Confronted with the fact that exact diagonalisation quickly becomes impossible when increasing the system size, variational approaches are typically employed as a scalable alternative: energy is minimised over a subset of all possible states and then different physical quantities are computed… ▽ More A ubiquitous problem in quantum physics is to understand the ground-state properties of many-body systems. Confronted with the fact that exact diagonalisation quickly becomes impossible when increasing the system size, variational approaches are typically employed as a scalable alternative: energy is minimised over a subset of all possible states and then different physical quantities are computed over the solution state. Despite remarkable success, rigorously speaking, all what variational methods offer are upper bounds on the ground-state energy. On the other hand, so-called relaxations of the ground-state problem based on semidefinite programming represent a complementary approach, providing lower bounds to the ground-state energy. However, in their current implementation, neither variational nor relaxation methods offer provable bound on other observables in the ground state beyond the energy. In this work, we show that the combination of the two classes of approaches can be used to derive certifiable bounds on the value of any observable in the ground state, such as correlation functions of arbitrary order, structure factors, or order parameters. We illustrate the power of this approach in paradigmatic examples of 1D and 2D spin-one-half Heisenberg models. To improve the scalability of the method, we exploit the symmetries and sparsity of the considered systems to reach sizes of hundreds of particles at much higher precision than previous works. Our analysis therefore shows how to obtain certifiable bounds on many-body ground-state properties beyond energy in a scalable way. △ Less

Submitted 4 March, 2024; v1 submitted 9 October, 2023; originally announced October 2023.

Comments: 25 pages, 14 figures, 15 tables

A real moment-HSOS hierarchy for complex polynomial optimization with real coefficients

Authors: Jie Wang, Victor Magron

Abstract: This paper proposes a real moment-HSOS hierarchy for complex polynomial optimization problems with real coefficients. We show that this hierarchy provides the same sequence of lower bounds as the complex analogue, yet is much cheaper to solve. In addition, we prove that global optimality is achieved when the ranks of the moment matrix and certain submatrix equal two in case that a sphere constrain… ▽ More This paper proposes a real moment-HSOS hierarchy for complex polynomial optimization problems with real coefficients. We show that this hierarchy provides the same sequence of lower bounds as the complex analogue, yet is much cheaper to solve. In addition, we prove that global optimality is achieved when the ranks of the moment matrix and certain submatrix equal two in case that a sphere constraint is present, and as a consequence, the complex polynomial optimization problem has either two real optimal solutions or a pair of conjugate optimal solutions. A simple procedure for extracting a pair of conjugate optimal solutions is given in the latter case. Various numerical examples are presented to demonstrate the efficiency of this new hierarchy, and an application to polyphase code design is also provided. △ Less

Submitted 25 February, 2024; v1 submitted 28 August, 2023; originally announced August 2023.

Comments: 20 pages, 8 tables

Symmetry reduction and recovery of trajectories of optimal control problems via measure relaxations

Authors: Nicolas Augier, Didier Henrion, Milan Korda, Victor Magron

Abstract: We address the problem of symmetry reduction of optimal control problems under the action of a finite group from a measure relaxation viewpoint. We propose a method based on the moment-SOS aka Lasserre hierarchy which allows one to significantly reduce the computation time and memory requirements compared to the case without symmetry reduction. We show that the recovery of optimal trajectories boi… ▽ More We address the problem of symmetry reduction of optimal control problems under the action of a finite group from a measure relaxation viewpoint. We propose a method based on the moment-SOS aka Lasserre hierarchy which allows one to significantly reduce the computation time and memory requirements compared to the case without symmetry reduction. We show that the recovery of optimal trajectories boils down to solving a symmetric parametric polynomial system. Then we illustrate our method on the symmetric integrator and the time-optimal inversion of qubits. △ Less

Submitted 7 July, 2023; originally announced July 2023.

Comments: 38 pages, 23 figures

MSC Class: 49M20; 90C22; 93C10; 28A99

Sums of squares certificates for polynomial moment inequalities

Authors: Igor Klep, Victor Magron, Jurij Volčič

Abstract: This paper introduces and develops the algebraic framework of moment polynomials, which are polynomial expressions in commuting variables and their formal mixed moments. Their positivity and optimization over probability measures supported on semialgebraic sets and subject to moment polynomial constraints is investigated. A positive solution to Hilbert's 17th problem for pseudo-moments is given. O… ▽ More This paper introduces and develops the algebraic framework of moment polynomials, which are polynomial expressions in commuting variables and their formal mixed moments. Their positivity and optimization over probability measures supported on semialgebraic sets and subject to moment polynomial constraints is investigated. A positive solution to Hilbert's 17th problem for pseudo-moments is given. On the other hand, moment polynomials positive on actual measures are shown to be sums of squares and formal moments of squares up to arbitrarily small perturbation of their coefficients. When only measures supported on a bounded semialgebraic set are considered, a stronger algebraic certificate for moment polynomial positivity is derived. This result gives rise to a converging hierarchy of semidefinite programs for moment polynomial optimization. Finally, as an application, two open nonlinear Bell inequalities from quantum physics are settled. △ Less

Submitted 10 May, 2024; v1 submitted 9 June, 2023; originally announced June 2023.

Comments: 35 pages

MSC Class: 13J30; 44A60; 60E15; 90C22; 46G12; 47L60; 81-08

Minimal Sparsity for Second-Order Moment-SOS Relaxations of the AC-OPF Problem

Authors: Adrien Le Franc, Victor Magron, Jean-Bernard Lasserre, Manuel Ruiz, Patrick Panciatici

Abstract: AC-OPF (Alternative Current Optimal Power Flow)aims at minimizing the operating costs of a power gridunder physical constraints on voltages and power injections.Its mathematical formulation results in a nonconvex polynomial optimizationproblem which is hard to solve in general,but that can be tackled by a sequence of SDP(Semidefinite Programming) relaxationscorresponding to the steps of the moment… ▽ More AC-OPF (Alternative Current Optimal Power Flow)aims at minimizing the operating costs of a power gridunder physical constraints on voltages and power injections.Its mathematical formulation results in a nonconvex polynomial optimizationproblem which is hard to solve in general,but that can be tackled by a sequence of SDP(Semidefinite Programming) relaxationscorresponding to the steps of the moment-SOS (Sums-Of-Squares) hierarchy.Unfortunately, the size of these SDPs grows drastically in the hierarchy,so that even second-order relaxationsexploiting the correlative sparsity pattern of AC-OPFare hardly numerically tractable for largeinstances -- with thousands of power buses.Our contribution lies in a new sparsityframework, termed minimal sparsity, inspiredfrom the specific structure of power flowequations.Despite its heuristic nature, numerical examples show that minimal sparsity allows the computation ofhighly accurate second-order moment-SOS relaxationsof AC-OPF, while requiring far less computing time and memory resources than the standard correlative sparsity pattern. Thus, we manage to compute second-order relaxations on test caseswith about 6000 power buses, which we believe to be unprecedented. △ Less

Submitted 30 May, 2023; originally announced May 2023.

Convergence rates for sums-of-squares hierarchies with correlative sparsity

Authors: Milan Korda, Victor Magron, Rodolfo Rios-Zertuche

Abstract: This work derives upper bounds on the convergence rate of the moment-sum-of-squares hierarchy with correlative sparsity for global minimization of polynomials on compact basic semialgebraic sets. The main conclusion is that both sparse hierarchies based on the Schmüdgen and Putinar Positivstellensätze enjoy a polynomial rate of convergence that depends on the size of the largest clique in the spar… ▽ More This work derives upper bounds on the convergence rate of the moment-sum-of-squares hierarchy with correlative sparsity for global minimization of polynomials on compact basic semialgebraic sets. The main conclusion is that both sparse hierarchies based on the Schmüdgen and Putinar Positivstellensätze enjoy a polynomial rate of convergence that depends on the size of the largest clique in the sparsity graph but not on the ambient dimension. Interestingly, the sparse bounds outperform the best currently available bounds for the dense hierarchy when the maximum clique size is sufficiently small compared to the ambient dimension and the performance is measured by the running time of an interior point method required to obtain a bound on the global minimum of a given accuracy. △ Less

Submitted 26 March, 2023; originally announced March 2023.

Comments: 23 pages

MSC Class: 90C23 (Primary) 13P25; 65K10 (Secondary) ACM Class: G.1.6

Peak Estimation of Time Delay Systems using Occupation Measures

Authors: Jared Miller, Milan Korda, Victor Magron, Mario Sznaier

Abstract: This work proposes a method to compute the maximum value obtained by a state function along trajectories of a Delay Differential Equation (DDE). An example of this task is finding the maximum number of infected people in an epidemic model with a nonzero incubation period. The variables of this peak estimation problem include the stop** time and the original history (restricted to a class of admi… ▽ More This work proposes a method to compute the maximum value obtained by a state function along trajectories of a Delay Differential Equation (DDE). An example of this task is finding the maximum number of infected people in an epidemic model with a nonzero incubation period. The variables of this peak estimation problem include the stop** time and the original history (restricted to a class of admissible histories). The original nonconvex DDE peak estimation problem is approximated by an infinite-dimensional Linear Program (LP) in occupation measures, inspired by existing measure-based methods in peak estimation and optimal control. This LP is approximated from above by a sequence of Semidefinite Programs (SDPs) through the moment-Sum of Squares (SOS) hierarchy. Effectiveness of this scheme in providing peak estimates for DDEs is demonstrated with provided examples △ Less

Submitted 5 April, 2023; v1 submitted 22 March, 2023; originally announced March 2023.

Comments: 34 pages, 14 figures, 3 tables

Sums of squares representations on singular loci

Authors: Ngoc Hoang Anh Mai, Victor Magron

Abstract: The problem of characterizing a real polynomial $f$ as a sum of squares of polynomials on a real algebraic variety $V$ dates back to the pioneering work of Hilbert in [Mathematische Annalen 32.3 (1888): 342-350]. In this paper, we investigate this problem with a focus on cases where the real zeros of $f$ on $V$ are singular points of $V$. By using optimality conditions and irreducible decompositio… ▽ More The problem of characterizing a real polynomial $f$ as a sum of squares of polynomials on a real algebraic variety $V$ dates back to the pioneering work of Hilbert in [Mathematische Annalen 32.3 (1888): 342-350]. In this paper, we investigate this problem with a focus on cases where the real zeros of $f$ on $V$ are singular points of $V$. By using optimality conditions and irreducible decomposition, we provide a positive answer to the following essential question of polynomial optimization: Are there always exact semidefinite programs to compute the minimum value attained by a given polynomial over a given real algebraic variety? Our answer implies that Lasserre's hierarchy, which is known as a bridge between convex and non-convex programs with algebraic structures, has finite convergence not only in the generic case but also in the general case. As a result, we constructively prove that each hyperbolic program is equivalent to a semidefinite program. △ Less

Submitted 9 March, 2023; originally announced March 2023.

Comments: 27 pages. arXiv admin note: substantial text overlap with arXiv:2211.12440

Pourchet's theorem in action: decomposing univariate nonnegative polynomials as sums of five squares

Authors: Victor Magron, Przemysław Koprowski, Tristan Vaccon

Abstract: Pourchet proved in 1971 that every nonnegative univariate polynomial with rational coefficients is a sum of five or fewer squares. Nonetheless, there are no known algorithms for constructing such a decomposition. The sole purpose of the present paper is to present a set of algorithms that decompose a given nonnegative polynomial into a sum of six (five under some unproven conjecture or when allowi… ▽ More Pourchet proved in 1971 that every nonnegative univariate polynomial with rational coefficients is a sum of five or fewer squares. Nonetheless, there are no known algorithms for constructing such a decomposition. The sole purpose of the present paper is to present a set of algorithms that decompose a given nonnegative polynomial into a sum of six (five under some unproven conjecture or when allowing weights) squares of polynomials. Moreover, we prove that the binary complexity can be expressed polynomially in terms of classical operations of computer algebra and algorithmic number theory. △ Less

Submitted 4 February, 2023; originally announced February 2023.

Comments: 10 pages, 9 algorithms, submitted at the ISSAC 2023 conference

State polynomials: positivity, optimization and nonlinear Bell inequalities

Authors: Igor Klep, Victor Magron, Jurij Volčič, Jie Wang

Abstract: This paper introduces state polynomials, i.e., polynomials in noncommuting variables and formal states of their products. A state analog of Artin's solution to Hilbert's 17th problem is proved showing that state polynomials, positive over all matrices and matricial states, are sums of squares with denominators. Somewhat surprisingly, it is also established that a Krivine-Stengle Positivstellensatz… ▽ More This paper introduces state polynomials, i.e., polynomials in noncommuting variables and formal states of their products. A state analog of Artin's solution to Hilbert's 17th problem is proved showing that state polynomials, positive over all matrices and matricial states, are sums of squares with denominators. Somewhat surprisingly, it is also established that a Krivine-Stengle Positivstellensatz fails to hold in the state polynomial setting. Further, archimedean Positivstellensätze in the spirit of Putinar and Helton-McCullough are presented leading to a hierarchy of semidefinite relaxations converging monotonically to the optimum of a state polynomial subject to state constraints. This hierarchy can be seen as a state analog of the Lasserre hierarchy for optimization of polynomials, and the Navascués-Pironio-Acín scheme for optimization of noncommutative polynomials. The motivation behind this theory arises from the study of correlations in quantum networks. Determining the maximal quantum violation of a polynomial Bell inequality for an arbitrary network is reformulated as a state polynomial optimization problem. Several examples of quadratic Bell inequalities in the bipartite and the bilocal tripartite scenario are analyzed. To reduce the size of the constructed SDPs, sparsity, sign symmetry and conditional expectation of the observables' group structure are exploited. To obtain the above-mentioned results, techniques from noncommutative algebra, real algebraic geometry, operator theory, and convex optimization are employed. △ Less

Submitted 3 August, 2023; v1 submitted 29 January, 2023; originally announced January 2023.

Time-dependent moments from partial differential equations and the time-dependent set of atoms

Authors: Raúl E. Curto, Philipp J. di Dio, Milan Korda, Victor Magron

Abstract: We study the time-dependent moments and associated polynomials arising from the partial differential equation $\partial_t f = νΔf + g\cdot\nabla f + h\cdot f$, and consider in detail the dual equation. For the heat equation we find that several non-negative polynomials which are not sums of squares become sums of squares under the heat equation in finite time. We show that every non-negative polyn… ▽ More We study the time-dependent moments and associated polynomials arising from the partial differential equation $\partial_t f = νΔf + g\cdot\nabla f + h\cdot f$, and consider in detail the dual equation. For the heat equation we find that several non-negative polynomials which are not sums of squares become sums of squares under the heat equation in finite time. We show that every non-negative polynomial in $\mathbb{R}[x,y,z]_{\leq 4}$ becomes a sum of squares in finite time under the heat equation. We solve the problem of moving atoms under the equation $\partial_t f = g\cdot\nabla f + h\cdot f$ with $f_0 = μ_0$ being a finitely atomic measure. The time evolution $μ_t = \sum_{i=1}^k c_i(t)\cdot δ_{x_i(t)}$ of the atom positions $x_i(t)$ are described by the transport term $g\cdot\nabla$ and the time-dependent coefficients $c_i(t)$ have an explicit solution depending on $x_i(t)$, $h$, and $\mathrm{div}\, g$. △ Less

Submitted 13 March, 2023; v1 submitted 8 November, 2022; originally announced November 2022.

Comments: Extended Results

MSC Class: Primary: 47A57; 44A60; Secondary: 30E05; 65D32

Exploiting ideal-sparsity in the generalized moment problem with application to matrix factorization ranks

Authors: Milan Korda, Monique Laurent, Victor Magron, Andries Steenkamp

Abstract: We explore a new type of sparsity for the generalized moment problem (GMP) that we call ideal-sparsity. This sparsity exploits the presence of equality constraints requiring the measure to be supported on the variety of an ideal generated by bilinear monomials modeled by an associated graph. We show that this enables an equivalent sparse reformulation of the GMP, where the single (high dimensional… ▽ More We explore a new type of sparsity for the generalized moment problem (GMP) that we call ideal-sparsity. This sparsity exploits the presence of equality constraints requiring the measure to be supported on the variety of an ideal generated by bilinear monomials modeled by an associated graph. We show that this enables an equivalent sparse reformulation of the GMP, where the single (high dimensional) measure variable is replaced by several (lower-dimensional) measure variables supported on the maximal cliques of the graph. We explore the resulting hierarchies of moment-based relaxations for the original dense formulation of GMP and this new, equivalent ideal-sparse reformulation, when applied to the problem of bounding nonnegative- and completely positive matrix factorization ranks. We show that the ideal-sparse hierarchies provide bounds that are at least as good (and often tighter) as those obtained from the dense hierarchy. This is in sharp contrast to the situation when exploiting correlative sparsity, as is most common in the literature, where the resulting bounds are weaker than the dense bounds. Moreover, while correlative sparsity requires the underlying graph to be chordal, no such assumption is needed for ideal-sparsity. Numerical results show that the ideal-sparse bounds are often tighter and much faster to compute than their dense analogs. △ Less

Submitted 9 July, 2023; v1 submitted 20 September, 2022; originally announced September 2022.

Comments: 36 pages, 3 figures

Tractable hierarchies of convex relaxations for polynomial optimization on the nonnegative orthant

Authors: Ngoc Hoang Anh Mai, Victor Magron, Jean-Bernard Lasserre, Kim-Chuan Toh

Abstract: We consider polynomial optimization problems (POP) on a semialgebraic set contained in the nonnegative orthant (every POP on a compact set can be put in this format by a simple translation of the origin). Such a POP can be converted to an equivalent POP by squaring each variable. Using even symmetry and the concept of factor width, we propose a hierarchy of semidefinite relaxations based on the ex… ▽ More We consider polynomial optimization problems (POP) on a semialgebraic set contained in the nonnegative orthant (every POP on a compact set can be put in this format by a simple translation of the origin). Such a POP can be converted to an equivalent POP by squaring each variable. Using even symmetry and the concept of factor width, we propose a hierarchy of semidefinite relaxations based on the extension of Pólya's Positivstellensatz by Dickinson-Povh. As its distinguishing and crucial feature, the maximal matrix size of each resulting semidefinite relaxation can be chosen arbitrarily and in addition, we prove that the sequence of values returned by the new hierarchy converges to the optimal value of the original POP at the rate $O(\varepsilon^{-c})$ if the semialgebraic set has nonempty interior. When applied to (i) robustness certification of multi-layer neural networks and (ii) computation of positive maximal singular values, our method based on Pólya's Positivstellensatz provides better bounds and runs several hundred times faster than the standard Moment-SOS hierarchy. △ Less

Submitted 13 September, 2022; originally announced September 2022.

Comments: 39 pages, 15 tables

Sparse Polynomial Optimization: Theory and Practice

Authors: Victor Magron, Jie Wang

Abstract: The problem of minimizing a polynomial over a set of polynomial inequalities is an NP-hard non-convex problem. Thanks to powerful results from real algebraic geometry, one can convert this problem into a nested sequence of finite-dimensional convex problems. At each step of the associated hierarchy, one needs to solve a fixed size semidefinite program, which can be in turn solved with efficient nu… ▽ More The problem of minimizing a polynomial over a set of polynomial inequalities is an NP-hard non-convex problem. Thanks to powerful results from real algebraic geometry, one can convert this problem into a nested sequence of finite-dimensional convex problems. At each step of the associated hierarchy, one needs to solve a fixed size semidefinite program, which can be in turn solved with efficient numerical tools. On the practical side however, there is no-free lunch and such optimization methods usually encompass severe scalability issues. Fortunately, for many applications, we can look at the problem in the eyes and exploit the inherent data structure arising from the cost and constraints describing the problem, for instance sparsity or symmetries. This book presents several research efforts to tackle this scientific challenge with important computational implications, and provides the development of alternative optimization schemes that scale well in terms of computational complexity, at least in some identified class of problems. The presented algorithmic framework in this book mainly exploits the sparsity structure of the input data to solve large-scale polynomial optimization problems. We present sparsity-exploiting hierarchies of relaxations, for either unconstrained or constrained problems. By contrast with the dense hierarchies, they provide faster approximation of the solution in practice but also come with the same theoretical convergence guarantees. Our framework is not restricted to static polynomial optimization, and we expose hierarchies of approximations for values of interest arising from the analysis of dynamical systems. We also present various extensions to problems involving noncommuting variables, e.g., matrices of arbitrary size or quantum physic operators. △ Less

Submitted 25 August, 2022; v1 submitted 23 August, 2022; originally announced August 2022.

Comments: 220 pages, to appear in Series on Optimization and Its Applications, World Scientific Press

Urysohn in action: separating semialgebraic sets by polynomials

Authors: Milan Korda, Jean-Bernard Lasserre, Alexey Lazarev, Victor Magron, Simone Naldi

Abstract: A classical result from topology called Uryshon's lemma asserts the existence of a continuous separator of two disjoint closed sets in a sufficiently regular topological space. In this work we make a search for this separator constructive and efficient in the context of real algebraic geometry. Namely, given two compact disjoint basic semialgebraic sets which are contained in an $n$-dimensional bo… ▽ More A classical result from topology called Uryshon's lemma asserts the existence of a continuous separator of two disjoint closed sets in a sufficiently regular topological space. In this work we make a search for this separator constructive and efficient in the context of real algebraic geometry. Namely, given two compact disjoint basic semialgebraic sets which are contained in an $n$-dimensional box, we provide an algorithm that computes a separating polynomial greater than or equal to 1 on the first set and less than or equal to 0 on the second one. △ Less

Submitted 1 July, 2022; originally announced July 2022.

Comments: 4 pages, 1 figure, submitted as en extended abstract for the last POEMA workshop

Tractable semidefinite bounds of positive maximal singular values

Authors: Victor Magron, Ngoc Hoang Anh Mai, Yoshio Ebihara, Hayato Waki

Abstract: We focus on computing certified upper bounds for the positive maximal singular value (PMSV) of a given matrix. The PMSV problem boils down to maximizing a quadratic polynomial on the intersection of the unit sphere and the nonnegative orthant. We provide a hierarchy of tractable semidefinite relaxations to approximate the value of the latter polynomial optimization problem as closely as desired. T… ▽ More We focus on computing certified upper bounds for the positive maximal singular value (PMSV) of a given matrix. The PMSV problem boils down to maximizing a quadratic polynomial on the intersection of the unit sphere and the nonnegative orthant. We provide a hierarchy of tractable semidefinite relaxations to approximate the value of the latter polynomial optimization problem as closely as desired. This hierarchy is based on an extension of Pólya's representation theorem. Doing so, positive polynomials can be decomposed as weighted sums of squares of $s$-nomials, where $s$ can be a priori fixed ($s=1$ corresponds to monomials, $s=2$ corresponds to binomials, etc.). This in turn allows us to control the size of the resulting semidefinite relaxations. △ Less

Submitted 17 February, 2022; originally announced February 2022.

Comments: 4 pages, 1 table, submitted to MTNS as extended abstract

Exact SOHS decompositions of trigonometric univariate polynomials with Gaussian coefficients

Authors: Victor Magron, Mohab Safey El Din, Markus Schweighofer, Trung Hieu Vu

Abstract: Certifying the positivity of trigonometric polynomials is of first importance for design problems in discrete-time signal processing. It is well known from the Riesz-Fejéz spectral factorization theorem that any trigonometric univariate polynomial positive on the unit circle can be decomposed as a Hermitian square with complex coefficients. Here we focus on the case of polynomials with Gaussian in… ▽ More Certifying the positivity of trigonometric polynomials is of first importance for design problems in discrete-time signal processing. It is well known from the Riesz-Fejéz spectral factorization theorem that any trigonometric univariate polynomial positive on the unit circle can be decomposed as a Hermitian square with complex coefficients. Here we focus on the case of polynomials with Gaussian integer coefficients, i.e., with real and imaginary parts being integers. We design, analyze and compare, theoretically and practically,three hybrid numeric-symbolic algorithms computing weighted sums of Hermitian squares decompositions for trigonometric univariate polynomials positive on the unit circle with Gaussian coefficients. The numerical steps the first and second algorithm rely on are complex root isolation and semidefinite programming, respectively. An exact sum of Hermitian squares decomposition is obtained thanks to compensation techniques. The third algorithm, also based on complex semidefinite programming, is an adaptation of the rounding and projection algorithm by Peyrl and Parrilo. For all three algorithms, we prove bit complexity and output size estimates that are polynomial in the degree of the input and linear in the maximum bitsize of its coefficients. We compare their performance on randomly chosen benchmarks, and further design a certified finite impulse filter. △ Less

Submitted 4 October, 2023; v1 submitted 14 February, 2022; originally announced February 2022.

Comments: 9 pages, 1 table

Stability Analysis of Recurrent Neural Networks by IQC with Copositive Mutipliers

Authors: Yoshio Ebihara, Hayato Waki, Victor Magron, Ngoc Hoang Anh Mai, Dimitri Peaucelle, Sophie Tarbouriech

Abstract: This paper is concerned with the stability analysis of the recurrent neural networks (RNNs) by means of the integral quadratic constraint (IQC) framework. The rectified linear unit (ReLU) is typically employed as the activation function of the RNN, and the ReLU has specific nonnegativity properties regarding its input and output signals. Therefore, it is effective if we can derive IQC-based stabil… ▽ More This paper is concerned with the stability analysis of the recurrent neural networks (RNNs) by means of the integral quadratic constraint (IQC) framework. The rectified linear unit (ReLU) is typically employed as the activation function of the RNN, and the ReLU has specific nonnegativity properties regarding its input and output signals. Therefore, it is effective if we can derive IQC-based stability conditions with multipliers taking care of such nonnegativity properties. However, such nonnegativity (linear) properties are hardly captured by the existing multipliers defined on the positive semidefinite cone. To get around this difficulty, we loosen the standard positive semidefinite cone to the copositive cone, and employ copositive multipliers to capture the nonnegativity properties. We show that, within the framework of the IQC, we can employ copositive multipliers (or their inner approximation) together with existing multipliers such as Zames-Falb multipliers and polytopic bounding multipliers, and this directly enables us to ensure that the introduction of the copositive multipliers leads to better (no more conservative) results. We finally illustrate the effectiveness of the IQC-based stability conditions with the copositive multipliers by numerical examples. △ Less

Submitted 9 February, 2022; originally announced February 2022.

Comments: 6 pages, 2 figures. arXiv admin note: text overlap with arXiv:2011.12726

Journal ref: Proceedings of the Control and Decision Conference (CDC) 2021

Exploiting Term Sparsity in Moment-SOS hierarchy for Dynamical Systems

Authors: Jie Wang, Corbinian Schlosser, Milan Korda, Victor Magron

Abstract: In this paper, we develop a dynamical system counterpart to the term sparsity sum-of-squares (TSSOS) algorithm proposed for static polynomial optimization. This allows for computational savings and improved scalability while preserving convergence guarantees when sum-of-squares methods are applied to problems from dynamical systems, including the problems of approximating region of attraction, the… ▽ More In this paper, we develop a dynamical system counterpart to the term sparsity sum-of-squares (TSSOS) algorithm proposed for static polynomial optimization. This allows for computational savings and improved scalability while preserving convergence guarantees when sum-of-squares methods are applied to problems from dynamical systems, including the problems of approximating region of attraction, the maximum positively invariant set, and the global attractor. At its core, the method exploits the algebraic structure of the data, thereby complementing existing methods that exploit causality relations among the states of the dynamical system. The procedure encompasses sign symmetries of the dynamical system as was already revealed for polynomial optimization. Numerical examples demonstrate the efficiency of the approach in the presence of this type of sparsity. △ Less

Submitted 9 October, 2023; v1 submitted 16 November, 2021; originally announced November 2021.

Comments: 7 pages, 4 figures. IEEE Transactions on Automatic Control (2023)

MSC Class: 93C10; 93C15; 93C20; 90C22; 90C23; 34C11; 37C10

Revisiting semidefinite programming approaches to options pricing: complexity and computational perspectives

Authors: Didier Henrion, Felix Kirschner, Etienne de Klerk, Milan Korda, Jean-Bernard Lasserre, Victor Magron

Abstract: In this paper we consider the problem of finding bounds on the prices of options depending on multiple assets without assuming any underlying model on the price dynamics, but only the absence of arbitrage opportunities. We formulate this as a generalized moment problem and utilize the well-known Moment-Sum-of-Squares (SOS) hierarchy of Lasserre to obtain bounds on the range of the possible prices.… ▽ More In this paper we consider the problem of finding bounds on the prices of options depending on multiple assets without assuming any underlying model on the price dynamics, but only the absence of arbitrage opportunities. We formulate this as a generalized moment problem and utilize the well-known Moment-Sum-of-Squares (SOS) hierarchy of Lasserre to obtain bounds on the range of the possible prices. A complementary approach (also due to Lasserre) is employed for comparison. We present several numerical examples to demonstrate the viability of our approach. The framework we consider makes it possible to incorporate different kinds of observable data, such as moment information, as well as observable prices of options on the assets of interest. △ Less

Submitted 3 June, 2022; v1 submitted 15 November, 2021; originally announced November 2021.

Comments: 20 pages

Certifying Global Optimality of AC-OPF Solutions via sparse polynomial optimization

Authors: Jie Wang, Victor Magron, Jean B. Lasserre

Abstract: We report the experimental results on certifying 1% global optimality of solutions of AC-OPF instances from PGLiB via the CS-TSSOS hierarchy -- a moment-SOS based hierarchy that exploits both correlative and term sparsity, which can provide tighter SDP relaxations than Shor's relaxation. Our numerical experiments demonstrate that the CS-TSSOS hierarchy scales well with the problem size and is inde… ▽ More We report the experimental results on certifying 1% global optimality of solutions of AC-OPF instances from PGLiB via the CS-TSSOS hierarchy -- a moment-SOS based hierarchy that exploits both correlative and term sparsity, which can provide tighter SDP relaxations than Shor's relaxation. Our numerical experiments demonstrate that the CS-TSSOS hierarchy scales well with the problem size and is indeed useful in certifying global optimality of solutions for large-scale real world problems, e.g., the AC-OPF problem. In particular, we are able to certify 1% global optimality for a challenging AC-OPF instance with 6515 buses involving 14398 real variables and 63577 constraints. △ Less

Submitted 20 July, 2022; v1 submitted 21 September, 2021; originally announced September 2021.

Comments: 18 pages, 5 figures, 4 tables

MSC Class: 90C23; 14P10; 90C22; 90C26; 12D15

Noncommutative Polynomial Optimization

Authors: Abhishek Bhardwaj, Igor Klep, Victor Magron

Abstract: In this chapter we present the sums of Hermitian squares approach to noncommutative polynomial optimization problems. This is an extension of the sums of squares approach for polynomial optimization arising from real algebraic geometry. We provide a gentle introduction to the underlying theory of this methodology and highlight its importance. In this chapter we present the sums of Hermitian squares approach to noncommutative polynomial optimization problems. This is an extension of the sums of squares approach for polynomial optimization arising from real algebraic geometry. We provide a gentle introduction to the underlying theory of this methodology and highlight its importance. △ Less

Submitted 2 September, 2021; originally announced September 2021.

Comments: 10 pages

Dimension-free entanglement detection in multipartite Werner states

Authors: Felix Huber, Igor Klep, Victor Magron, Jurij Volčič

Abstract: Werner states are multipartite quantum states that are invariant under the diagonal conjugate action of the unitary group. This paper gives a complete characterization of their entanglement that is independent of the underlying local Hilbert space: for every entangled Werner state there exists a dimension-free entanglement witness. The construction of such a witness is formulated as an optimizatio… ▽ More Werner states are multipartite quantum states that are invariant under the diagonal conjugate action of the unitary group. This paper gives a complete characterization of their entanglement that is independent of the underlying local Hilbert space: for every entangled Werner state there exists a dimension-free entanglement witness. The construction of such a witness is formulated as an optimization problem. To solve it, two semidefinite programming hierarchies are introduced. The first one is derived using real algebraic geometry applied to positive polynomials in the entries of a Gram matrix, and is complete in the sense that for every entangled Werner state it converges to a witness. The second one is based on a sum-of-squares certificate for the positivity of trace polynomials in noncommuting variables, and is a relaxation that involves smaller semidefinite constraints. △ Less

Submitted 11 October, 2022; v1 submitted 19 August, 2021; originally announced August 2021.

Comments: updated abstract

MSC Class: 81P42; 46N50; 20C35; 90C22; 81-08; 16R30; 13J30

Journal ref: Communications in Mathematical Physics 396, 1051-1070 (2022)

Sum of Squares Decompositions of Polynomials over their Gradient Ideals with Rational Coefficients

Authors: Victor Magron, Mohab Safey El Din, Trung-Hieu Vu

Abstract: Assessing non-negativity of multivariate polynomials over the reals, through the computation of {\em certificates of non-negativity}, is a topical issue in polynomial optimization. This is usually tackled through the computation of {\em sums-of-squares decompositions} which rely on efficient numerical solvers for semi-definite programming. This method faces two difficulties. The first one is that… ▽ More Assessing non-negativity of multivariate polynomials over the reals, through the computation of {\em certificates of non-negativity}, is a topical issue in polynomial optimization. This is usually tackled through the computation of {\em sums-of-squares decompositions} which rely on efficient numerical solvers for semi-definite programming. This method faces two difficulties. The first one is that the certificates obtained this way are {\em approximate} and then non-exact. The second one is due to the fact that not all non-negative polynomials are sums-of-squares. In this paper, we build on previous works by Parrilo, Nie, Demmel and Sturmfels who introduced certificates of non-negativity modulo {\em gradient ideals}. We prove that, actually, such certificates can be obtained {\em exactly}, over the rationals if the polynomial under consideration has rational coefficients and we provide {\em exact} algorithms to compute them. We analyze the bit complexity of these algorithms and deduce bit size bounds of such certificates. △ Less

Submitted 25 July, 2021; originally announced July 2021.

Comments: 24 pages, 2 tables

Noncommutative Christoffel-Darboux Kernels

Authors: Serban T. Belinschi, Victor Magron, Victor Vinnikov

Abstract: We introduce from an analytic perspective Christoffel-Darboux kernels associated to bounded, tracial noncommutative distributions. We show that properly normalized traces, respectively norms, of evaluations of such kernels on finite dimensional matrices yield classical plurisubharmonic functions as the degree tends to infinity, and show that they are comparable to certain noncommutative versions o… ▽ More We introduce from an analytic perspective Christoffel-Darboux kernels associated to bounded, tracial noncommutative distributions. We show that properly normalized traces, respectively norms, of evaluations of such kernels on finite dimensional matrices yield classical plurisubharmonic functions as the degree tends to infinity, and show that they are comparable to certain noncommutative versions of the Siciak extremal function. We prove estimates for Siciak functions associated to free products of distributions, and use the classical theory of plurisubharmonic functions in order to propose a notion of support for noncommutative distributions. We conclude with some conjectures and numerical experiments. △ Less

Submitted 12 January, 2022; v1 submitted 11 June, 2021; originally announced June 2021.

Comments: 48 pages, 2 figures

Semialgebraic Representation of Monotone Deep Equilibrium Models and Applications to Certification

Authors: Tong Chen, Jean-Bernard Lasserre, Victor Magron, Edouard Pauwels

Abstract: Deep equilibrium models are based on implicitly defined functional relations and have shown competitive performance compared with the traditional deep networks. Monotone operator equilibrium networks (monDEQ) retain interesting performance with additional theoretical guaranties. Existing certification tools for classical deep networks cannot directly be applied to monDEQs for which much fewer tool… ▽ More Deep equilibrium models are based on implicitly defined functional relations and have shown competitive performance compared with the traditional deep networks. Monotone operator equilibrium networks (monDEQ) retain interesting performance with additional theoretical guaranties. Existing certification tools for classical deep networks cannot directly be applied to monDEQs for which much fewer tools exist. We introduce a semialgebraic representation for ReLU based monDEQs which allows to approximate the corresponding input output relation by semidefinite programming (SDP). We present several applications to network certification and obtain SDP models for the following problems : robustness certification, Lipschitz constant estimation, ellipsoidal uncertainty propagation. We use these models to certify robustness of monDEQs w.r.t. a general $L_q$ norm. Experimental results show that the proposed models outperform existing approaches for monDEQ certification. Furthermore, our investigations suggest that monDEQs are much more robust to $L_2$ perturbations than $L_{\infty}$ perturbations. △ Less

Submitted 2 June, 2021; originally announced June 2021.

Comments: 16 pages, 4 tables, 2 figures

On the complexity of Putinar-Vasilescu's Positivstellensatz

Authors: Ngoc Hoang Anh Mai, Victor Magron

Abstract: We provide a new degree bound on the weighted sum-of-squares (SOS) polynomials for Putinar-Vasilescu's Positivstellensatz. This leads to another Positivstellensatz saying that if $f$ is a polynomial of degree at most $2 d_f$ nonnegative on a semialgebraic set having nonempty interior defined by finitely many polynomial inequalities $g_j(x)\ge 0$, $j=1,\dots,m$ with $g_1:=L-\|x\|_2^2$ for some… ▽ More We provide a new degree bound on the weighted sum-of-squares (SOS) polynomials for Putinar-Vasilescu's Positivstellensatz. This leads to another Positivstellensatz saying that if $f$ is a polynomial of degree at most $2 d_f$ nonnegative on a semialgebraic set having nonempty interior defined by finitely many polynomial inequalities $g_j(x)\ge 0$, $j=1,\dots,m$ with $g_1:=L-\|x\|_2^2$ for some $L>0$, then there exist positive constants $\bar c$ and $c$ depending on $f,g_j$ such that for any $\varepsilon>0$, for all $k\ge \bar c \varepsilon^{-c}$, $f$ has the decomposition \begin{equation} \begin{array}{l} (1+\|x\|_2^2)^k(f+\varepsilon)=σ_0+\sum_{j=1}^m σ_jg_j \,, \end{array} \end{equation} for some SOS polynomials $σ_j$ being such that the degrees of $σ_0,σ_jg_j$ are at most $2(d_f+k)$. Here $\|\cdot\|_2$ denotes the $\ell_2$ vector norm. As a consequence, we obtain a converging hierarchy of semidefinite relaxations for lower bounds in polynomial optimization on basic compact semialgebraic sets. The complexity of this hierarchy is $\mathcal{O}(\varepsilon^{-c})$ for prescribed accuracy $\varepsilon>0$. In particular, if $m=L=1$ then $c=65$, yielding the complexity $\mathcal{O}(\varepsilon^{-65})$ for the minimization of a polynomial on the unit ball. Our result improves the complexity bound $\mathcal{O}(\exp(\varepsilon^{-c}))$ due to Nie and Schweighofer in [Journal of Complexity 23.1 (2007): 135-150]. △ Less

Submitted 27 May, 2021; v1 submitted 23 April, 2021; originally announced April 2021.

Comments: 24 pages, 1 figure

Exploiting Sparsity in Complex Polynomial Optimization

Authors: Jie Wang, Victor Magron

Abstract: In this paper, we study the sparsity-adapted complex moment-Hermitian sum of squares (moment-HSOS) hierarchy for complex polynomial optimization problems, where the sparsity includes correlative sparsity and term sparsity. We compare the strengths of the sparsity-adapted complex moment-HSOS hierarchy with the sparsity-adapted real moment-SOS hierarchy on either randomly generated complex polynomia… ▽ More In this paper, we study the sparsity-adapted complex moment-Hermitian sum of squares (moment-HSOS) hierarchy for complex polynomial optimization problems, where the sparsity includes correlative sparsity and term sparsity. We compare the strengths of the sparsity-adapted complex moment-HSOS hierarchy with the sparsity-adapted real moment-SOS hierarchy on either randomly generated complex polynomial optimization problems or the AC optimal power flow problem. The results of numerical experiments show that the sparsity-adapted complex moment-HSOS hierarchy provides a trade-off between the computational cost and the quality of obtained bounds for large-scale complex polynomial optimization problems. △ Less

Submitted 23 March, 2021; originally announced March 2021.

Comments: 20 pages, 6 tables

TSSOS: a Julia library to exploit sparsity for large-scale polynomial optimization

Authors: Victor Magron, Jie Wang

Abstract: The Julia library TSSOS aims at hel** polynomial optimizers to solve large-scale problems with sparse input data. The underlying algorithmic framework is based on exploiting correlative and term sparsity to obtain a new moment-SOS hierarchy involving potentially much smaller positive semidefinite matrices. TSSOS can be applied to numerous problems ranging from power networks to eigenvalue and tr… ▽ More The Julia library TSSOS aims at hel** polynomial optimizers to solve large-scale problems with sparse input data. The underlying algorithmic framework is based on exploiting correlative and term sparsity to obtain a new moment-SOS hierarchy involving potentially much smaller positive semidefinite matrices. TSSOS can be applied to numerous problems ranging from power networks to eigenvalue and trace optimization of noncommutative polynomials, involving up to tens of thousands of variables and constraints. △ Less

Submitted 1 March, 2021; originally announced March 2021.

Comments: 10 pages, 2 figures, 2 tables

The Constant Trace Property in Noncommutative Optimization

Authors: Ngoc Hoang Anh Mai, Abhishek Bhardwaj, Victor Magron

Abstract: In this article, we show that each semidefinite relaxation of a ball-constrained noncommutative polynomial optimization problem can be cast as a semidefinite program with a constant trace matrix variable. We then demonstrate how this constant trace property can be exploited via first order numerical methods to solve efficiently the semidefinite relaxations of the noncommutative problem. In this article, we show that each semidefinite relaxation of a ball-constrained noncommutative polynomial optimization problem can be cast as a semidefinite program with a constant trace matrix variable. We then demonstrate how this constant trace property can be exploited via first order numerical methods to solve efficiently the semidefinite relaxations of the noncommutative problem. △ Less

Submitted 3 February, 2021; originally announced February 2021.

Comments: 8 pages, 3 tables

Stability of Linear Systems under Extended Weakly-Hard Constraints

Authors: Nils Vreman, Paolo Pazzaglia, Jie Wang, Victor Magron, Martina Maggio

Abstract: Control systems can show robustness to many events, like disturbances and model inaccuracies. It is natural to speculate that they are also robust to sporadic deadline misses when implemented as digital tasks on an embedded platform. This paper proposes a comprehensive stability analysis for control systems subject to deadline misses, leveraging a new formulation to describe the patterns experienc… ▽ More Control systems can show robustness to many events, like disturbances and model inaccuracies. It is natural to speculate that they are also robust to sporadic deadline misses when implemented as digital tasks on an embedded platform. This paper proposes a comprehensive stability analysis for control systems subject to deadline misses, leveraging a new formulation to describe the patterns experienced by the control task under different handling strategies. Such analysis brings the assessment of control systems robustness to computational problems one step closer to the controller implementation. △ Less

Submitted 30 August, 2022; v1 submitted 27 January, 2021; originally announced January 2021.

Comments: 6 pages, 1 table, published in IEEE Control Systems Letters and in the proceedings of the Control and Decision Conference 2022

A Sublevel Moment-SOS Hierarchy for Polynomial Optimization

Authors: Tong Chen, Jean-Bernard Lasserre, Victor Magron, Edouard Pauwels

Abstract: We introduce a sublevel Moment-SOS hierarchy where each SDP relaxation can be viewed as an intermediate (or interpolation) between the d-th and (d+1)-th order SDP relaxations of the Moment-SOS hierarchy (dense or sparse version). With the flexible choice of determining the size (level) and number (depth) of subsets in the SDP relaxation, one is able to obtain different improvements compared to the… ▽ More We introduce a sublevel Moment-SOS hierarchy where each SDP relaxation can be viewed as an intermediate (or interpolation) between the d-th and (d+1)-th order SDP relaxations of the Moment-SOS hierarchy (dense or sparse version). With the flexible choice of determining the size (level) and number (depth) of subsets in the SDP relaxation, one is able to obtain different improvements compared to the d-th order relaxation, based on the machine memory capacity. In particular, we provide numerical experiments for d=1 and various types of problems both in combinatorial optimization (Max-Cut, Mixed Integer Programming) and deep learning (robustness certification, Lipschitz constant of neural networks), where the standard Lasserre's relaxation (or its sparse variant) is computationally intractable. In our numerical results, the lower bounds from the sublevel relaxations improve the bound from Shor's relaxation (first order Lasserre's relaxation) and are significantly closer to the optimal value or to the best-known lower/upper bounds. △ Less

Submitted 13 January, 2021; originally announced January 2021.

Comments: 25 pages, 13 tables

Exploiting constant trace property in large-scale polynomial optimization

Authors: Ngoc Hoang Anh Mai, Jean-Bernard Lasserre, Victor Magron, Jie Wang

Abstract: We prove that every semidefinite moment relaxation of a polynomial optimization problem (POP) with a ball constraint can be reformulated as a semidefinite program involving a matrix with constant trace property (CTP). As a result such moment relaxations can be solved efficiently by first-order methods that exploit CTP, e.g., the conditional gradient-based augmented Lagrangian method. We also exten… ▽ More We prove that every semidefinite moment relaxation of a polynomial optimization problem (POP) with a ball constraint can be reformulated as a semidefinite program involving a matrix with constant trace property (CTP). As a result such moment relaxations can be solved efficiently by first-order methods that exploit CTP, e.g., the conditional gradient-based augmented Lagrangian method. We also extend this CTP-exploiting framework to large-scale POPs with different sparsity structures. The efficiency and scalability of our framework are illustrated on second-order moment relaxations for various randomly generated quadratically constrained quadratic programs. △ Less

Submitted 16 December, 2020; originally announced December 2020.

Comments: 43 pages, 6 algorithms, 23 tables

SONC Optimization and Exact Nonnegativity Certificates via Second-Order Cone Programming

Authors: Victor Magron, Jie Wang

Abstract: The second-order cone (SOC) is a class of simple convex cones and optimizing over them can be done more efficiently than with semidefinite programming. It is interesting both in theory and in practice to investigate which convex cones admit a representation using SOCs, given that they have a strong expressive ability. In this paper, we prove constructively that the cone of sums of nonnegative circ… ▽ More The second-order cone (SOC) is a class of simple convex cones and optimizing over them can be done more efficiently than with semidefinite programming. It is interesting both in theory and in practice to investigate which convex cones admit a representation using SOCs, given that they have a strong expressive ability. In this paper, we prove constructively that the cone of sums of nonnegative circuits (SONC) admits a SOC representation. Based on this, we give a new algorithm for unconstrained polynomial optimization via SOC programming. We also provide a hybrid numeric-symbolic scheme which combines the numerical procedure with a rounding-projection algorithm to obtain exact nonnegativity certificates. Numerical experiments demonstrate the efficiency of our algorithm for polynomials with fairly large degree and number of variables. △ Less

Submitted 24 August, 2021; v1 submitted 14 December, 2020; originally announced December 2020.

Comments: 29 pages, 7 tables, 6 figures, extended version of the article published in the proceedings of ISSAC 2020. arXiv admin note: text overlap with arXiv:1906.06179

Minimizing rational functions: a hierarchy of approximations via pushforward measures

Authors: Jean Bernard Lasserre, Victor Magron, Swann Marx, Olivier Zahm

Abstract: This paper is concerned with minimizing a sum of rational functions over a compact set of high-dimension. Our approach relies on the second Lasserre's hierarchy (also known as the upper bounds hierarchy) formulated on the pushforward measure in order to work in a space of smaller dimension. We show that in the general case the minimum can be approximated as closely as desired from above with a hie… ▽ More This paper is concerned with minimizing a sum of rational functions over a compact set of high-dimension. Our approach relies on the second Lasserre's hierarchy (also known as the upper bounds hierarchy) formulated on the pushforward measure in order to work in a space of smaller dimension. We show that in the general case the minimum can be approximated as closely as desired from above with a hierarchy of semidefinite programs problems or, in the particular case of a single fraction, with a hierarchy of generalized eigenvalue problems. We numerically illustrate the potential of using the pushforward measure rather than the standard upper bounds hierarchy. In our opinion, this potential should be a strong incentive to investigate a related challenging problem interesting in its own; namely integrating an arbitrary power of a given polynomial on a simple set (e.g., unit box or unit sphere) with respect to Lebesgue or Haar measure. △ Less

Submitted 10 December, 2020; originally announced December 2020.

Comments: 24 pages, 3 tables

$l_2$ Induced Norm Analysis of Discrete-Time LTI Systems for Nonnegative Input Signals and Its Application to Stability Analysis of Recurrent Neural Networks

Authors: Yoshio Ebihara, Hayato Waki, Victor Magron, Ngoc Hoang Anh Mai, Dimitri Peaucelle, Sophie Tarbouriech

Abstract: In this paper, we focus on the "positive" $l_2$ induced norm of discrete-time linear time-invariant systems where the input signals are restricted to be nonnegative. To cope with the nonnegativity of the input signals, we employ copositive programming as the mathematical tool for the analysis. Then, by applying an inner approximation to the copositive cone, we derive numerically tractable semidefi… ▽ More In this paper, we focus on the "positive" $l_2$ induced norm of discrete-time linear time-invariant systems where the input signals are restricted to be nonnegative. To cope with the nonnegativity of the input signals, we employ copositive programming as the mathematical tool for the analysis. Then, by applying an inner approximation to the copositive cone, we derive numerically tractable semidefinite programming problems for the upper and lower bound computation of the "positive" $l_2$ induced norm. This norm is typically useful for the stability analysis of feedback systems constructed from an LTI system and nonlinearities where the nonlinear elements provide only nonnegative signals. As a concrete example, we illustrate the usefulness of the "positive" $l_2$ induced norm for the stability analysis of recurrent neural networks with activation functions being rectified linear units. △ Less

Submitted 25 November, 2020; originally announced November 2020.

Comments: 8 pages, 3 figures

Exploiting term sparsity in Noncommutative Polynomial Optimization

Authors: Jie Wang, Victor Magron

Abstract: We provide a new hierarchy of semidefinite programming relaxations, called NCTSSOS, to solve large-scale sparse noncommutative polynomial optimization problems. This hierarchy features the exploitation of term sparsity hidden in the input data for eigenvalue and trace optimization problems. NCTSSOS complements the recent work that exploits correlative sparsity for noncommutative optimization probl… ▽ More We provide a new hierarchy of semidefinite programming relaxations, called NCTSSOS, to solve large-scale sparse noncommutative polynomial optimization problems. This hierarchy features the exploitation of term sparsity hidden in the input data for eigenvalue and trace optimization problems. NCTSSOS complements the recent work that exploits correlative sparsity for noncommutative optimization problems by Klep, Magron and Povh in arXiv:1909.00569, and is the noncommutative analogue of the TSSOS framework by Wang, Magron and Lasserre in arXiv:1912.08899. We also propose an extension exploiting simultaneously correlative and term sparsity, as done previously in the commutative case arXiv:2005.02828. Under certain conditions, we prove that the optimums of the NCTSSOS hierarchy converge to the optimum of the corresponding dense SDP relaxation. We illustrate the efficiency and scalability of NCTSSOS by solving eigenvalue/trace optimization problems from the literature as well as randomly generated examples involving up to several thousands of variables. △ Less

Submitted 14 October, 2020; originally announced October 2020.

Comments: 33 pages, 5 figures, 12 tables

MSC Class: Primary; 47N10; 90C22; Secondary; 12D15; 14P10

Correlations constrained by composite measurements

Authors: John H. Selby, Ana Belén Sainz, Victor Magron, Łukasz Czekaj, Michał Horodecki

Abstract: How to understand the set of correlations admissible in nature is one outstanding open problem in the core of the foundations of quantum theory. Here we take a complementary viewpoint to the device-independent approach, and explore the correlations that physical theories may feature when restricted by some particular constraints on their measurements. We show that demanding that a theory exhibits… ▽ More How to understand the set of correlations admissible in nature is one outstanding open problem in the core of the foundations of quantum theory. Here we take a complementary viewpoint to the device-independent approach, and explore the correlations that physical theories may feature when restricted by some particular constraints on their measurements. We show that demanding that a theory exhibits {a composite} measurement imposes a hierarchy of constraints on the structure of its sets of states and effects, which translate to a hierarchy of constraints on the allowed correlations themselves. We moreover focus on the particular case where one demands the existence of a correlated measurement that reads out the parity of local fiducial measurements. By formulating a non-linear Optimisation Problem, and semidefinite relaxations of it, we explore the consequences of the existence of such a parity reading measurement for violations of Bell inequalities. In particular, we show that in certain situations this assumption has surprisingly strong consequences, namely, that Tsirelson's bound can be recovered. △ Less

Submitted 3 August, 2023; v1 submitted 10 September, 2020; originally announced September 2020.

Comments: 43 pages + appendices. V4 published version

Journal ref: Quantum 7, 1080 (2023)

SparseJSR: A Fast Algorithm to Compute Joint Spectral Radius via Sparse SOS Decompositions

Authors: Jie Wang, Martina Maggio, Victor Magron

Abstract: This paper focuses on the computation of joint spectral radii (JSR), when the involved matrices are sparse. We provide a sparse variant of the procedure proposed by Parrilo and Jadbabaie, to compute upper bounds of the JSR by means of sum-of-squares (SOS) relaxations. Our resulting iterative algorithm, called SparseJSR, is based on the term sparsity SOS (TSSOS) framework, developed by Wang, Magron… ▽ More This paper focuses on the computation of joint spectral radii (JSR), when the involved matrices are sparse. We provide a sparse variant of the procedure proposed by Parrilo and Jadbabaie, to compute upper bounds of the JSR by means of sum-of-squares (SOS) relaxations. Our resulting iterative algorithm, called SparseJSR, is based on the term sparsity SOS (TSSOS) framework, developed by Wang, Magron and Lasserre, yielding SOS decompositions of polynomials with arbitrary sparse support. SparseJSR exploits the sparsity of the input matrices to significantly reduce the computational burden associated with the JSR computation. Our algorithmic framework is then successfully applied to compute upper bounds for JSR, on randomly generated benchmarks as well as on problems arising from stability proofs of controllers, in relation with possible hardware and software faults. △ Less

Submitted 18 March, 2021; v1 submitted 26 August, 2020; originally announced August 2020.

Comments: 6 pages, 2 figures, 3 tables

A hierarchy of spectral relaxations for polynomial optimization

Authors: Ngoc Hoang Anh Mai, Victor Magron, Jean-Bernard Lasserre

Abstract: We show that (i) any constrained polynomial optimization problem (POP) has an equivalent formulation on a variety contained in an Euclidean sphere and (ii) the resulting semidefinite relaxations in the moment-SOS hierarchy have the constant trace property (CTP) for the involved matrices. We then exploit the CTP to avoid solving the semidefinite relaxations via interior-point methods and rather use… ▽ More We show that (i) any constrained polynomial optimization problem (POP) has an equivalent formulation on a variety contained in an Euclidean sphere and (ii) the resulting semidefinite relaxations in the moment-SOS hierarchy have the constant trace property (CTP) for the involved matrices. We then exploit the CTP to avoid solving the semidefinite relaxations via interior-point methods and rather use ad-hoc spectral methods that minimize the largest eigenvalue of a matrix pencil. Convergence to the optimal value of the semidefinite relaxation is guaranteed. As a result we obtain a hierarchy of nonsmooth "spectral relaxations" of the initial POP. Efficiency and robustness of this spectral hierarchy is tested against several equality constrained POPs on a sphere as well as on a sample of randomly generated quadratically constrained quadratic problems (QCQPs). △ Less

Submitted 17 July, 2020; originally announced July 2020.

Comments: 38 pages, 6 figures, 11 tables

Optimization over trace polynomials

Authors: Igor Klep, Victor Magron, Jurij Volčič

Abstract: Motivated by recent progress in quantum information theory, this article aims at optimizing trace polynomials, i.e., polynomials in noncommuting variables and traces of their products. A novel Positivstellensatz certifying positivity of trace polynomials subject to trace constraints is presented, and a hierarchy of semidefinite relaxations converging monotonically to the optimum of a trace polynom… ▽ More Motivated by recent progress in quantum information theory, this article aims at optimizing trace polynomials, i.e., polynomials in noncommuting variables and traces of their products. A novel Positivstellensatz certifying positivity of trace polynomials subject to trace constraints is presented, and a hierarchy of semidefinite relaxations converging monotonically to the optimum of a trace polynomial subject to tracial constraints is provided. This hierarchy can be seen as a tracial analog of the Pironio, Navascués and Acín scheme [New J. Phys., 2008] for optimization of noncommutative polynomials. The Gelfand-Naimark-Segal (GNS) construction is applied to extract optimizers of the trace optimization problem if flatness and extremality conditions are satisfied. These conditions are sufficient to obtain finite convergence of our hierarchy. The results obtained are applied to violations of polynomial Bell inequalities in quantum information theory. The main techniques used in this paper are inspired by real algebraic geometry, operator theory, and noncommutative algebra. △ Less

Submitted 29 March, 2021; v1 submitted 22 June, 2020; originally announced June 2020.

Comments: 32 pages

MSC Class: 46N50; 90C22; 47N10; 13J30

Journal ref: Ann. Henri Poincaré 23 (2022) 67-100

Computer-assisted proofs for Lyapunov stability via Sums of Squares certificates and Constructive Analysis

Authors: Grigory Devadze, Victor Magron, Stefan Streif

Abstract: We provide a computer-assisted approach to ensure that a given continuous or discrete-time polynomial system is (asymptotically) stable. Our framework relies on constructive analysis together with formally certified sums of squares Lyapunov functions. The crucial steps are formalized within of the proof assistant Minlog. We illustrate our approach with various examples issued from the control syst… ▽ More We provide a computer-assisted approach to ensure that a given continuous or discrete-time polynomial system is (asymptotically) stable. Our framework relies on constructive analysis together with formally certified sums of squares Lyapunov functions. The crucial steps are formalized within of the proof assistant Minlog. We illustrate our approach with various examples issued from the control system literature. △ Less

Submitted 17 June, 2020; originally announced June 2020.

Comments: 21 pages

CS-TSSOS: Correlative and term sparsity for large-scale polynomial optimization

Authors: Jie Wang, Victor Magron, Jean B. Lasserre, Ngoc Hoang Anh Mai

Abstract: This work proposes a new moment-SOS hierarchy, called CS-TSSOS, for solving large-scale sparse polynomial optimization problems. Its novelty is to exploit simultaneously correlative sparsity and term sparsity by combining advantages of two existing frameworks for sparse polynomial optimization. The former is due to Waki et al. while the latter was initially proposed by Wang et al. and later exploi… ▽ More This work proposes a new moment-SOS hierarchy, called CS-TSSOS, for solving large-scale sparse polynomial optimization problems. Its novelty is to exploit simultaneously correlative sparsity and term sparsity by combining advantages of two existing frameworks for sparse polynomial optimization. The former is due to Waki et al. while the latter was initially proposed by Wang et al. and later exploited in the TSSOS hierarchy. In doing so we obtain CS-TSSOS -- a two-level hierarchy of semidefinite programming relaxations with (i), the crucial property to involve blocks of SDP matrices and (ii), the guarantee of convergence to the global optimum under certain conditions. We demonstrate its efficiency and scalability on several large-scale instances of the celebrated Max-Cut problem and the important industrial optimal power flow problem, involving up to six thousand variables and tens of thousands of constraints. △ Less

Submitted 8 June, 2021; v1 submitted 6 May, 2020; originally announced May 2020.

Comments: 28 pages, 8 figures, 8 tables