Showing 1–15 of 15 results for author: Lasserre, J

Verifying Properties of Binary Neural Networks Using Sparse Polynomial Optimization

Authors: Jianting Yang, Srećko Ðurašinović, Jean-Bernard Lasserre, Victor Magron, Jun Zhao

Abstract: This paper explores methods for verifying the properties of Binary Neural Networks (BNNs), focusing on robustness against adversarial attacks. Despite their lower computational and memory needs, BNNs, like their full-precision counterparts, are also sensitive to input perturbations. Established methods for solving this problem are predominantly based on Satisfiability Modulo Theories and Mixed-Int… ▽ More This paper explores methods for verifying the properties of Binary Neural Networks (BNNs), focusing on robustness against adversarial attacks. Despite their lower computational and memory needs, BNNs, like their full-precision counterparts, are also sensitive to input perturbations. Established methods for solving this problem are predominantly based on Satisfiability Modulo Theories and Mixed-Integer Linear Programming techniques, which are characterized by NP complexity and often face scalability issues. We introduce an alternative approach using Semidefinite Programming relaxations derived from sparse Polynomial Optimization. Our approach, compatible with continuous input space, not only mitigates numerical issues associated with floating-point calculations but also enhances verification scalability through the strategic use of tighter first-order semidefinite relaxations. We demonstrate the effectiveness of our method in verifying robustness against both $\|.\|_\infty$ and $\|.\|_2$-based adversarial attacks. △ Less

Submitted 27 May, 2024; originally announced May 2024.

Comments: 22 pages, 2 figures, 7 tables

Uncertainty Quantification of Set-Membership Estimation in Control and Perception: Revisiting the Minimum Enclosing Ellipsoid

Authors: Yukai Tang, Jean-Bernard Lasserre, Heng Yang

Abstract: Set-membership estimation (SME) outputs a set estimator that guarantees to cover the groundtruth. Such sets are, however, defined by (many) abstract (and potentially nonconvex) constraints and therefore difficult to manipulate. We present tractable algorithms to compute simple and tight overapproximations of SME in the form of minimum enclosing ellipsoids (MEE). We first introduce the hierarchy of… ▽ More Set-membership estimation (SME) outputs a set estimator that guarantees to cover the groundtruth. Such sets are, however, defined by (many) abstract (and potentially nonconvex) constraints and therefore difficult to manipulate. We present tractable algorithms to compute simple and tight overapproximations of SME in the form of minimum enclosing ellipsoids (MEE). We first introduce the hierarchy of enclosing ellipsoids proposed by Nie and Demmel (2005), based on sums-of-squares relaxations, that asymptotically converge to the MEE of a basic semialgebraic set. This framework, however, struggles in modern control and perception problems due to computational challenges. We contribute three computational enhancements to make this framework practical, namely constraints pruning, generalized relaxed Chebyshev center, and handling non-Euclidean geometry. We showcase numerical examples on system identification and object pose estimation. △ Less

Submitted 20 May, 2024; v1 submitted 27 November, 2023; originally announced November 2023.

Comments: Accepted to 6th Learning for Dynamics and Control (L4DC) as oral presentation

Tractable hierarchies of convex relaxations for polynomial optimization on the nonnegative orthant

Authors: Ngoc Hoang Anh Mai, Victor Magron, Jean-Bernard Lasserre, Kim-Chuan Toh

Abstract: We consider polynomial optimization problems (POP) on a semialgebraic set contained in the nonnegative orthant (every POP on a compact set can be put in this format by a simple translation of the origin). Such a POP can be converted to an equivalent POP by squaring each variable. Using even symmetry and the concept of factor width, we propose a hierarchy of semidefinite relaxations based on the ex… ▽ More We consider polynomial optimization problems (POP) on a semialgebraic set contained in the nonnegative orthant (every POP on a compact set can be put in this format by a simple translation of the origin). Such a POP can be converted to an equivalent POP by squaring each variable. Using even symmetry and the concept of factor width, we propose a hierarchy of semidefinite relaxations based on the extension of Pólya's Positivstellensatz by Dickinson-Povh. As its distinguishing and crucial feature, the maximal matrix size of each resulting semidefinite relaxation can be chosen arbitrarily and in addition, we prove that the sequence of values returned by the new hierarchy converges to the optimal value of the original POP at the rate $O(\varepsilon^{-c})$ if the semialgebraic set has nonempty interior. When applied to (i) robustness certification of multi-layer neural networks and (ii) computation of positive maximal singular values, our method based on Pólya's Positivstellensatz provides better bounds and runs several hundred times faster than the standard Moment-SOS hierarchy. △ Less

Submitted 13 September, 2022; originally announced September 2022.

Comments: 39 pages, 15 tables

Piecewise-Linear Motion Planning amidst Static, Moving, or Morphing Obstacles

Authors: Bachir El Khadir, Jean Bernard Lasserre, Vikas Sindhwani

Abstract: We propose a novel method for planning shortest length piecewise-linear motions through complex environments punctured with static, moving, or even morphing obstacles. Using a moment optimization approach, we formulate a hierarchy of semidefinite programs that yield increasingly refined lower bounds converging monotonically to the optimal path length. For computational tractability, our global m… ▽ More We propose a novel method for planning shortest length piecewise-linear motions through complex environments punctured with static, moving, or even morphing obstacles. Using a moment optimization approach, we formulate a hierarchy of semidefinite programs that yield increasingly refined lower bounds converging monotonically to the optimal path length. For computational tractability, our global moment optimization approach motivates an iterative motion planner that outperforms competing sampling-based and nonlinear optimization baselines. Our method natively handles continuous time constraints without any need for time discretization, and has the potential to scale better with dimensions compared to popular sampling-based methods. △ Less

Submitted 16 October, 2020; originally announced October 2020.

CS-TSSOS: Correlative and term sparsity for large-scale polynomial optimization

Authors: Jie Wang, Victor Magron, Jean B. Lasserre, Ngoc Hoang Anh Mai

Abstract: This work proposes a new moment-SOS hierarchy, called CS-TSSOS, for solving large-scale sparse polynomial optimization problems. Its novelty is to exploit simultaneously correlative sparsity and term sparsity by combining advantages of two existing frameworks for sparse polynomial optimization. The former is due to Waki et al. while the latter was initially proposed by Wang et al. and later exploi… ▽ More This work proposes a new moment-SOS hierarchy, called CS-TSSOS, for solving large-scale sparse polynomial optimization problems. Its novelty is to exploit simultaneously correlative sparsity and term sparsity by combining advantages of two existing frameworks for sparse polynomial optimization. The former is due to Waki et al. while the latter was initially proposed by Wang et al. and later exploited in the TSSOS hierarchy. In doing so we obtain CS-TSSOS -- a two-level hierarchy of semidefinite programming relaxations with (i), the crucial property to involve blocks of SDP matrices and (ii), the guarantee of convergence to the global optimum under certain conditions. We demonstrate its efficiency and scalability on several large-scale instances of the celebrated Max-Cut problem and the important industrial optimal power flow problem, involving up to six thousand variables and tens of thousands of constraints. △ Less

Submitted 8 June, 2021; v1 submitted 6 May, 2020; originally announced May 2020.

Comments: 28 pages, 8 figures, 8 tables

Semialgebraic Optimization for Lipschitz Constants of ReLU Networks

Authors: Tong Chen, Jean-Bernard Lasserre, Victor Magron, Edouard Pauwels

Abstract: The Lipschitz constant of a network plays an important role in many applications of deep learning, such as robustness certification and Wasserstein Generative Adversarial Network. We introduce a semidefinite programming hierarchy to estimate the global and local Lipschitz constant of a multiple layer deep neural network. The novelty is to combine a polynomial lifting for ReLU functions derivatives… ▽ More The Lipschitz constant of a network plays an important role in many applications of deep learning, such as robustness certification and Wasserstein Generative Adversarial Network. We introduce a semidefinite programming hierarchy to estimate the global and local Lipschitz constant of a multiple layer deep neural network. The novelty is to combine a polynomial lifting for ReLU functions derivatives with a weak generalization of Putinar's positivity certificate. This idea could also apply to other, nearly sparse, polynomial optimization problems in machine learning. We empirically demonstrate that our method provides a trade-off with respect to state of the art linear programming approach, and in some cases we obtain better bounds in less time. △ Less

Submitted 28 October, 2020; v1 submitted 10 February, 2020; originally announced February 2020.

Comments: NeurIPS 2020

Studio2Shop: from studio photo shoots to fashion articles

Authors: Julia Lasserre, Katharina Rasch, Roland Vollgraf

Abstract: Fashion is an increasingly important topic in computer vision, in particular the so-called street-to-shop task of matching street images with shop images containing similar fashion items. Solving this problem promises new means of making fashion searchable and hel** shoppers find the articles they are looking for. This paper focuses on finding pieces of clothing worn by a person in full-body or… ▽ More Fashion is an increasingly important topic in computer vision, in particular the so-called street-to-shop task of matching street images with shop images containing similar fashion items. Solving this problem promises new means of making fashion searchable and hel** shoppers find the articles they are looking for. This paper focuses on finding pieces of clothing worn by a person in full-body or half-body images with neutral backgrounds. Such images are ubiquitous on the web and in fashion blogs, and are typically studio photos, we refer to this setting as studio-to-shop. Recent advances in computational fashion include the development of domain-specific numerical representations. Our model Studio2Shop builds on top of such representations and uses a deep convolutional network trained to match a query image to the numerical feature vectors of all the articles annotated in this image. Top-$k$ retrieval evaluation on test query images shows that the correct items are most often found within a range that is sufficiently small for building realistic visual search engines for the studio-to-shop setting. △ Less

Submitted 2 July, 2018; originally announced July 2018.

Comments: 12 pages, 9 figures (Figure 1 has 5 subfigures, Figure 2 has 3 subfigures), 7 tables

Journal ref: Proceedings of the 7th International Conference on Pattern Recognition Applications and Methods (January 16-18, 2018, in Funchal, Madeira, Portugal), Vol. 1 (ISBN 978-989-758-276-9), P. 37-48

Approximate Optimal Designs for Multivariate Polynomial Regression

Authors: Yohann De Castro, Fabrice Gamboa, Didier Henrion, Roxana Hess, Jean-Bernard Lasserre

Abstract: We introduce a new approach aiming at computing approximate optimal designs for multivariate polynomial regressions on compact (semi-algebraic) design spaces. We use the moment-sum-of-squares hierarchy of semidefinite programming problems to solve numerically the approximate optimal design problem. The geometry of the design is recovered via semidefinite programming duality theory. This article sh… ▽ More We introduce a new approach aiming at computing approximate optimal designs for multivariate polynomial regressions on compact (semi-algebraic) design spaces. We use the moment-sum-of-squares hierarchy of semidefinite programming problems to solve numerically the approximate optimal design problem. The geometry of the design is recovered via semidefinite programming duality theory. This article shows that the hierarchy converges to the approximate optimal design as the order of the hierarchy increases. Furthermore, we provide a dual certificate ensuring finite convergence of the hierarchy and showing that the approximate optimal design can be computed numerically with our method. As a byproduct, we revisit the equivalence theorem of the experimental design theory: it is linked to the Christoffel polynomial and it characterizes finite convergence of the moment-sum-of-square hierarchies. △ Less

Submitted 25 October, 2017; v1 submitted 9 June, 2017; originally announced June 2017.

Comments: 30 Pages, 8 Figures. arXiv admin note: substantial text overlap with arXiv:1703.01777

MSC Class: 62K05; 90C25 (Primary) 41A10; 49M29; 90C90; 15A15 (secondary)

The empirical Christoffel function with applications in data analysis

Authors: Jean-Bernard Lasserre, Edouard Pauwels

Abstract: We illustrate the potential applications in machine learning of the Christoffel function, or more precisely, its empirical counterpart associated with a counting measure uniformly supported on a finite set of points. Firstly, we provide a thresholding scheme which allows to approximate the support of a measure from a finite subset of its moments with strong asymptotic guaranties. Secondly, we prov… ▽ More We illustrate the potential applications in machine learning of the Christoffel function, or more precisely, its empirical counterpart associated with a counting measure uniformly supported on a finite set of points. Firstly, we provide a thresholding scheme which allows to approximate the support of a measure from a finite subset of its moments with strong asymptotic guaranties. Secondly, we provide a consistency result which relates the empirical Christoffel function and its population counterpart in the limit of large samples. Finally, we illustrate the relevance of our results on simulated and real world datasets for several applications in statistics and machine learning: (a) density and support estimation from finite samples, (b) outlier and novelty detection and (c) affine matching. △ Less

Submitted 7 February, 2019; v1 submitted 11 January, 2017; originally announced January 2017.

Sorting out typicality with the inverse moment matrix SOS polynomial

Authors: Jean-Bernard Lasserre, Edouard Pauwels

Abstract: We study a surprising phenomenon related to the representation of a cloud of data points using polynomials. We start with the previously unnoticed empirical observation that, given a collection (a cloud) of data points, the sublevel sets of a certain distinguished polynomial capture the shape of the cloud very accurately. This distinguished polynomial is a sum-of-squares (SOS) derived in a simple… ▽ More We study a surprising phenomenon related to the representation of a cloud of data points using polynomials. We start with the previously unnoticed empirical observation that, given a collection (a cloud) of data points, the sublevel sets of a certain distinguished polynomial capture the shape of the cloud very accurately. This distinguished polynomial is a sum-of-squares (SOS) derived in a simple manner from the inverse of the empirical moment matrix. In fact, this SOS polynomial is directly related to orthogonal polynomials and the Christoffel function. This allows to generalize and interpret extremality properties of orthogonal polynomials and to provide a mathematical rationale for the observed phenomenon. Among diverse potential applications, we illustrate the relevance of our results on a network intrusion detection task for which we obtain performances similar to existing dedicated methods reported in the literature. △ Less

Submitted 14 June, 2016; v1 submitted 13 June, 2016; originally announced June 2016.

Exact solutions to Super Resolution on semi-algebraic domains in higher dimensions

Authors: Y De Castro, F Gamboa, D Henrion, J. -B Lasserre

Abstract: We investigate the multi-dimensional Super Resolution problem on closed semi-algebraic domains for various sampling schemes such as Fourier or moments. We present a new semidefinite programming (SDP) formulation of the 1 -minimization in the space of Radon measures in the multi-dimensional frame on semi-algebraic sets. While standard approaches have focused on SDP relaxations of the dual program (… ▽ More We investigate the multi-dimensional Super Resolution problem on closed semi-algebraic domains for various sampling schemes such as Fourier or moments. We present a new semidefinite programming (SDP) formulation of the 1 -minimization in the space of Radon measures in the multi-dimensional frame on semi-algebraic sets. While standard approaches have focused on SDP relaxations of the dual program (a popular approach is based on Gram matrix representations), this paper introduces an exact formulation of the primal 1 -minimization exact recovery problem of Super Resolution that unleashes standard techniques (such as moment-sum-of-squares hier-archies) to overcome intrinsic limitations of previous works in the literature. Notably, we show that one can exactly solve the Super Resolution problem in dimension greater than 2 and for a large family of domains described by semi-algebraic sets. △ Less

Submitted 9 February, 2015; originally announced February 2015.

Approximating Pareto Curves using Semidefinite Relaxations

Authors: Victor Magron, Didier Henrion, Jean-Bernard Lasserre

Abstract: We consider the problem of constructing an approximation of the Pareto curve associated with the multiobjective optimization problem $\min_{\mathbf{x} \in \mathbf{S}}\{ (f_1(\mathbf{x}), f_2(\mathbf{x})) \}$, where $f_1$ and $f_2$ are two conflicting polynomial criteria and $\mathbf{S} \subset \mathbb{R}^n$ is a compact basic semialgebraic set. We provide a systematic numerical scheme to approxima… ▽ More We consider the problem of constructing an approximation of the Pareto curve associated with the multiobjective optimization problem $\min_{\mathbf{x} \in \mathbf{S}}\{ (f_1(\mathbf{x}), f_2(\mathbf{x})) \}$, where $f_1$ and $f_2$ are two conflicting polynomial criteria and $\mathbf{S} \subset \mathbb{R}^n$ is a compact basic semialgebraic set. We provide a systematic numerical scheme to approximate the Pareto curve. We start by reducing the initial problem into a scalarized polynomial optimization problem (POP). Three scalarization methods lead to consider different parametric POPs, namely (a) a weighted convex sum approximation, (b) a weighted Chebyshev approximation, and (c) a parametric sublevel set approximation. For each case, we have to solve a semidefinite programming (SDP) hierarchy parametrized by the number of moments or equivalently the degree of a polynomial sums of squares approximation of the Pareto curve. When the degree of the polynomial approximation tends to infinity, we provide guarantees of convergence to the Pareto curve in $L^2$-norm for methods (a) and (b), and $L^1$-norm for method (c). △ Less

Submitted 16 June, 2014; v1 submitted 18 April, 2014; originally announced April 2014.

Report number: Rapport LAAS n° 14137

The inverse moment problem for convex polytopes

Authors: Nick Gravin, Jean Lasserre, Dmitrii Pasechnik, Sinai Robins

Abstract: The goal of this paper is to present a general and novel approach for the reconstruction of any convex d-dimensional polytope P, from knowledge of its moments. In particular, we show that the vertices of an N-vertex polytope in R^d can be reconstructed from the knowledge of O(DN) axial moments (w.r.t. to an unknown polynomial measure od degree D) in d+1 distinct generic directions. Our approach is… ▽ More The goal of this paper is to present a general and novel approach for the reconstruction of any convex d-dimensional polytope P, from knowledge of its moments. In particular, we show that the vertices of an N-vertex polytope in R^d can be reconstructed from the knowledge of O(DN) axial moments (w.r.t. to an unknown polynomial measure od degree D) in d+1 distinct generic directions. Our approach is based on the collection of moment formulas due to Brion, Lawrence, Khovanskii-Pukhikov, and Barvinok that arise in the discrete geometry of polytopes, and what variously known as Prony's method, or Vandermonde factorization of finite rank Hankel matrices. △ Less

Submitted 22 April, 2012; v1 submitted 28 June, 2011; originally announced June 2011.

Comments: LaTeX2e, 24 pages including 1 appendix

MSC Class: 52B11; 42B10; 47B35 ACM Class: F.2.1; F.2.2

Journal ref: Discrete and Computational Geometry, 48(2012), pp. 596--621

Certificates and relaxations for integer programming and the semi-group membership problem

Authors: Jean Lasserre, S. Zeron

Abstract: We consider integer programming and the semi-group membership problem. We provide the following theorem of the alternative: the system Ax=b has no nonnegative integral solution x if and only if p(b) <0 for some given polynomial p whose vector of coefficients lies in a convex cone that we characterize. We also provide a hierarchy of linear programming relaxations, where the continuous case Ax=b w… ▽ More We consider integer programming and the semi-group membership problem. We provide the following theorem of the alternative: the system Ax=b has no nonnegative integral solution x if and only if p(b) <0 for some given polynomial p whose vector of coefficients lies in a convex cone that we characterize. We also provide a hierarchy of linear programming relaxations, where the continuous case Ax=b with x real and nonnegative, describes the first relaxation in the hierarchy. △ Less

Submitted 11 May, 2009; originally announced May 2009.

Comments: 21 pages

MSC Class: 90; C10

Semidefinite Programming for Min-Max Problems and Games

Authors: Rida Laraki, Jean B. Lasserre

Abstract: We introduce two min-max problems: the first problem is to minimize the supremum of finitely many rational functions over a compact basic semi-algebraic set whereas the second problem is a 2-player zero-sum polynomial game in randomized strategies and with compact basic semi-algebraic pure strategy sets. It is proved that their optimal solution can be approximated by solving a hierarchy of semid… ▽ More We introduce two min-max problems: the first problem is to minimize the supremum of finitely many rational functions over a compact basic semi-algebraic set whereas the second problem is a 2-player zero-sum polynomial game in randomized strategies and with compact basic semi-algebraic pure strategy sets. It is proved that their optimal solution can be approximated by solving a hierarchy of semidefinite relaxations, in the spirit of the moment approach developed in Lasserre. This provides a unified approach and a class of algorithms to approximate all Nash equilibria and min-max strategies of many static and dynamic games. Each semidefinite relaxation can be solved in time which is polynomial in its input size and practice from global optimization suggests that very often few relaxations are needed for a good approximation (and sometimes even finite convergence). △ Less

Submitted 16 December, 2009; v1 submitted 17 October, 2008; originally announced October 2008.

Comments: 21 pages

Report number: Rapport LAAS 08582 MSC Class: 91A06; 91A25; 90C22; 47N10; 65K05