-
Complexity of Geometric programming in the Turing model and application to nonnegative tensors
Authors:
Shmuel Friedland,
Stéphane Gaubert
Abstract:
We consider a geometric programming problem consisting in minimizing a function given by the supremum of finitely many log-Laplace transforms of discrete nonnegative measures on a Euclidean space. Under a coerciveness assumption, we show that a $\varepsilon$-minimizer can be computed in a time that is polynomial in the input size and in $|\log\varepsilon|$. This is obtained by establishing bit-siz…
▽ More
We consider a geometric programming problem consisting in minimizing a function given by the supremum of finitely many log-Laplace transforms of discrete nonnegative measures on a Euclidean space. Under a coerciveness assumption, we show that a $\varepsilon$-minimizer can be computed in a time that is polynomial in the input size and in $|\log\varepsilon|$. This is obtained by establishing bit-size estimates on approximate minimizers and by applying the ellipsoid method. We also derive polynomial iteration complexity bounds for the interior point method applied to the same class of problems. We deduce that the spectral radius of a partially symmetric, weakly irreducible nonnegative tensor can be approximated within $\varepsilon$ error in poly-time. For strongly irreducible tensors, we also show that the logarithm of the positive eigenvector is poly-time computable. Our results also yield that the the maximum of a nonnegative homogeneous $d$-form in the unit ball with respect to $d$-Hölder norm can be approximated in poly-time. In particular, the spectral radius of uniform weighted hypergraphs and some known upper bounds for the clique number of uniform hypergraphs are poly-time computable.
△ Less
Submitted 19 March, 2024; v1 submitted 25 January, 2023;
originally announced January 2023.
-
Tensor optimal transport, distance between sets of measures and tensor scaling
Authors:
Shmuel Friedland
Abstract:
We study the optimal transport problem for $d>2$ discrete measures. This is a linear programming problem on $d$-tensors. It gives a way to compute a "distance" between two sets of discrete measures. We introduce an entropic regularization term, which gives rise to a scaling of tensors. We give a variation of the celebrated Sinkhorn scaling algorithm. We show that this algorithm can be viewed as a…
▽ More
We study the optimal transport problem for $d>2$ discrete measures. This is a linear programming problem on $d$-tensors. It gives a way to compute a "distance" between two sets of discrete measures. We introduce an entropic regularization term, which gives rise to a scaling of tensors. We give a variation of the celebrated Sinkhorn scaling algorithm. We show that this algorithm can be viewed as a partial minimization algorithm of a strictly convex function. Under appropriate conditions the rate of convergence is geometric and we estimate the rate. Our results are generalizations of known results for the classical case of two discrete measures.
△ Less
Submitted 24 July, 2021; v1 submitted 2 May, 2020;
originally announced May 2020.
-
Symmetric Grothendieck inequality
Authors:
Shmuel Friedland,
Lek-Heng Lim
Abstract:
We establish an analogue of the Grothendieck inequality where the rectangular matrix is replaced by a symmetric/Hermitian matrix and the bilinear form by a quadratic form. We call this the symmetric Grothendieck inequality; despite its name, it is a generalization -- the original Grothendieck inequality is a special case. While there are other proposals for such an inequality, ours differs in two…
▽ More
We establish an analogue of the Grothendieck inequality where the rectangular matrix is replaced by a symmetric/Hermitian matrix and the bilinear form by a quadratic form. We call this the symmetric Grothendieck inequality; despite its name, it is a generalization -- the original Grothendieck inequality is a special case. While there are other proposals for such an inequality, ours differs in two important ways: (i) we have no additional requirement like positive semidefiniteness; (ii) our symmetric Grothendieck constant is universal, i.e., independent of the matrix and its dimensions. A consequence of our symmetric Grothendieck inequality is a "conic Grothendieck inequality" for any family of cones of symmetric matrices: The original Grothendieck inequality is a special case; as is the Nesterov $π/2$-Theorem, which corresponds to the cones of positive semidefinite matrices; as well as the Goemans-Williamson inequality, which corresponds to the cones of Laplacians. For yet other cones, e.g., of diagonally dominant matrices, we get new Grothendieck-like inequalities. A slight extension leads to a unified framework that treats any Grothendieck-like inequality as an inequality between two norms within a family of "Grothendieck norms" restricted to a family of cones. This allows us to place on equal footing the Goemans-Williamson inequality, Nesterov $π/2$-Theorem, Ben-Tal-Nemirovski-Roos $4/π$-Theorem, generalized Grothendieck inequality, order-$p$ Grothendieck inequality, rank-constrained positive semidefinite Grothendieck inequality; and in turn allows us to simplify proofs, extend results from real to complex, obtain new bounds or establish sharpness of existing ones. The symmetric Grothendieck inequality may also be applied to obtain polynomial-time approximation bounds for NP-hard combinatorial, integer, and nonconvex optimization problems.
△ Less
Submitted 16 March, 2020;
originally announced March 2020.
-
Grothendieck constant is norm of Strassen matrix multiplication tensor
Authors:
**jie Zhang,
Shmuel Friedland,
Lek-Heng Lim
Abstract:
We show that two important quantities from two disparate areas of complexity theory --- Strassen's exponent of matrix multiplication $ω$ and Grothendieck's constant $K_G$ --- are intimately related. They are different measures of size for the same underlying object --- the matrix multiplication tensor, i.e., the $3$-tensor or bilinear operator…
▽ More
We show that two important quantities from two disparate areas of complexity theory --- Strassen's exponent of matrix multiplication $ω$ and Grothendieck's constant $K_G$ --- are intimately related. They are different measures of size for the same underlying object --- the matrix multiplication tensor, i.e., the $3$-tensor or bilinear operator $μ_{l,m,n} : \mathbb{F}^{l \times m} \times \mathbb{F}^{m \times n} \to \mathbb{F}^{l \times n}$, $(A,B) \mapsto AB$ defined by matrix-matrix product over $\mathbb{F} = \mathbb{R}$ or $\mathbb{C}$. It is well-known that Strassen's exponent of matrix multiplication is the greatest lower bound on (the log of) a tensor rank of $μ_{l,m,n}$. We will show that Grothendieck's constant is the least upper bound on a tensor norm of $μ_{l,m,n}$, taken over all $l, m, n \in \mathbb{N}$. Aside from relating the two celebrated quantities, this insight allows us to rewrite Grothendieck's inequality as a norm inequality \[ \lVertμ_{l,m,n}\rVert_{1,2,\infty} =\max_{X,Y,M\neq0}\frac{|\operatorname{tr}(XMY)|}{\lVert X\rVert_{1,2}\lVert Y\rVert_{2,\infty}\lVert M\rVert_{\infty,1}}\le K_G. \] We prove that Grothendieck's inequality is unique: If we generalize the $(1,2,\infty)$-norm to arbitrary $p,q, r \in [1, \infty]$, \[ \lVertμ_{l,m,n}\rVert_{p,q,r}=\max_{X,Y,M\neq0}\frac{|\operatorname{tr}(XMY)|}{\|X\|_{p,q}\|Y\|_{q,r}\|M\|_{r,p}}, \] then $(p,q,r )=(1,2,\infty)$ is, up to cyclic permutations, the only choice for which $\lVertμ_{l,m,n}\rVert_{p,q,r}$ is uniformly bounded by a constant independent of $l,m,n$.
△ Less
Submitted 5 June, 2018; v1 submitted 13 November, 2017;
originally announced November 2017.
-
The Computational Complexity of Duality
Authors:
Shmuel Friedland,
Lek-Heng Lim
Abstract:
We show that for any given norm ball or proper cone, weak membership in its dual ball or dual cone is polynomial-time reducible to weak membership in the given ball or cone. A consequence is that the weak membership or membership problem for a ball or cone is NP-hard if and only if the corresponding problem for the dual ball or cone is NP-hard. In a similar vein, we show that computation of the du…
▽ More
We show that for any given norm ball or proper cone, weak membership in its dual ball or dual cone is polynomial-time reducible to weak membership in the given ball or cone. A consequence is that the weak membership or membership problem for a ball or cone is NP-hard if and only if the corresponding problem for the dual ball or cone is NP-hard. In a similar vein, we show that computation of the dual norm of a given norm is polynomial-time reducible to computation of the given norm. This extends to convex functions satisfying a polynomial growth condition: for such a given function, computation of its Fenchel dual/conjugate is polynomial-time reducible to computation of the given function. Hence the computation of a norm or a convex function of polynomial-growth is NP-hard if and only if the computation of its dual norm or Fenchel dual is NP-hard. We discuss implications of these results on the weak membership problem for a symmetric convex body and its polar dual, the polynomial approximability of Mahler volume, and the weak membership problem for the epigraph of a convex function with polynomial growth and that of its Fenchel dual.
△ Less
Submitted 23 July, 2016; v1 submitted 27 January, 2016;
originally announced January 2016.
-
A Simple Spectral Algorithm for Recovering Planted Partitions
Authors:
Sam Cole,
Shmuel Friedland,
Lev Reyzin
Abstract:
In this paper, we consider the planted partition model, in which $n = ks$ vertices of a random graph are partitioned into $k$ "clusters," each of size $s$. Edges between vertices in the same cluster and different clusters are included with constant probability $p$ and $q$, respectively (where $0 \le q < p \le 1$). We give an efficient algorithm that, with high probability, recovers the clusters as…
▽ More
In this paper, we consider the planted partition model, in which $n = ks$ vertices of a random graph are partitioned into $k$ "clusters," each of size $s$. Edges between vertices in the same cluster and different clusters are included with constant probability $p$ and $q$, respectively (where $0 \le q < p \le 1$). We give an efficient algorithm that, with high probability, recovers the clusters as long as the cluster sizes are are least $Ω(\sqrt{n})$. Informally, our algorithm constructs the projection operator onto the dominant $k$-dimensional eigenspace of the graph's adjacency matrix and uses it to recover one cluster at a time. To our knowledge, our algorithm is the first purely spectral algorithm which runs in polynomial time and works even when $s = Θ(\sqrt n)$, though there have been several non-spectral algorithms which accomplish this. Our algorithm is also among the simplest of these spectral algorithms, and its proof of correctness illustrates the usefulness of the Cauchy integral formula in this domain.
△ Less
Submitted 25 August, 2017; v1 submitted 2 March, 2015;
originally announced March 2015.
-
Nuclear Norm of Higher-Order Tensors
Authors:
Shmuel Friedland,
Lek-Heng Lim
Abstract:
We establish several mathematical and computational properties of the nuclear norm for higher-order tensors. We show that like tensor rank, tensor nuclear norm is dependent on the choice of base field --- the value of the nuclear norm of a real 3-tensor depends on whether we regard it as a real 3-tensor or a complex 3-tensor with real entries. We show that every tensor has a nuclear norm attaining…
▽ More
We establish several mathematical and computational properties of the nuclear norm for higher-order tensors. We show that like tensor rank, tensor nuclear norm is dependent on the choice of base field --- the value of the nuclear norm of a real 3-tensor depends on whether we regard it as a real 3-tensor or a complex 3-tensor with real entries. We show that every tensor has a nuclear norm attaining decomposition and every symmetric tensor has a symmetric nuclear norm attaining decomposition. There is a corresponding notion of nuclear rank that, unlike tensor rank, is upper semicontinuous. We establish an analogue of Banach's theorem for tensor spectral norm and Comon's conjecture for tensor rank --- for a symmetric tensor, its symmetric nuclear norm always equals its nuclear norm. We show that computing tensor nuclear norm is NP-hard in several sense. Deciding weak membership in the nuclear norm unit ball of 3-tensors is NP-hard, as is finding an $\varepsilon$-approximation of nuclear norm for 3-tensors. In addition, the problem of computing spectral or nuclear norm of a 4-tensor is NP-hard, even if we restrict the 4-tensor to be bi-Hermitian, bisymmetric, positive semidefinite, nonnegative valued, or all of the above. We discuss some simple polynomial-time approximation bounds. As an aside, we show that the nuclear $(p,q)$-norm of a matrix is NP-hard in general but can be computed in polynomial-time if $p=1$, $q = 1$, or $p=q=2$, with closed-form expressions for the nuclear $(1,q)$- and $(p,1)$-norms.
△ Less
Submitted 17 May, 2016; v1 submitted 22 October, 2014;
originally announced October 2014.
-
Two algorithms for compressed sensing of sparse tensors
Authors:
Shmuel Friedland,
Qun Li,
Dan Schonfeld,
Edgar A. Bernal
Abstract:
Compressed sensing (CS) exploits the sparsity of a signal in order to integrate acquisition and compression. CS theory enables exact reconstruction of a sparse signal from relatively few linear measurements via a suitable nonlinear minimization process. Conventional CS theory relies on vectorial data representation, which results in good compression ratios at the expense of increased computational…
▽ More
Compressed sensing (CS) exploits the sparsity of a signal in order to integrate acquisition and compression. CS theory enables exact reconstruction of a sparse signal from relatively few linear measurements via a suitable nonlinear minimization process. Conventional CS theory relies on vectorial data representation, which results in good compression ratios at the expense of increased computational complexity. In applications involving color images, video sequences, and multi-sensor networks, the data is intrinsically of high-order, and thus more suitably represented in tensorial form. Standard applications of CS to higher-order data typically involve representation of the data as long vectors that are in turn measured using large sampling matrices, thus imposing a huge computational and memory burden. In this chapter, we introduce Generalized Tensor Compressed Sensing (GTCS)--a unified framework for compressed sensing of higher-order tensors which preserves the intrinsic structure of tensorial data with reduced computational complexity at reconstruction. We demonstrate that GTCS offers an efficient means for representation of multidimensional data by providing simultaneous acquisition and compression from all tensor modes. In addition, we propound two reconstruction procedures, a serial method (GTCS-S) and a parallelizable method (GTCS-P), both capable of recovering a tensor based on noiseless and noisy observations. We then compare the performance of the proposed methods with Kronecker compressed sensing (KCS) and multi-way compressed sensing (MWCS). We demonstrate experimentally that GTCS outperforms KCS and MWCS in terms of both reconstruction accuracy (within a range of compression ratios) and processing speed. The major disadvantage of our methods (and of MWCS as well), is that the achieved compression ratios may be worse than those offered by KCS.
△ Less
Submitted 3 September, 2014; v1 submitted 5 April, 2014;
originally announced April 2014.
-
Compressive Sensing of Sparse Tensors
Authors:
Shmuel Friedland,
Qun Li,
Dan Schonfeld
Abstract:
Compressive sensing (CS) has triggered enormous research activity since its first appearance. CS exploits the signal's sparsity or compressibility in a particular domain and integrates data compression and acquisition, thus allowing exact reconstruction through relatively few non-adaptive linear measurements. While conventional CS theory relies on data representation in the form of vectors, many d…
▽ More
Compressive sensing (CS) has triggered enormous research activity since its first appearance. CS exploits the signal's sparsity or compressibility in a particular domain and integrates data compression and acquisition, thus allowing exact reconstruction through relatively few non-adaptive linear measurements. While conventional CS theory relies on data representation in the form of vectors, many data types in various applications such as color imaging, video sequences, and multi-sensor networks, are intrinsically represented by higher-order tensors. Application of CS to higher-order data representation is typically performed by conversion of the data to very long vectors that must be measured using very large sampling matrices, thus imposing a huge computational and memory burden. In this paper, we propose Generalized Tensor Compressive Sensing (GTCS)--a unified framework for compressive sensing of higher-order tensors which preserves the intrinsic structure of tensor data with reduced computational complexity at reconstruction. GTCS offers an efficient means for representation of multidimensional data by providing simultaneous acquisition and compression from all tensor modes. In addition, we propound two reconstruction procedures, a serial method (GTCS-S) and a parallelizable method (GTCS-P). We then compare the performance of the proposed method with Kronecker compressive sensing (KCS) and multi way compressive sensing (MWCS). We demonstrate experimentally that GTCS outperforms KCS and MWCS in terms of both reconstruction accuracy (within a range of compression ratios) and processing speed. The major disadvantage of our methods (and of MWCS as well), is that the compression ratios may be worse than that offered by KCS.
△ Less
Submitted 3 September, 2014; v1 submitted 24 May, 2013;
originally announced May 2013.
-
Submodular spectral functions of principal submatrices of a hermitian matrix, extensions and applications
Authors:
S. Friedland,
S. Gaubert
Abstract:
We extend the multiplicative submodularity of the principal determinants of a nonnegative definite hermitian matrix to other spectral functions. We show that if $f$ is the primitive of a function that is operator monotone on an interval containing the spectrum of a hermitian matrix $A$, then the function $I\mapsto {\rm tr} f(A[I])$ is supermodular, meaning that…
▽ More
We extend the multiplicative submodularity of the principal determinants of a nonnegative definite hermitian matrix to other spectral functions. We show that if $f$ is the primitive of a function that is operator monotone on an interval containing the spectrum of a hermitian matrix $A$, then the function $I\mapsto {\rm tr} f(A[I])$ is supermodular, meaning that ${\rm tr} f(A[I])+{\rm tr} f(A[J])\leq {\rm tr} f(A[I\cup J])+{\rm tr} f(A[I\cap J])$, where $A[I]$ denotes the $I\times I$ principal submatrix of $A$. We discuss extensions to self-adjoint operators on infinite dimensional Hilbert space and to $M$-matrices. We discuss an application to CUR approximation of nonnegative hermitian matrices.
△ Less
Submitted 19 June, 2012; v1 submitted 20 July, 2010;
originally announced July 2010.
-
Graph isomorphism and volumes of convex bodies
Authors:
Shmuel Friedland
Abstract:
We show that a nontrivial graph isomorphism problem of two undirected graphs, and more generally, the permutation similarity of two given $n\times n$ matrices, is equivalent to equalities of volumes of the induced three convex bounded polytopes intersected with a given sequence of balls, centered at the origin with radii $t_i\in (0,\sqrt{n-1})$, where $\{t_i\}$ is an increasing sequence convergi…
▽ More
We show that a nontrivial graph isomorphism problem of two undirected graphs, and more generally, the permutation similarity of two given $n\times n$ matrices, is equivalent to equalities of volumes of the induced three convex bounded polytopes intersected with a given sequence of balls, centered at the origin with radii $t_i\in (0,\sqrt{n-1})$, where $\{t_i\}$ is an increasing sequence converging to $\sqrt{n-1}$. These polytopes are characterized by $n^2$ inequalities in at most $n^2$ variables. The existence of fpras for computing volumes of convex bodies gives rise to a semi-frpas of order $O^*(n^{14})$ at most to find if given two undirected graphs are isomorphic.
△ Less
Submitted 9 November, 2009;
originally announced November 2009.
-
On the graph isomorphism problem
Authors:
Shmuel Friedland
Abstract:
We relate the graph isomorphism problem to the solvability of certain systems of linear equations with nonnegative variables. This version replaces the two previous versions of this paper.
We relate the graph isomorphism problem to the solvability of certain systems of linear equations with nonnegative variables. This version replaces the two previous versions of this paper.
△ Less
Submitted 10 January, 2008; v1 submitted 2 January, 2008;
originally announced January 2008.
-
FPRAS for computing a lower bound for weighted matching polynomial of graphs
Authors:
Shmuel Friedland
Abstract:
We give a fully polynomial randomized approximation scheme to compute a lower bound for the matching polynomial of any weighted graph at a positive argument. For the matching polynomial of complete bipartite graphs with bounded weights these lower bounds are asymptotically optimal.
We give a fully polynomial randomized approximation scheme to compute a lower bound for the matching polynomial of any weighted graph at a positive argument. For the matching polynomial of complete bipartite graphs with bounded weights these lower bounds are asymptotically optimal.
△ Less
Submitted 12 April, 2007; v1 submitted 6 March, 2007;
originally announced March 2007.
-
A polynomial-time approximation algorithm for the number of k-matchings in bipartite graphs
Authors:
Shmuel Friedland,
Daniel Levy
Abstract:
We show that the number of $k$-matching in a given undirected graph
$G$ is equal to the number of perfect matching of the corresponding graph
$G_k$ on an even number of vertices divided by a suitable factor.
If $G$ is bipartite then one can construct a bipartite $G_k$.
For bipartite graphs this result implies that the number of $k$-matching has a polynomial-time approximation algorithm. The…
▽ More
We show that the number of $k$-matching in a given undirected graph
$G$ is equal to the number of perfect matching of the corresponding graph
$G_k$ on an even number of vertices divided by a suitable factor.
If $G$ is bipartite then one can construct a bipartite $G_k$.
For bipartite graphs this result implies that the number of $k$-matching has a polynomial-time approximation algorithm. The above results are extended to permanents and hafnians of corresponding matrices.
△ Less
Submitted 28 July, 2006;
originally announced July 2006.
-
Fast Monte-Carlo Low Rank Approximations for Matrices
Authors:
Shmuel Friedland,
Mostafa Kaveh,
Amir Niknejad,
Hossein Zare
Abstract:
In many applications, it is of interest to approximate data, given by mxn matrix A, by a matrix B of at most rank k, which is much smaller than m and n. The best approximation is given by singular value decomposition, which is too time consuming for very large m and n. We present here a Monte Carlo algorithm for iteratively computing a k-rank approximation to the data consisting of mxn matrix A.…
▽ More
In many applications, it is of interest to approximate data, given by mxn matrix A, by a matrix B of at most rank k, which is much smaller than m and n. The best approximation is given by singular value decomposition, which is too time consuming for very large m and n. We present here a Monte Carlo algorithm for iteratively computing a k-rank approximation to the data consisting of mxn matrix A. Each iteration involves the reading of O(k) of columns or rows of A. The complexity of our algorithm is O(kmn). Our algorithm, distinguished from other known algorithms, guarantees that each iteration is a better k-rank approximation than the previous iteration. We believe that this algorithm will have many applications in data mining, data storage and data analysis.
△ Less
Submitted 26 October, 2005;
originally announced October 2005.
