-
Approximating maps into manifolds with lower curvature bounds
Authors:
Simon Jacobsson,
Raf Vandebril,
Joeri van der Veken,
Nick Vannieuwenhoven
Abstract:
Many interesting functions arising in applications map into Riemannian manifolds. We present an algorithm, using the manifold exponential and logarithm, for approximating such functions. Our approach extends approximation techniques for functions into linear spaces so that we can upper bound the forward error in terms of a lower bound on the manifold's sectional curvature. Furthermore, when the se…
▽ More
Many interesting functions arising in applications map into Riemannian manifolds. We present an algorithm, using the manifold exponential and logarithm, for approximating such functions. Our approach extends approximation techniques for functions into linear spaces so that we can upper bound the forward error in terms of a lower bound on the manifold's sectional curvature. Furthermore, when the sectional curvature of a manifold is nonnegative, such as for compact Lie groups, the error is guaranteed to not be worse than in the linear case. We implement the algorithm in a Julia package and apply it to two example problems from Krylov subspaces and dynamic low-rank approximation, respectively. For these examples, the maps are confirmed to be well approximated by our algorithm.
△ Less
Submitted 25 March, 2024;
originally announced March 2024.
-
Hadamard-Hitchcock decompositions: identifiability and computation
Authors:
Alessandro Oneto,
Nick Vannieuwenhoven
Abstract:
A Hadamard-Hitchcock decomposition of a multidimensional array is a decomposition that expresses the latter as a Hadamard product of several tensor rank decompositions. Such decompositions can encode probability distributions that arise from statistical graphical models associated to complete bipartite graphs with one layer of observed random variables and one layer of hidden ones, usually called…
▽ More
A Hadamard-Hitchcock decomposition of a multidimensional array is a decomposition that expresses the latter as a Hadamard product of several tensor rank decompositions. Such decompositions can encode probability distributions that arise from statistical graphical models associated to complete bipartite graphs with one layer of observed random variables and one layer of hidden ones, usually called restricted Boltzmann machines. We establish generic identifiability of Hadamard-Hitchcock decompositions by exploiting the reshaped Kruskal criterion for tensor rank decompositions. A flexible algorithm leveraging existing decomposition algorithms for tensor rank decomposition is introduced for computing a Hadamard-Hitchcock decomposition. Numerical experiments illustrate its computational performance and numerical accuracy.
△ Less
Submitted 12 August, 2023;
originally announced August 2023.
-
What part of a numerical problem is ill-conditioned?
Authors:
Nick Dewaele,
Nick Vannieuwenhoven
Abstract:
Many numerical problems with input $x$ and output $y$ can be formulated as an system of equations $F(x, y) = 0$ where the goal is to solve for $y$. The condition number measures the change of $y$ for small perturbations to $x$. From this numerical problem, one can derive a (typically underdetermined) subproblem by omitting any number of constraints from $F$. We propose a condition number for under…
▽ More
Many numerical problems with input $x$ and output $y$ can be formulated as an system of equations $F(x, y) = 0$ where the goal is to solve for $y$. The condition number measures the change of $y$ for small perturbations to $x$. From this numerical problem, one can derive a (typically underdetermined) subproblem by omitting any number of constraints from $F$. We propose a condition number for underdetermined systems that relates the condition number of a numerical problem to those of its subproblems. We illustrate the use of our technique by computing the condition of two problems that do not have a finite condition number in the classic sense: any two-factor matrix decompositions and Tucker decompositions.
△ Less
Submitted 19 May, 2023;
originally announced May 2023.
-
The condition number of singular subspaces, revisited
Authors:
Nick Vannieuwenhoven
Abstract:
I revisit the condition number of computing left and right singular subspaces from [J.-G. Sun, Perturbation analysis of singular subspaces and deflating subspaces, Numer. Math. 73(2), pp. 235--263, 1996]. For real and complex matrices, I present an alternative computation of this condition number in the Euclidean distance on the input space of matrices and the chordal, Grassmann, and Procrustes di…
▽ More
I revisit the condition number of computing left and right singular subspaces from [J.-G. Sun, Perturbation analysis of singular subspaces and deflating subspaces, Numer. Math. 73(2), pp. 235--263, 1996]. For real and complex matrices, I present an alternative computation of this condition number in the Euclidean distance on the input space of matrices and the chordal, Grassmann, and Procrustes distances on the output Grassmannian manifold of linear subspaces. Up to a small factor, this condition number equals the inverse minimum singular value gap between the singular values corresponding to the selected singular subspace and those not selected.
△ Less
Submitted 23 August, 2023; v1 submitted 17 August, 2022;
originally announced August 2022.
-
Group-invariant tensor train networks for supervised learning
Authors:
Brent Sprangers,
Nick Vannieuwenhoven
Abstract:
Invariance has recently proven to be a powerful inductive bias in machine learning models. One such class of predictive or generative models are tensor networks. We introduce a new numerical algorithm to construct a basis of tensors that are invariant under the action of normal matrix representations of an arbitrary discrete group. This method can be up to several orders of magnitude faster than p…
▽ More
Invariance has recently proven to be a powerful inductive bias in machine learning models. One such class of predictive or generative models are tensor networks. We introduce a new numerical algorithm to construct a basis of tensors that are invariant under the action of normal matrix representations of an arbitrary discrete group. This method can be up to several orders of magnitude faster than previous approaches. The group-invariant tensors are then combined into a group-invariant tensor train network, which can be used as a supervised machine learning model. We applied this model to a protein binding classification problem, taking into account problem-specific invariances, and obtained prediction accuracy in line with state-of-the-art deep learning approaches.
△ Less
Submitted 27 September, 2023; v1 submitted 30 June, 2022;
originally announced June 2022.
-
Almost all subgeneric third-order Chow decompositions are identifiable
Authors:
Douglas A. Torrance,
Nick Vannieuwenhoven
Abstract:
For real and complex homogeneous cubic polyomials in $n+1$ variables, we prove that the Chow variety of products of linear forms is generically complex identifiable for all ranks up to the generic rank minus two. By integrating fundamental results of [Oeding, Hyperdeterminants of polynomials, Adv. Math., 2012], [Casarotti and Mella, From non defectivity to identifiability, J. Eur. Math. Soc., 2021…
▽ More
For real and complex homogeneous cubic polyomials in $n+1$ variables, we prove that the Chow variety of products of linear forms is generically complex identifiable for all ranks up to the generic rank minus two. By integrating fundamental results of [Oeding, Hyperdeterminants of polynomials, Adv. Math., 2012], [Casarotti and Mella, From non defectivity to identifiability, J. Eur. Math. Soc., 2021], and [Torrance and Vannieuwenhoven, All secant varieties of the Chow variety are nondefective for cubics and quaternary forms, Trans. Amer. Math. Soc., 2021] the proof is reduced to only those cases in up to $103$ variables. These remaining cases are proved using the Hessian criterion for tangential weak defectivity from [Chiantini, Ottaviani, and Vannieuwenhoven, An algorithm for generic and low-rank specific identifiability of complex tensors, SIAM J. Matrix Anal. Appl., 2014]. We also establish that the smooth loci of the real and complex Chow varieties are immersed minimal submanifolds in their usual ambient spaces.
△ Less
Submitted 13 December, 2021;
originally announced December 2021.
-
Three decompositions of symmetric tensors have similar condition numbers
Authors:
Nick Dewaele,
Paul Breiding,
Nick Vannieuwenhoven
Abstract:
We relate the condition numbers of computing three decompositions of symmetric tensors: the canonical polyadic decomposition, the Waring decomposition, and a Tucker-compressed Waring decomposition. Based on this relation we can speed up the computation of these condition numbers by orders of magnitude
We relate the condition numbers of computing three decompositions of symmetric tensors: the canonical polyadic decomposition, the Waring decomposition, and a Tucker-compressed Waring decomposition. Based on this relation we can speed up the computation of these condition numbers by orders of magnitude
△ Less
Submitted 8 October, 2021;
originally announced October 2021.
-
When can forward stable algorithms be composed stably?
Authors:
Carlos Beltrán,
Vanni Noferini,
Nick Vannieuwenhoven
Abstract:
We state some widely satisfied hypotheses, depending only on two functions $g$ and $h$, under which the composition of a stable algorithm for $g$ and a stable algorithm for $h$ is a stable algorithm for the composition $g \circ h$.
We state some widely satisfied hypotheses, depending only on two functions $g$ and $h$, under which the composition of a stable algorithm for $g$ and a stable algorithm for $h$ is a stable algorithm for the composition $g \circ h$.
△ Less
Submitted 25 May, 2022; v1 submitted 22 September, 2021;
originally announced September 2021.
-
Algebraic compressed sensing
Authors:
Paul Breiding,
Fulvio Gesmundo,
Mateusz Michałek,
Nick Vannieuwenhoven
Abstract:
We introduce the broad subclass of algebraic compressed sensing problems, where structured signals are modeled either explicitly or implicitly via polynomials. This includes, for instance, low-rank matrix and tensor recovery. We employ powerful techniques from algebraic geometry to study well-posedness of sufficiently general compressed sensing problems, including existence, local recoverability,…
▽ More
We introduce the broad subclass of algebraic compressed sensing problems, where structured signals are modeled either explicitly or implicitly via polynomials. This includes, for instance, low-rank matrix and tensor recovery. We employ powerful techniques from algebraic geometry to study well-posedness of sufficiently general compressed sensing problems, including existence, local recoverability, global uniqueness, and local smoothness. Our main results are summarized in thirteen questions and answers in algebraic compressed sensing. Most of our answers concerning the minimum number of required measurements for existence, recoverability, and uniqueness of algebraic compressed sensing problems are optimal and depend only on the dimension of the model.
△ Less
Submitted 24 June, 2024; v1 submitted 30 August, 2021;
originally announced August 2021.
-
Tensor completion using geodesics on Segre manifolds
Authors:
Lars Swijsen,
Joeri Van der Veken,
Nick Vannieuwenhoven
Abstract:
We propose a Riemannian conjugate gradient (CG) optimization method for finding low rank approximations of incomplete tensors. Our main contribution consists of an explicit expression of the geodesics on the Segre manifold. These are exploited in our algorithm to perform the retractions. We apply our method to movie rating predictions in a recommender system for the MovieLens dataset, and identifi…
▽ More
We propose a Riemannian conjugate gradient (CG) optimization method for finding low rank approximations of incomplete tensors. Our main contribution consists of an explicit expression of the geodesics on the Segre manifold. These are exploited in our algorithm to perform the retractions. We apply our method to movie rating predictions in a recommender system for the MovieLens dataset, and identification of pure fluorophores via fluorescent spectroscopy with missing data. In this last application, we recover the tensor decomposition from less than $10\%$ of the data.
△ Less
Submitted 2 August, 2021;
originally announced August 2021.
-
The condition number of many tensor decompositions is invariant under Tucker compression
Authors:
Nick Dewaele,
Paul Breiding,
Nick Vannieuwenhoven
Abstract:
We characterise the sensitivity of several additive tensor decompositions with respect to perturbations of the original tensor. These decompositions include canonical polyadic decompositions, block term decompositions, and sums of tree tensor networks. Our main result shows that the condition number of all these decompositions is invariant under Tucker compression. This result can dramatically spe…
▽ More
We characterise the sensitivity of several additive tensor decompositions with respect to perturbations of the original tensor. These decompositions include canonical polyadic decompositions, block term decompositions, and sums of tree tensor networks. Our main result shows that the condition number of all these decompositions is invariant under Tucker compression. This result can dramatically speed up the computation of the condition number in practical applications. We give the example of an $265\times 371\times 7$ tensor of rank $3$ from a food science application whose condition number was computed in $6.9$ milliseconds by exploiting our new theorem, representing a speedup of four orders of magnitude over the previous state of the art.
△ Less
Submitted 24 June, 2021;
originally announced June 2021.
-
A Normal Form Algorithm for Tensor Rank Decomposition
Authors:
Simon Telen,
Nick Vannieuwenhoven
Abstract:
We propose a new numerical algorithm for computing the tensor rank decomposition or canonical polyadic decomposition of higher-order tensors subject to a rank and genericity constraint. Reformulating this computational problem as a system of polynomial equations allows us to leverage recent numerical linear algebra tools from computational algebraic geometry. We characterize the complexity of our…
▽ More
We propose a new numerical algorithm for computing the tensor rank decomposition or canonical polyadic decomposition of higher-order tensors subject to a rank and genericity constraint. Reformulating this computational problem as a system of polynomial equations allows us to leverage recent numerical linear algebra tools from computational algebraic geometry. We characterize the complexity of our algorithm in terms of an algebraic property of this polynomial system -- the multigraded regularity. We prove effective bounds for many tensor formats and ranks, which are of independent interest for overconstrained polynomial system solving. Moreover, we conjecture a general formula for the multigraded regularity, yielding a (parameterized) polynomial time complexity for the tensor rank decomposition problem in the considered setting. Our numerical experiments show that our algorithm can outperform state-of-the-art numerical algorithms by an order of magnitude in terms of accuracy, computation time, and memory consumption.
△ Less
Submitted 14 November, 2022; v1 submitted 12 March, 2021;
originally announced March 2021.
-
Sensitivity of low-rank matrix recovery
Authors:
Paul Breiding,
Nick Vannieuwenhoven
Abstract:
We characterize the first-order sensitivity of approximately recovering a low-rank matrix from linear measurements, a standard problem in compressed sensing. A special case covered by our analysis is approximating an incomplete matrix by a low-rank matrix. We give an algorithm for computing the associated condition number and demonstrate experimentally how the number of linear measurements affects…
▽ More
We characterize the first-order sensitivity of approximately recovering a low-rank matrix from linear measurements, a standard problem in compressed sensing. A special case covered by our analysis is approximating an incomplete matrix by a low-rank matrix. We give an algorithm for computing the associated condition number and demonstrate experimentally how the number of linear measurements affects it.
In addition, we study the condition number of the rank-r matrix approximation problem. It measures in the Frobenius norm by how much an infinitesimal perturbation to an arbitrary input matrix is amplified in the movement of its best rank-r approximation. We give an explicit formula for the condition number, which shows that it does depend on the relative singular value gap between the rth and (r+1)th singular values of the input matrix.
△ Less
Submitted 5 October, 2021; v1 submitted 28 February, 2021;
originally announced March 2021.
-
All secant varieties of the Chow variety are nondefective for cubics and quaternary forms
Authors:
Douglas A. Torrance,
Nick Vannieuwenhoven
Abstract:
The Chow rank of a form is the length of its smallest decomposition into a sum of products of linear forms. For a generic form, this corresponds to finding the smallest secant variety of the Chow variety which fills the ambient space. We determine the Chow rank of generic cubics and quaternary forms by proving nondefectivity of all involved secant varieties. The main new ingredient in our proof is…
▽ More
The Chow rank of a form is the length of its smallest decomposition into a sum of products of linear forms. For a generic form, this corresponds to finding the smallest secant variety of the Chow variety which fills the ambient space. We determine the Chow rank of generic cubics and quaternary forms by proving nondefectivity of all involved secant varieties. The main new ingredient in our proof is the generalization of a technique by [Brambilla and Ottaviani, On the Alexander--Hirschowitz theorem, J. Pure Appl. Algebra, 2008] that consists of employing Terracini's lemma and Newton's backward difference formula to compute the dimensions of secant varieties of arbitrary projective varieties. Via this inductive construction, the proof of nondefectivity ultimately reduces to proving a number of base cases. These are settled via a computer-assisted proof because of the large dimensions of the spaces involved. The largest base case required in our proof consisted of computing the dimension of a vector space constructed from the $400$th secant variety of a degree-$82$ Chow variety embedded in $\mathbb{P}^{98769}$.
△ Less
Submitted 22 September, 2020; v1 submitted 25 May, 2020;
originally announced May 2020.
-
The condition number of Riemannian approximation problems
Authors:
Paul Breiding,
Nick Vannieuwenhoven
Abstract:
We consider the local sensitivity of least-squares formulations of inverse problems. The sets of inputs and outputs of these problems are assumed to have the structures of Riemannian manifolds. The problems we consider include the approximation problem of finding the nearest point on a Riemannian embedded submanifold from a given point in the ambient space. We characterize the first-order sensitiv…
▽ More
We consider the local sensitivity of least-squares formulations of inverse problems. The sets of inputs and outputs of these problems are assumed to have the structures of Riemannian manifolds. The problems we consider include the approximation problem of finding the nearest point on a Riemannian embedded submanifold from a given point in the ambient space. We characterize the first-order sensitivity, i.e., condition number, of local minimizers and critical points to arbitrary perturbations of the input of the least-squares problem. This condition number involves the Weingarten map of the input manifold, which measures the amount by which the input manifold curves in its ambient space. We validate our main results through experiments with the $n$-camera triangulation problem in computer vision.
△ Less
Submitted 7 December, 2020; v1 submitted 26 September, 2019;
originally announced September 2019.
-
The average condition number of most tensor rank decomposition problems is infinite
Authors:
Carlos Beltrán,
Paul Breiding,
Nick Vannieuwenhoven
Abstract:
The tensor rank decomposition, or canonical polyadic decomposition, is the decomposition of a tensor into a sum of rank-1 tensors. The condition number of the tensor rank decomposition measures the sensitivity of the rank-1 summands with respect to structured perturbations. Those are perturbations preserving the rank of the tensor that is decomposed. On the other hand, the angular condition number…
▽ More
The tensor rank decomposition, or canonical polyadic decomposition, is the decomposition of a tensor into a sum of rank-1 tensors. The condition number of the tensor rank decomposition measures the sensitivity of the rank-1 summands with respect to structured perturbations. Those are perturbations preserving the rank of the tensor that is decomposed. On the other hand, the angular condition number measures the perturbations of the rank-1 summands up to scaling.
We show for random rank-2 tensors that the expected value of the condition number is infinite for a wide range of choices of the density. Under a mild additional assumption, we show that the same is true for most higher ranks $r\geq 3$ as well. In fact, as the dimensions of the tensor tend to infinity, asymptotically all ranks are covered by our analysis. On the contrary, we show that rank-2 tensors have finite expected angular condition number. Based on numerical experiments, we conjecture that this could also be true for higher ranks.
Our results underline the high computational complexity of computing tensor rank decompositions. We discuss consequences of our results for algorithm design and for testing algorithms computing tensor rank decompositions.
△ Less
Submitted 20 September, 2022; v1 submitted 13 March, 2019;
originally announced March 2019.
-
Pencil-based algorithms for tensor rank decomposition are not stable
Authors:
Carlos Beltrán,
Paul Breiding,
Nick Vannieuwenhoven
Abstract:
We prove the existence of an open set of $n_1\times n_2 \times n_3$ tensors of rank $r$ on which a popular and efficient class of algorithms for computing tensor rank decompositions based on a reduction to a linear matrix pencil, typically followed by a generalized eigendecomposition, is arbitrarily numerically forward unstable. Our analysis shows that this problem is caused by the fact that the c…
▽ More
We prove the existence of an open set of $n_1\times n_2 \times n_3$ tensors of rank $r$ on which a popular and efficient class of algorithms for computing tensor rank decompositions based on a reduction to a linear matrix pencil, typically followed by a generalized eigendecomposition, is arbitrarily numerically forward unstable. Our analysis shows that this problem is caused by the fact that the condition number of the tensor rank decomposition can be much larger for $n_1 \times n_2 \times 2$ tensors than for the $n_1\times n_2 \times n_3$ input tensor. Moreover, we present a lower bound for the limiting distribution of the condition number of random tensor rank decompositions of third-order tensors. The numerical experiments illustrate that for random tensor rank decompositions one should anticipate a loss of precision of a few digits.
△ Less
Submitted 11 July, 2018;
originally announced July 2018.
-
On the average condition number of tensor rank decompositions
Authors:
Paul Breiding,
Nick Vannieuwenhoven
Abstract:
We compute the expected value of powers of the geometric condition number of random tensor rank decompositions. It is shown in particular that the expected value of the condition number of $n_1\times n_2 \times 2$ tensors with a random rank-$r$ decomposition, given by factor matrices with independent and identically distributed standard normal entries, is infinite. This entails that it is expected…
▽ More
We compute the expected value of powers of the geometric condition number of random tensor rank decompositions. It is shown in particular that the expected value of the condition number of $n_1\times n_2 \times 2$ tensors with a random rank-$r$ decomposition, given by factor matrices with independent and identically distributed standard normal entries, is infinite. This entails that it is expected and probable that such a rank-$r$ decomposition is sensitive to perturbations of the tensor. Moreover, it provides concrete further evidence that tensor decomposition can be a challenging problem, also from the numerical point of view. On the other hand, we provide strong theoretical and empirical evidence that tensors of size $n_1~\times~n_2~\times~n_3$ with all $n_1,n_2,n_3 \ge 3$ have a finite average condition number. This suggests there exists a gap in the expected sensitivity of tensors between those of format $n_1\times n_2 \times 2$ and other order-3 tensors. For establishing these results, we show that a natural weighted distance from a tensor rank decomposition to the locus of ill-posed decompositions with an infinite geometric condition number is bounded from below by the inverse of this condition number. That is, we prove one inequality towards a so-called condition number theorem for the tensor rank decomposition.
△ Less
Submitted 2 January, 2019; v1 submitted 5 January, 2018;
originally announced January 2018.
-
Identifiability beyond Kruskal's bound for symmetric tensors of degree 4
Authors:
Elena Angelini,
Luca Chiantini,
Nick Vannieuwenhoven
Abstract:
We show how methods of algebraic geometry can produce criteria for the identifiability of specific tensors that reach beyond the range of applicability of the celebrated Kruskal criterion. More specifically, we deal with the symmetric identifiability of symmetric tensors in Sym$^4(\mathbb{C}^{n+1})$, i.e., quartic hypersurfaces in a projective space $\mathbb{P}^n$, that have a decomposition in 2n+…
▽ More
We show how methods of algebraic geometry can produce criteria for the identifiability of specific tensors that reach beyond the range of applicability of the celebrated Kruskal criterion. More specifically, we deal with the symmetric identifiability of symmetric tensors in Sym$^4(\mathbb{C}^{n+1})$, i.e., quartic hypersurfaces in a projective space $\mathbb{P}^n$, that have a decomposition in 2n+1 summands of rank 1. This is the first case where the reshaped Kruskal criterion no longer applies. We present an effective algorithm, based on efficient linear algebra computations, that checks if the given decomposition is minimal and unique. The criterion is based on the application of advanced geometric tools, like Castelnuovo's lemma for the existence of rational normal curves passing through a finite set of points, and the Cayley-Bacharach condition on the postulation of finite sets. In order to apply these tools to our situation, we prove a reformulation of these results, hereby extending classical results such as Castelnuovo's lemma and the analysis of Geramita, Kreuzer, and Robbiano, "Cayley-Bacharach schemes and their canonical modules", Trans. Amer. Math. Soc. 339:443-452, 1993.
△ Less
Submitted 12 December, 2017;
originally announced December 2017.
-
A Riemannian Trust Region Method for the Canonical Tensor Rank Approximation Problem
Authors:
Paul Breiding,
Nick Vannieuwenhoven
Abstract:
The canonical tensor rank approximation problem (TAP) consists of approximating a real-valued tensor by one of low canonical rank, which is a challenging non-linear, non-convex, constrained optimization problem, where the constraint set forms a non-smooth semi-algebraic set. We introduce a Riemannian Gauss-Newton method with trust region for solving small-scale, dense TAPs. The novelty of our appr…
▽ More
The canonical tensor rank approximation problem (TAP) consists of approximating a real-valued tensor by one of low canonical rank, which is a challenging non-linear, non-convex, constrained optimization problem, where the constraint set forms a non-smooth semi-algebraic set. We introduce a Riemannian Gauss-Newton method with trust region for solving small-scale, dense TAPs. The novelty of our approach is threefold. First, we parametrize the constraint set as the Cartesian product of Segre manifolds, hereby formulating the TAP as a Riemannian optimization problem, and we argue why this parametrization is among the theoretically best possible. Second, an original ST-HOSVD-based retraction operator is proposed. Third, we introduce a hot restart mechanism that efficiently detects when the optimization process is tending to an ill-conditioned tensor rank decomposition and which often yields a quick escape path from such spurious decompositions. Numerical experiments show improvements of up to three orders of magnitude in terms of the expected time to compute a successful solution over existing state-of-the-art methods.
△ Less
Submitted 11 July, 2018; v1 submitted 31 August, 2017;
originally announced September 2017.
-
Convergence analysis of Riemannian Gauss-Newton methods and its connection with the geometric condition number
Authors:
Paul Breiding,
Nick Vannieuwenhoven
Abstract:
We obtain estimates of the multiplicative constants appearing in local convergence results of the Riemannian Gauss-Newton method for least squares problems on manifolds and relate them to the geometric condition number of [P. Bürgisser and F. Cucker, Condition: The Geometry of Numerical Algorithms, 2013].
We obtain estimates of the multiplicative constants appearing in local convergence results of the Riemannian Gauss-Newton method for least squares problems on manifolds and relate them to the geometric condition number of [P. Bürgisser and F. Cucker, Condition: The Geometry of Numerical Algorithms, 2013].
△ Less
Submitted 16 October, 2017; v1 submitted 8 August, 2017;
originally announced August 2017.
-
The condition number of join decompositions
Authors:
Paul Breiding,
Nick Vannieuwenhoven
Abstract:
The join set of a finite collection of smooth embedded submanifolds of a mutual vector space is defined as their Minkowski sum. Join decompositions generalize some ubiquitous decompositions in multilinear algebra, namely tensor rank, Waring, partially symmetric rank and block term decompositions. This paper examines the numerical sensitivity of join decompositions to perturbations; specifically, w…
▽ More
The join set of a finite collection of smooth embedded submanifolds of a mutual vector space is defined as their Minkowski sum. Join decompositions generalize some ubiquitous decompositions in multilinear algebra, namely tensor rank, Waring, partially symmetric rank and block term decompositions. This paper examines the numerical sensitivity of join decompositions to perturbations; specifically, we consider the condition number for general join decompositions. It is characterized as a distance to a set of ill-posed points in a supplementary product of Grassmannians. We prove that this condition number can be computed efficiently as the smallest singular value of an auxiliary matrix. For some special join sets, we characterized the behavior of sequences in the join set converging to the latter's boundary points. Finally, we specialize our discussion to the tensor rank and Waring decompositions and provide several numerical experiments confirming the key results.
△ Less
Submitted 14 May, 2018; v1 submitted 24 November, 2016;
originally announced November 2016.
-
Effective criteria for specific identifiability of tensors and forms
Authors:
Luca Chiantini,
Giorgio Ottaviani,
Nick Vannieuwenhoven
Abstract:
In applications where the tensor rank decomposition arises, one often relies on its identifiability properties for interpreting the individual rank-$1$ terms appearing in the decomposition. Several criteria for identifiability have been proposed in the literature, however few results exist on how frequently they are satisfied. We propose to call a criterion effective if it is satisfied on a dense,…
▽ More
In applications where the tensor rank decomposition arises, one often relies on its identifiability properties for interpreting the individual rank-$1$ terms appearing in the decomposition. Several criteria for identifiability have been proposed in the literature, however few results exist on how frequently they are satisfied. We propose to call a criterion effective if it is satisfied on a dense, open subset of the smallest semi-algebraic set enclosing the set of rank-$r$ tensors. We analyze the effectiveness of Kruskal's criterion when it is combined with resha**. It is proved that this criterion is effective for both real and complex tensors in its entire range of applicability, which is usually much smaller than the smallest typical rank. Our proof explains when resha**-based algorithms for computing tensor rank decompositions may be expected to recover the decomposition. Specializing the analysis to symmetric tensors or forms reveals that the reshaped Kruskal criterion may even be effective up to the smallest typical rank for some third, fourth and sixth order symmetric tensors of small dimension as well as for binary forms of degree at least three. We extended this result to $4 \times 4 \times 4 \times 4$ symmetric tensors by analyzing the Hilbert function, resulting in a criterion for symmetric identifiability that is effective up to symmetric rank $8$, which is optimal.
△ Less
Submitted 1 September, 2016;
originally announced September 2016.
-
A condition number for the tensor rank decomposition
Authors:
Nick Vannieuwenhoven
Abstract:
The tensor rank decomposition problem consists of recovering the unique set of parameters representing a robustly identifiable low-rank tensor when the coordinate representation of the tensor is presented as input. A condition number for this problem measuring the sensitivity of the parameters to an infinitesimal change to the tensor is introduced and analyzed. It is demonstrated that the absolute…
▽ More
The tensor rank decomposition problem consists of recovering the unique set of parameters representing a robustly identifiable low-rank tensor when the coordinate representation of the tensor is presented as input. A condition number for this problem measuring the sensitivity of the parameters to an infinitesimal change to the tensor is introduced and analyzed. It is demonstrated that the absolute condition number coincides with the inverse of the least singular value of Terracini's matrix. Several basic properties of this condition number are investigated.
△ Less
Submitted 31 March, 2016;
originally announced April 2016.
-
Most secant varieties of tangential varieties to Veronese varieties are nondefective
Authors:
Hirotachi Abo,
Nick Vannieuwenhoven
Abstract:
We prove a conjecture stated by Catalisano, Geramita, and Gimigliano in 2002, which claims that the secant varieties of tangential varieties to the $d$th Veronese embedding of the projective $n$-space $\mathbb{P}^n$ have the expected dimension, modulo a few well-known exceptions. As Bernardi, Catalisano, Gimigliano, and Idá demonstrated that the proof of this conjecture may be reduced to the case…
▽ More
We prove a conjecture stated by Catalisano, Geramita, and Gimigliano in 2002, which claims that the secant varieties of tangential varieties to the $d$th Veronese embedding of the projective $n$-space $\mathbb{P}^n$ have the expected dimension, modulo a few well-known exceptions. As Bernardi, Catalisano, Gimigliano, and Idá demonstrated that the proof of this conjecture may be reduced to the case of cubics, i.e., $d=3$, the main contribution of this work is the resolution of this base case. The proposed proof proceeds by induction on the dimension $n$ of the projective space via a specialization argument. This reduces the proof to a large number of initial cases for the induction, which were settled using a computer-assisted proof. The individual base cases were computationally challenging problems. Indeed, the largest base case required us to deal with the tangential variety to the third Veronese embedding of $\mathbb{P}^{79}$ in $\mathbb{P}^{88559}$.
△ Less
Submitted 30 October, 2015; v1 submitted 7 October, 2015;
originally announced October 2015.
-
On generic identifiability of symmetric tensors of subgeneric rank
Authors:
Luca Chiantini,
Giorgio Ottaviani,
Nick Vannieuwenhoven
Abstract:
We prove that the general symmetric tensor in $S^d {\mathbb C}^{n+1}$ of rank r is identifiable, provided that r is smaller than the generic rank. That is, its Waring decomposition as a sum of r powers of linear forms is unique. Only three exceptional cases arise, all of which were known in the literature. Our original contribution regards the case of cubics ($d=3$), while for $d\ge 4$ we rely on…
▽ More
We prove that the general symmetric tensor in $S^d {\mathbb C}^{n+1}$ of rank r is identifiable, provided that r is smaller than the generic rank. That is, its Waring decomposition as a sum of r powers of linear forms is unique. Only three exceptional cases arise, all of which were known in the literature. Our original contribution regards the case of cubics ($d=3$), while for $d\ge 4$ we rely on known results on weak defectivity by Ballico, Ciliberto, Chiantini, and Mella.
△ Less
Submitted 8 January, 2016; v1 submitted 2 April, 2015;
originally announced April 2015.
-
An algorithm for generic and low-rank specific identifiability of complex tensors
Authors:
Luca Chiantini,
Giorgio Ottaviani,
Nick Vannieuwenhoven
Abstract:
We propose a new sufficient condition for verifying whether generic rank-r complex tensors of arbitrary order admit a unique decomposition as a linear combination of rank-1 tensors. A practical algorithm is proposed for verifying this condition, with which it was established that in all spaces of dimension less than 15000, with a few known exceptions, listed in the paper, generic identifiability h…
▽ More
We propose a new sufficient condition for verifying whether generic rank-r complex tensors of arbitrary order admit a unique decomposition as a linear combination of rank-1 tensors. A practical algorithm is proposed for verifying this condition, with which it was established that in all spaces of dimension less than 15000, with a few known exceptions, listed in the paper, generic identifiability holds for ranks up to one less than the generic rank of the space. This is the largest possible rank value for which generic identifiability can hold, except for spaces with a perfect shape. The algorithm can also verify the identifiability of a given specific rank-r decomposition, provided that it can be shown to correspond to a nonsingular point of the r-th order secant variety. For sufficiently small rank, which nevertheless improves upon the known bounds for specific identifiability, some local equations of this variety are known, allowing us to verify this property. As a particular example of our approach, we prove the identifiability of a specific 5x5x5 tensor of rank 7, which cannot be handled by the conditions recently provided in [I. Domanov and L. De Lathauwer, On the Uniqueness of the Canonical Polyadic Decomposition of third-order tensors--Part II: Uniqueness of the overall decomposition, SIAM J. Matrix Anal. Appl. 34(3), 2013]. Finally, we also present a surprising new class of weakly-defective Segre varieties that nevertheless turns out to admit a generically unique decomposition.
△ Less
Submitted 17 March, 2014;
originally announced March 2014.
