-
Extended Successive Convex Approximation for Phase Retrieval with Dictionary Learning
Authors:
Tianyi Liu,
Andreas M. Tillmann,
Yang Yang,
Yonina C. Eldar,
Marius Pesavento
Abstract:
Phase retrieval aims at reconstructing unknown signals from magnitude measurements of linear mixtures. In this paper, we consider the phase retrieval with dictionary learning problem, which includes an additional prior information that the measured signal admits a sparse representation over an unknown dictionary. The task is to jointly estimate the dictionary and the sparse representation from mag…
▽ More
Phase retrieval aims at reconstructing unknown signals from magnitude measurements of linear mixtures. In this paper, we consider the phase retrieval with dictionary learning problem, which includes an additional prior information that the measured signal admits a sparse representation over an unknown dictionary. The task is to jointly estimate the dictionary and the sparse representation from magnitude-only measurements. To this end, we study two complementary formulations and develop efficient parallel algorithms by extending the successive convex approximation framework using a smooth majorization. The first algorithm is termed compact-SCAphase and is preferable in the case of less diverse mixture models. It employs a compact formulation that avoids the use of auxiliary variables. The proposed algorithm is highly scalable and has reduced parameter tuning cost. The second algorithm, referred to as SCAphase, uses auxiliary variables and is favorable in the case of highly diverse mixture models. It also renders simple incorporation of additional side constraints. The performance of both methods is evaluated when applied to blind sparse channel estimation from subband magnitude measurements in a multi-antenna random access network. Simulation results demonstrate the efficiency of the proposed techniques compared to state-of-the-art methods.
△ Less
Submitted 21 January, 2023; v1 submitted 12 September, 2021;
originally announced September 2021.
-
Cardinality Minimization, Constraints, and Regularization: A Survey
Authors:
Andreas M. Tillmann,
Daniel Bienstock,
Andrea Lodi,
Alexandra Schwartz
Abstract:
We survey optimization problems that involve the cardinality of variable vectors in constraints or the objective function. We provide a unified viewpoint on the general problem classes and models, and give concrete examples from diverse application fields such as signal and image processing, portfolio selection, or machine learning. The paper discusses general-purpose modeling techniques and broad…
▽ More
We survey optimization problems that involve the cardinality of variable vectors in constraints or the objective function. We provide a unified viewpoint on the general problem classes and models, and give concrete examples from diverse application fields such as signal and image processing, portfolio selection, or machine learning. The paper discusses general-purpose modeling techniques and broadly applicable as well as problem-specific exact and heuristic solution approaches. While our perspective is that of mathematical optimization, a main goal of this work is to reach out to and build bridges between the different communities in which cardinality optimization problems are frequently encountered. In particular, we highlight that modern mixed-integer programming, which is often regarded as impractical due to commonly unsatisfactory behavior of black-box solvers applied to generic problem formulations, can in fact produce provably high-quality or even optimal solutions for cardinality optimization problems, even in large-scale real-world settings. Achieving such performance typically draws on the merits of problem-specific knowledge that may stem from different fields of application and, e.g., shed light on structural properties of a model or its solutions, or lead to the development of efficient heuristics; we also provide some illustrative examples.
△ Less
Submitted 8 August, 2022; v1 submitted 17 June, 2021;
originally announced June 2021.
-
Exact separation of forbidden-set cuts associated with redundant parity checks of binary linear codes
Authors:
Christian Puchert,
Andreas M. Tillmann
Abstract:
In recent years, several integer programming (IP) approaches were developed for maximum-likelihood decoding and minimum distance computation for binary linear codes. Two aspects in particular have been demonstrated to improve the performance of IP solvers as well as adaptive linear programming decoders: the dynamic generation of forbidden-set (FS) inequalities, a family of valid cutting planes, an…
▽ More
In recent years, several integer programming (IP) approaches were developed for maximum-likelihood decoding and minimum distance computation for binary linear codes. Two aspects in particular have been demonstrated to improve the performance of IP solvers as well as adaptive linear programming decoders: the dynamic generation of forbidden-set (FS) inequalities, a family of valid cutting planes, and the utilization of so-called redundant parity-checks (RPCs). However, to date, it had remained unclear how to solve the exact RPC separation problem (i.e., to determine whether or not there exists any violated FS inequality w.r.t. any known or unknown parity-check). In this note, we prove NP-hardness of this problem. Moreover, we formulate an IP model that combines the search for most violated FS cuts with the generation of RPCs, and report on computational experiments. Empirically, for various instances of the minimum distance problem, it turns out that while utilizing the exact separation IP does not appear to provide a computational advantage, it can apparently be avoided altogether by combining heuristics to generate RPC-based cuts.
△ Less
Submitted 7 April, 2020;
originally announced April 2020.
-
Structured Discrete Shape Approximation: Theoretical Complexity and Practical Algorithm
Authors:
Andreas M. Tillmann,
Leif Kobbelt
Abstract:
We consider the problem of approximating a two-dimensional shape contour (or curve segment) using discrete assembly systems, which allow to build geometric structures based on limited sets of node and edge types subject to edge length and orientation restrictions. We show that already deciding feasibility of such approximation problems is NP-hard, and remains intractable even for very simple setup…
▽ More
We consider the problem of approximating a two-dimensional shape contour (or curve segment) using discrete assembly systems, which allow to build geometric structures based on limited sets of node and edge types subject to edge length and orientation restrictions. We show that already deciding feasibility of such approximation problems is NP-hard, and remains intractable even for very simple setups. We then devise an algorithmic framework that combines shape sampling with exact cardinality-minimization to obtain good approximations using few components. As a particular application and showcase example, we discuss approximating shape contours using the classical Zometool construction kit and provide promising computational results, demonstrating that our algorithm is capable of obtaining good shape representations within reasonable time, in spite of the problem's general intractability. We conclude the paper with an outlook on possible extensions of the developed methodology, in particular regarding 3D shape approximation tasks.
△ Less
Submitted 1 February, 2021; v1 submitted 19 September, 2019;
originally announced September 2019.
-
Joint Antenna Selection and Phase-Only Beamforming Using Mixed-Integer Nonlinear Programming
Authors:
Tobias Fischer,
Ganapati Hegde,
Frederic Matter,
Marius Pesavento,
Marc E. Pfetsch,
Andreas M. Tillmann
Abstract:
In this paper, we consider the problem of joint antenna selection and analog beamformer design in downlink single-group multicast networks. Our objective is to reduce the hardware costs by minimizing the number of required phase shifters at the transmitter while fulfilling given distortion limits at the receivers. We formulate the problem as an L0 minimization problem and devise a novel branch-and…
▽ More
In this paper, we consider the problem of joint antenna selection and analog beamformer design in downlink single-group multicast networks. Our objective is to reduce the hardware costs by minimizing the number of required phase shifters at the transmitter while fulfilling given distortion limits at the receivers. We formulate the problem as an L0 minimization problem and devise a novel branch-and-cut based algorithm to solve the resulting mixed-integer nonlinear program to optimality. We also propose a suboptimal heuristic algorithm to solve the above problem approximately with a low computational complexity. Computational results illustrate that the solutions produced by the proposed heuristic algorithm are optimal in most cases. The results also indicate that the performance of the optimal methods can be significantly improved by initializing with the result of the suboptimal method.
△ Less
Submitted 22 February, 2018;
originally announced February 2018.
-
A Primal-Dual Homotopy Algorithm for $\ell_{1}$-Minimization with $\ell_{\infty}$-Constraints
Authors:
Christoph Brauer,
Dirk A. Lorenz,
Andreas M. Tillmann
Abstract:
In this paper we propose a primal-dual homotopy method for $\ell_1$-minimization problems with infinity norm constraints in the context of sparse reconstruction. The natural homotopy parameter is the value of the bound for the constraints and we show that there exists a piecewise linear solution path with finitely many break points for the primal problem and a respective piecewise constant path fo…
▽ More
In this paper we propose a primal-dual homotopy method for $\ell_1$-minimization problems with infinity norm constraints in the context of sparse reconstruction. The natural homotopy parameter is the value of the bound for the constraints and we show that there exists a piecewise linear solution path with finitely many break points for the primal problem and a respective piecewise constant path for the dual problem. We show that by solving a small linear program, one can jump to the next primal break point and then, solving another small linear program, a new optimal dual solution is calculated which enables the next such jump in the subsequent iteration. Using a theorem of the alternative, we show that the method never gets stuck and indeed calculates the whole path in a finite number of steps.
Numerical experiments demonstrate the effectiveness of our algorithm. In many cases, our method significantly outperforms commercial LP solvers; this is possible since our approach employs a sequence of considerably simpler auxiliary linear programs that can be solved efficiently with specialized active-set strategies.
△ Less
Submitted 31 October, 2016;
originally announced October 2016.
-
Sparse Recovery With Integrality Constraints
Authors:
Jan-Hendrik Lange,
Marc E. Pfetsch,
Bianca M. Seib,
Andreas M. Tillmann
Abstract:
We investigate conditions for the unique recoverability of sparse integer-valued signals from a small number of linear measurements. Both the objective of minimizing the number of nonzero components, the so-called $\ell_0$-norm, as well as its popular substitute, the $\ell_1$-norm, are covered. Furthermore, integrality constraints and possible bounds on the variables are investigated. Our results…
▽ More
We investigate conditions for the unique recoverability of sparse integer-valued signals from a small number of linear measurements. Both the objective of minimizing the number of nonzero components, the so-called $\ell_0$-norm, as well as its popular substitute, the $\ell_1$-norm, are covered. Furthermore, integrality constraints and possible bounds on the variables are investigated. Our results show that the additional prior knowledge of signal integrality allows for recovering more signals than what can be guaranteed by the established recovery conditions from (continuous) compressed sensing. Moreover, even though the considered problems are \NP-hard in general (even with an $\ell_1$-objective), we investigate testing the $\ell_0$-recovery conditions via some numerical experiments. It turns out that the corresponding problems are quite hard to solve in practice using black-box software. However, medium-sized instances of $\ell_0$- and $\ell_1$-minimization with binary variables can be solved exactly within reasonable time.
△ Less
Submitted 17 September, 2019; v1 submitted 30 August, 2016;
originally announced August 2016.
-
DOLPHIn - Dictionary Learning for Phase Retrieval
Authors:
Andreas M. Tillmann,
Yonina C. Eldar,
Julien Mairal
Abstract:
We propose a new algorithm to learn a dictionary for reconstructing and sparsely encoding signals from measurements without phase. Specifically, we consider the task of estimating a two-dimensional image from squared-magnitude measurements of a complex-valued linear transformation of the original image. Several recent phase retrieval algorithms exploit underlying sparsity of the unknown signal in…
▽ More
We propose a new algorithm to learn a dictionary for reconstructing and sparsely encoding signals from measurements without phase. Specifically, we consider the task of estimating a two-dimensional image from squared-magnitude measurements of a complex-valued linear transformation of the original image. Several recent phase retrieval algorithms exploit underlying sparsity of the unknown signal in order to improve recovery performance. In this work, we consider such a sparse signal prior in the context of phase retrieval, when the sparsifying dictionary is not known in advance. Our algorithm jointly reconstructs the unknown signal - possibly corrupted by noise - and learns a dictionary such that each patch of the estimated image can be sparsely represented. Numerical experiments demonstrate that our approach can obtain significantly better reconstructions for phase retrieval problems with noise than methods that cannot exploit such "hidden" sparsity. Moreover, on the theoretical side, we provide a convergence result for our method.
△ Less
Submitted 3 August, 2016; v1 submitted 6 February, 2016;
originally announced February 2016.
-
On the Computational Intractability of Exact and Approximate Dictionary Learning
Authors:
Andreas M. Tillmann
Abstract:
The efficient sparse coding and reconstruction of signal vectors via linear observations has received a tremendous amount of attention over the last decade. In this context, the automated learning of a suitable basis or overcomplete dictionary from training data sets of certain signal classes for use in sparse representations has turned out to be of particular importance regarding practical signal…
▽ More
The efficient sparse coding and reconstruction of signal vectors via linear observations has received a tremendous amount of attention over the last decade. In this context, the automated learning of a suitable basis or overcomplete dictionary from training data sets of certain signal classes for use in sparse representations has turned out to be of particular importance regarding practical signal processing applications. Most popular dictionary learning algorithms involve NP-hard sparse recovery problems in each iteration, which may give some indication about the complexity of dictionary learning but does not constitute an actual proof of computational intractability. In this technical note, we show that learning a dictionary with which a given set of training signals can be represented as sparsely as possible is indeed NP-hard. Moreover, we also establish hardness of approximating the solution to within large factors of the optimal sparsity level. Furthermore, we give NP-hardness and non-approximability results for a recent dictionary learning variation called the sensor permutation problem. Along the way, we also obtain a new non-approximability result for the classical sparse recovery problem from compressed sensing.
△ Less
Submitted 3 August, 2014; v1 submitted 26 May, 2014;
originally announced May 2014.
-
Projection onto the Cosparse Set is NP-Hard
Authors:
Andreas M. Tillmann,
RĂ©mi Gribonval,
Marc E. Pfetsch
Abstract:
The computational complexity of a problem arising in the context of sparse optimization is considered, namely, the projection onto the set of $k$-cosparse vectors w.r.t. some given matrix $\Omeg$. It is shown that this projection problem is (strongly) \NP-hard, even in the special cases in which the matrix $\Omeg$ contains only ternary or bipolar coefficients. Interestingly, this is in contrast to…
▽ More
The computational complexity of a problem arising in the context of sparse optimization is considered, namely, the projection onto the set of $k$-cosparse vectors w.r.t. some given matrix $\Omeg$. It is shown that this projection problem is (strongly) \NP-hard, even in the special cases in which the matrix $\Omeg$ contains only ternary or bipolar coefficients. Interestingly, this is in contrast to the projection onto the set of $k$-sparse vectors, which is trivially solved by kee** only the $k$ largest coefficients.
△ Less
Submitted 11 March, 2014; v1 submitted 21 March, 2013;
originally announced March 2013.
-
The Computational Complexity of the Restricted Isometry Property, the Nullspace Property, and Related Concepts in Compressed Sensing
Authors:
Andreas M. Tillmann,
Marc E. Pfetsch
Abstract:
This paper deals with the computational complexity of conditions which guarantee that the NP-hard problem of finding the sparsest solution to an underdetermined linear system can be solved by efficient algorithms. In the literature, several such conditions have been introduced. The most well-known ones are the mutual coherence, the restricted isometry property (RIP), and the nullspace property (NS…
▽ More
This paper deals with the computational complexity of conditions which guarantee that the NP-hard problem of finding the sparsest solution to an underdetermined linear system can be solved by efficient algorithms. In the literature, several such conditions have been introduced. The most well-known ones are the mutual coherence, the restricted isometry property (RIP), and the nullspace property (NSP). While evaluating the mutual coherence of a given matrix is easy, it has been suspected for some time that evaluating RIP and NSP is computationally intractable in general. We confirm these conjectures by showing that for a given matrix A and positive integer k, computing the best constants for which the RIP or NSP hold is, in general, NP-hard. These results are based on the fact that determining the spark of a matrix is NP-hard, which is also established in this paper. Furthermore, we also give several complexity statements about problems related to the above concepts.
△ Less
Submitted 4 November, 2013; v1 submitted 9 May, 2012;
originally announced May 2012.
-
An Infeasible-Point Subgradient Method Using Adaptive Approximate Projections
Authors:
Dirk A. Lorenz,
Marc E. Pfetsch,
Andreas M. Tillmann
Abstract:
We propose a new subgradient method for the minimization of nonsmooth convex functions over a convex set. To speed up computations we use adaptive approximate projections only requiring to move within a certain distance of the exact projections (which decreases in the course of the algorithm). In particular, the iterates in our method can be infeasible throughout the whole procedure. Nevertheless,…
▽ More
We propose a new subgradient method for the minimization of nonsmooth convex functions over a convex set. To speed up computations we use adaptive approximate projections only requiring to move within a certain distance of the exact projections (which decreases in the course of the algorithm). In particular, the iterates in our method can be infeasible throughout the whole procedure. Nevertheless, we provide conditions which ensure convergence to an optimal feasible point under suitable assumptions. One convergence result deals with step size sequences that are fixed a priori. Two other results handle dynamic Polyak-type step sizes depending on a lower or upper estimate of the optimal objective function value, respectively. Additionally, we briefly sketch two applications: Optimization with convex chance constraints, and finding the minimum l1-norm solution to an underdetermined linear system, an important problem in Compressed Sensing.
△ Less
Submitted 8 March, 2013; v1 submitted 28 April, 2011;
originally announced April 2011.