-
The CausalBench challenge: A machine learning contest for gene network inference from single-cell perturbation data
Authors:
Mathieu Chevalley,
Jacob Sackett-Sanders,
Yusuf Roohani,
Pascal Notin,
Artemy Bakulin,
Dariusz Brzezinski,
Kaiwen Deng,
Yuanfang Guan,
Justin Hong,
Michael Ibrahim,
Wojciech Kotlowski,
Marcin Kowiel,
Panagiotis Misiakos,
Achille Nazaret,
Markus Püschel,
Chris Wendler,
Arash Mehrjou,
Patrick Schwab
Abstract:
In drug discovery, map** interactions between genes within cellular systems is a crucial early step. This helps formulate hypotheses regarding molecular mechanisms that could potentially be targeted by future medicines. The CausalBench Challenge was an initiative to invite the machine learning community to advance the state of the art in constructing gene-gene interaction networks. These network…
▽ More
In drug discovery, map** interactions between genes within cellular systems is a crucial early step. This helps formulate hypotheses regarding molecular mechanisms that could potentially be targeted by future medicines. The CausalBench Challenge was an initiative to invite the machine learning community to advance the state of the art in constructing gene-gene interaction networks. These networks, derived from large-scale, real-world datasets of single cells under various perturbations, are crucial for understanding the causal mechanisms underlying disease biology. Using the framework provided by the CausalBench benchmark, participants were tasked with enhancing the capacity of the state of the art methods to leverage large-scale genetic perturbation data. This report provides an analysis and summary of the methods submitted during the challenge to give a partial image of the state of the art at the time of the challenge. The winning solutions significantly improved performance compared to previous baselines, establishing a new state of the art for this critical task in biology and medicine.
△ Less
Submitted 29 August, 2023;
originally announced August 2023.
-
QIGen: Generating Efficient Kernels for Quantized Inference on Large Language Models
Authors:
Tommaso Pegolotti,
Elias Frantar,
Dan Alistarh,
Markus Püschel
Abstract:
We present ongoing work on a new automatic code generation approach for supporting quantized generative inference on LLMs such as LLaMA or OPT on off-the-shelf CPUs. Our approach is informed by the target architecture and a performance model, including both hardware characteristics and method-specific accuracy constraints. Results on CPU-based inference for LLaMA models show that our approach can…
▽ More
We present ongoing work on a new automatic code generation approach for supporting quantized generative inference on LLMs such as LLaMA or OPT on off-the-shelf CPUs. Our approach is informed by the target architecture and a performance model, including both hardware characteristics and method-specific accuracy constraints. Results on CPU-based inference for LLaMA models show that our approach can lead to high performance and high accuracy, comparing favorably to the best existing open-source solution. A preliminary implementation is available at https://github.com/IST-DASLab/QIGen.
△ Less
Submitted 7 July, 2023;
originally announced July 2023.
-
Learning DAGs from Data with Few Root Causes
Authors:
Panagiotis Misiakos,
Chris Wendler,
Markus Püschel
Abstract:
We present a novel perspective and algorithm for learning directed acyclic graphs (DAGs) from data generated by a linear structural equation model (SEM). First, we show that a linear SEM can be viewed as a linear transform that, in prior work, computes the data from a dense input vector of random valued root causes (as we will call them) associated with the nodes. Instead, we consider the case of…
▽ More
We present a novel perspective and algorithm for learning directed acyclic graphs (DAGs) from data generated by a linear structural equation model (SEM). First, we show that a linear SEM can be viewed as a linear transform that, in prior work, computes the data from a dense input vector of random valued root causes (as we will call them) associated with the nodes. Instead, we consider the case of (approximately) few root causes and also introduce noise in the measurement of the data. Intuitively, this means that the DAG data is produced by few data-generating events whose effect percolates through the DAG. We prove identifiability in this new setting and show that the true DAG is the global minimizer of the $L^0$-norm of the vector of root causes. For data with few root causes, with and without noise, we show superior performance compared to prior DAG learning methods.
△ Less
Submitted 23 January, 2024; v1 submitted 25 May, 2023;
originally announced May 2023.
-
Fast Möbius and Zeta Transforms
Authors:
Tommaso Pegolotti,
Bastian Seifert,
Markus Püschel
Abstract:
Möbius inversion of functions on partially ordered sets (posets) $\mathcal{P}$ is a classical tool in combinatorics. For finite posets it consists of two, mutually inverse, linear transformations called zeta and Möbius transform, respectively. In this paper we provide novel fast algorithms for both that require $O(nk)$ time and space, where $n = |\mathcal{P}|$ and $k$ is the width (length of longe…
▽ More
Möbius inversion of functions on partially ordered sets (posets) $\mathcal{P}$ is a classical tool in combinatorics. For finite posets it consists of two, mutually inverse, linear transformations called zeta and Möbius transform, respectively. In this paper we provide novel fast algorithms for both that require $O(nk)$ time and space, where $n = |\mathcal{P}|$ and $k$ is the width (length of longest antichain) of $\mathcal{P}$, compared to $O(n^2)$ for a direct computation. Our approach assumes that $\mathcal{P}$ is given as directed acyclic graph (DAG) $(\mathcal{E}, \mathcal{P})$. The algorithms are then constructed using a chain decomposition for a one time cost of $O(|\mathcal{E}| + |\mathcal{E}_\text{red}| k)$, where $\mathcal{E}_\text{red}$ is the number of edges in the DAG's transitive reduction. We show benchmarks with implementations of all algorithms including parallelized versions. The results show that our algorithms enable Möbius inversion on posets with millions of nodes in seconds if the defining DAGs are sufficiently sparse.
△ Less
Submitted 24 November, 2022;
originally announced November 2022.
-
Causal Fourier Analysis on Directed Acyclic Graphs and Posets
Authors:
Bastian Seifert,
Chris Wendler,
Markus Püschel
Abstract:
We present a novel form of Fourier analysis, and associated signal processing concepts, for signals (or data) indexed by edge-weighted directed acyclic graphs (DAGs). This means that our Fourier basis yields an eigendecomposition of a suitable notion of shift and convolution operators that we define. DAGs are the common model to capture causal relationships between data values and in this case our…
▽ More
We present a novel form of Fourier analysis, and associated signal processing concepts, for signals (or data) indexed by edge-weighted directed acyclic graphs (DAGs). This means that our Fourier basis yields an eigendecomposition of a suitable notion of shift and convolution operators that we define. DAGs are the common model to capture causal relationships between data values and in this case our proposed Fourier analysis relates data with its causes under a linearity assumption that we define. The definition of the Fourier transform requires the transitive closure of the weighted DAG for which several forms are possible depending on the interpretation of the edge weights. Examples include level of influence, distance, or pollution distribution. Our framework is different from prior GSP: it is specific to DAGs and leverages, and extends, the classical theory of Moebius inversion from combinatorics. For a prototypical application we consider DAGs modeling dynamic networks in which edges change over time. Specifically, we model the spread of an infection on such a DAG obtained from real-world contact tracing data and learn the infection signal from samples assuming sparsity in the Fourier domain.
△ Less
Submitted 9 August, 2023; v1 submitted 16 September, 2022;
originally announced September 2022.
-
PRIMA: General and Precise Neural Network Certification via Scalable Convex Hull Approximations
Authors:
Mark Niklas Müller,
Gleb Makarchuk,
Gagandeep Singh,
Markus Püschel,
Martin Vechev
Abstract:
Formal verification of neural networks is critical for their safe adoption in real-world applications. However, designing a precise and scalable verifier which can handle different activation functions, realistic network architectures and relevant specifications remains an open and difficult challenge. In this paper, we take a major step forward in addressing this challenge and present a new verif…
▽ More
Formal verification of neural networks is critical for their safe adoption in real-world applications. However, designing a precise and scalable verifier which can handle different activation functions, realistic network architectures and relevant specifications remains an open and difficult challenge. In this paper, we take a major step forward in addressing this challenge and present a new verification framework, called PRIMA. PRIMA is both (i) general: it handles any non-linear activation function, and (ii) precise: it computes precise convex abstractions involving multiple neurons via novel convex hull approximation algorithms that leverage concepts from computational geometry. The algorithms have polynomial complexity, yield fewer constraints, and minimize precision loss. We evaluate the effectiveness of PRIMA on a variety of challenging tasks from prior work. Our results show that PRIMA is significantly more precise than the state-of-the-art, verifying robustness to input perturbations for up to 20%, 30%, and 34% more images than existing work on ReLU-, Sigmoid-, and Tanh-based networks, respectively. Further, PRIMA enables, for the first time, the precise verification of a realistic neural network for autonomous driving within a few minutes.
△ Less
Submitted 28 February, 2022; v1 submitted 5 March, 2021;
originally announced March 2021.
-
Discrete Signal Processing on Meet/Join Lattices
Authors:
Markus Püschel,
Bastian Seifert,
Chris Wendler
Abstract:
A lattice is a partially ordered set supporting a meet (or join) operation that returns the largest lower bound (smallest upper bound) of two elements. Just like graphs, lattices are a fundamental structure that occurs across domains including social data analysis, natural language processing, computational chemistry and biology, and database theory. In this paper we introduce discrete-lattice sig…
▽ More
A lattice is a partially ordered set supporting a meet (or join) operation that returns the largest lower bound (smallest upper bound) of two elements. Just like graphs, lattices are a fundamental structure that occurs across domains including social data analysis, natural language processing, computational chemistry and biology, and database theory. In this paper we introduce discrete-lattice signal processing (DLSP), an SP framework for data, or signals, indexed by such lattices. We use the meet (or join) to define a shift operation and derive associated notions of filtering, Fourier basis and transform, and frequency response. We show that the spectrum of a lattice signal inherits the lattice structure of the signal domain and derive a sampling theorem. Finally, we show two prototypical applications: spectral analysis of formal concept lattices in social science and sampling and Wiener filtering of multiset lattices in combinatorial auctions. Formal concept lattices are a compressed representation of relations between objects and attributes. Since relations are equivalent to bipartite graphs and hypergraphs, DLSP offers a form of Fourier analysis for these structures.
△ Less
Submitted 6 July, 2021; v1 submitted 8 December, 2020;
originally announced December 2020.
-
Learning Set Functions that are Sparse in Non-Orthogonal Fourier Bases
Authors:
Chris Wendler,
Andisheh Amrollahi,
Bastian Seifert,
Andreas Krause,
Markus Püschel
Abstract:
Many applications of machine learning on discrete domains, such as learning preference functions in recommender systems or auctions, can be reduced to estimating a set function that is sparse in the Fourier domain. In this work, we present a new family of algorithms for learning Fourier-sparse set functions. They require at most $nk - k \log_2 k + k$ queries (set function evaluations), under mild…
▽ More
Many applications of machine learning on discrete domains, such as learning preference functions in recommender systems or auctions, can be reduced to estimating a set function that is sparse in the Fourier domain. In this work, we present a new family of algorithms for learning Fourier-sparse set functions. They require at most $nk - k \log_2 k + k$ queries (set function evaluations), under mild conditions on the Fourier coefficients, where $n$ is the size of the ground set and $k$ the number of non-zero Fourier coefficients. In contrast to other work that focused on the orthogonal Walsh-Hadamard transform, our novel algorithms operate with recently introduced non-orthogonal Fourier transforms that offer different notions of Fourier-sparsity. These naturally arise when modeling, e.g., sets of items forming substitutes and complements. We demonstrate effectiveness on several real-world applications.
△ Less
Submitted 29 March, 2021; v1 submitted 1 October, 2020;
originally announced October 2020.
-
Fourier Analysis-based Iterative Combinatorial Auctions
Authors:
Jakob Weissteiner,
Chris Wendler,
Sven Seuken,
Ben Lubin,
Markus Püschel
Abstract:
Recent advances in Fourier analysis have brought new tools to efficiently represent and learn set functions. In this paper, we bring the power of Fourier analysis to the design of combinatorial auctions (CAs). The key idea is to approximate bidders' value functions using Fourier-sparse set functions, which can be computed using a relatively small number of queries. Since this number is still too l…
▽ More
Recent advances in Fourier analysis have brought new tools to efficiently represent and learn set functions. In this paper, we bring the power of Fourier analysis to the design of combinatorial auctions (CAs). The key idea is to approximate bidders' value functions using Fourier-sparse set functions, which can be computed using a relatively small number of queries. Since this number is still too large for practical CAs, we propose a new hybrid design: we first use neural networks (NNs) to learn bidders' values and then apply Fourier analysis to the learned representations. On a technical level, we formulate a Fourier transform-based winner determination problem and derive its mixed integer program formulation. Based on this, we devise an iterative CA that asks Fourier-based queries. We experimentally show that our hybrid ICA achieves higher efficiency than prior auction designs, leads to a fairer distribution of social welfare, and significantly reduces runtime. With this paper, we are the first to leverage Fourier analysis in CA design and lay the foundation for future work in this area. Our code is available on GitHub: https://github.com/marketdesignresearch/FA-based-ICAs.
△ Less
Submitted 11 March, 2023; v1 submitted 22 September, 2020;
originally announced September 2020.
-
Scaling Polyhedral Neural Network Verification on GPUs
Authors:
Christoph Müller,
François Serre,
Gagandeep Singh,
Markus Püschel,
Martin Vechev
Abstract:
Certifying the robustness of neural networks against adversarial attacks is essential to their reliable adoption in safety-critical systems such as autonomous driving and medical diagnosis. Unfortunately, state-of-the-art verifiers either do not scale to bigger networks or are too imprecise to prove robustness, limiting their practical adoption. In this work, we introduce GPUPoly, a scalable verif…
▽ More
Certifying the robustness of neural networks against adversarial attacks is essential to their reliable adoption in safety-critical systems such as autonomous driving and medical diagnosis. Unfortunately, state-of-the-art verifiers either do not scale to bigger networks or are too imprecise to prove robustness, limiting their practical adoption. In this work, we introduce GPUPoly, a scalable verifier that can prove the robustness of significantly larger deep neural networks than previously possible. The key technical insight behind GPUPoly is the design of custom, sound polyhedra algorithms for neural network verification on a GPU. Our algorithms leverage the available GPU parallelism and inherent sparsity of the underlying verification task. GPUPoly scales to large networks: for example, it can prove the robustness of a 1M neuron, 34-layer deep residual network in approximately 34.5 ms. We believe GPUPoly is a promising step towards practical verification of real-world neural networks.
△ Less
Submitted 18 May, 2021; v1 submitted 20 July, 2020;
originally announced July 2020.
-
Digraph Signal Processing with Generalized Boundary Conditions
Authors:
Bastian Seifert,
Markus Püschel
Abstract:
Signal processing on directed graphs (digraphs) is problematic, since the graph shift, and thus associated filters, are in general not diagonalizable. Furthermore, the Fourier transform in this case is now obtained from the Jordan decomposition, which may not be computable at all for large graphs. We propose a novel and general solution for this problem based on matrix perturbation theory: We desi…
▽ More
Signal processing on directed graphs (digraphs) is problematic, since the graph shift, and thus associated filters, are in general not diagonalizable. Furthermore, the Fourier transform in this case is now obtained from the Jordan decomposition, which may not be computable at all for large graphs. We propose a novel and general solution for this problem based on matrix perturbation theory: We design an algorithm that adds a small number of edges to a given digraph to destroy nontrivial Jordan blocks. The obtained digraph is then diagonalizable and yields, as we show, an approximate eigenbasis and Fourier transform for the original digraph. We explain why and how this construction can be viewed as generalized form of boundary conditions, a common practice in signal processing. Our experiments with random and real world graphs show that we can scale to graphs with a few thousands nodes, and obtain Fourier transforms that are close to orthogonal while still diagonalizing an intuitive notion of convolution. Our method works with adjacency and Laplacian shift and can be used as preprocessing step to enable further processing as we show with a prototypical Wiener filter application.
△ Less
Submitted 8 February, 2021; v1 submitted 19 May, 2020;
originally announced May 2020.
-
Discrete Signal Processing with Set Functions
Authors:
Markus Püschel,
Chris Wendler
Abstract:
Set functions are functions (or signals) indexed by the powerset (set of all subsets) of a finite set N. They are fundamental and ubiquitous in many application domains and have been used, for example, to formally describe or quantify loss functions for semantic image segmentation, the informativeness of sensors in sensor networks the utility of sets of items in recommender systems, cooperative ga…
▽ More
Set functions are functions (or signals) indexed by the powerset (set of all subsets) of a finite set N. They are fundamental and ubiquitous in many application domains and have been used, for example, to formally describe or quantify loss functions for semantic image segmentation, the informativeness of sensors in sensor networks the utility of sets of items in recommender systems, cooperative games in game theory, or bidders in combinatorial auctions. In particular, the subclass of submodular functions occurs in many optimization and machine learning problems. In this paper, we derive discrete-set signal processing (SP), a novel shift-invariant linear signal processing framework for set functions. Discrete-set SP considers different notions of shift obtained from set union and difference operations. For each shift it provides associated notions of shift-invariant filters, convolution, Fourier transform, and frequency response. We provide intuition for our framework using the concept of generalized coverage function that we define, identify multivariate mutual information as a special case of a discrete-set spectrum, and motivate frequency ordering. Our work brings a new set of tools for analyzing and processing set functions, and, in particular, for dealing with their exponential nature. We show two prototypical applications and experiments: compression in submodular function optimization and sampling for preference elicitation in combinatorial auctions.
△ Less
Submitted 22 October, 2020; v1 submitted 28 January, 2020;
originally announced January 2020.
-
Powerset Convolutional Neural Networks
Authors:
Chris Wendler,
Dan Alistarh,
Markus Püschel
Abstract:
We present a novel class of convolutional neural networks (CNNs) for set functions, i.e., data indexed with the powerset of a finite set. The convolutions are derived as linear, shift-equivariant functions for various notions of shifts on set functions. The framework is fundamentally different from graph convolutions based on the Laplacian, as it provides not one but several basic shifts, one for…
▽ More
We present a novel class of convolutional neural networks (CNNs) for set functions, i.e., data indexed with the powerset of a finite set. The convolutions are derived as linear, shift-equivariant functions for various notions of shifts on set functions. The framework is fundamentally different from graph convolutions based on the Laplacian, as it provides not one but several basic shifts, one for each element in the ground set. Prototypical experiments with several set function classification tasks on synthetic datasets and on datasets derived from real-world hypergraphs demonstrate the potential of our new powerset CNNs.
△ Less
Submitted 15 January, 2020; v1 submitted 5 September, 2019;
originally announced September 2019.
-
Program Generation for Linear Algebra Using Multiple Layers of DSLs
Authors:
Daniele G. Spampinato,
Diego Fabregat-Traver,
Markus Püschel,
Paolo Bientinesi
Abstract:
Numerical software in computational science and engineering often relies on highly-optimized building blocks from libraries such as BLAS and LAPACK, and while such libraries provide portable performance for a wide range of computing architectures, they still present limitations in terms of flexibility. We advocate a domain-specific program generator capable of producing library routines tailored t…
▽ More
Numerical software in computational science and engineering often relies on highly-optimized building blocks from libraries such as BLAS and LAPACK, and while such libraries provide portable performance for a wide range of computing architectures, they still present limitations in terms of flexibility. We advocate a domain-specific program generator capable of producing library routines tailored to the specific needs of the application in terms of sizes, interface, and target architecture.
△ Less
Submitted 20 June, 2019;
originally announced June 2019.
-
On Linear Learning with Manycore Processors
Authors:
Eliza Wszola,
Celestine Mendler-Dünner,
Martin Jaggi,
Markus Püschel
Abstract:
A new generation of manycore processors is on the rise that offers dozens and more cores on a chip and, in a sense, fuses host processor and accelerator. In this paper we target the efficient training of generalized linear models on these machines. We propose a novel approach for achieving parallelism which we call Heterogeneous Tasks on Homogeneous Cores (HTHC). It divides the problem into multip…
▽ More
A new generation of manycore processors is on the rise that offers dozens and more cores on a chip and, in a sense, fuses host processor and accelerator. In this paper we target the efficient training of generalized linear models on these machines. We propose a novel approach for achieving parallelism which we call Heterogeneous Tasks on Homogeneous Cores (HTHC). It divides the problem into multiple fundamentally different tasks, which themselves are parallelized. For evaluation, we design a detailed, architecture-cognizant implementation of our scheme on a recent 72-core Knights Landing processor that is adaptive to the cache, memory, and core structure. Our library efficiently supports dense and sparse datasets as well as 4-bit quantized data for further possible gains in performance. We show benchmarks for Lasso and SVM with different data sets against straightforward parallel implementations and prior software. In particular, for Lasso on dense data, we improve the state-of-the-art by an order of magnitude.
△ Less
Submitted 5 February, 2020; v1 submitted 2 May, 2019;
originally announced May 2019.
-
Program Generation for Small-Scale Linear Algebra Applications
Authors:
Daniele G. Spampinato,
Diego Fabregat-Traver,
Paolo Bientinesi,
Markus Pueschel
Abstract:
We present SLinGen, a program generation system for linear algebra. The input to SLinGen is an application expressed mathematically in a linear-algebra-inspired language (LA) that we define. LA provides basic scalar/vector/matrix additions/multiplications and higher level operations including linear systems solvers, Cholesky and LU factorizations. The output of SLinGen is performance-optimized sin…
▽ More
We present SLinGen, a program generation system for linear algebra. The input to SLinGen is an application expressed mathematically in a linear-algebra-inspired language (LA) that we define. LA provides basic scalar/vector/matrix additions/multiplications and higher level operations including linear systems solvers, Cholesky and LU factorizations. The output of SLinGen is performance-optimized single-source C code, optionally vectorized with intrinsics. The target of SLinGen are small-scale computations on fixed-size operands, for which a straightforward implementation using optimized libraries (e.g., BLAS or LAPACK) is known to yield suboptimal performance (besides increasing code size and introducing dependencies), but which are crucial in control, signal processing, computer vision, and other domains. Internally, SLinGen uses synthesis and DSL-based techniques to optimize at a high level of abstraction. We benchmark our program generator on three prototypical applications: the Kalman filter, Gaussian process regression, and an L1-analysis convex solver, as well as basic routines including Cholesky factorization and solvers for the continuous-time Lyapunov and Sylvester equations. The results show significant speed-ups compared to straightforward C with Intel icc and clang with a polyhedral optimizer, as well as library-based and template-based implementations.
△ Less
Submitted 12 May, 2018;
originally announced May 2018.
-
Compressive Sensing Using Iterative Hard Thresholding with Low Precision Data Representation: Theory and Applications
Authors:
Nezihe Merve Gürel,
Kaan Kara,
Alen Stojanov,
Tyler Smith,
Thomas Lemmin,
Dan Alistarh,
Markus Püschel,
Ce Zhang
Abstract:
Modern scientific instruments produce vast amounts of data, which can overwhelm the processing ability of computer systems. Lossy compression of data is an intriguing solution, but comes with its own drawbacks, such as potential signal loss, and the need for careful optimization of the compression ratio. In this work, we focus on a setting where this problem is especially acute: compressive sensin…
▽ More
Modern scientific instruments produce vast amounts of data, which can overwhelm the processing ability of computer systems. Lossy compression of data is an intriguing solution, but comes with its own drawbacks, such as potential signal loss, and the need for careful optimization of the compression ratio. In this work, we focus on a setting where this problem is especially acute: compressive sensing frameworks for interferometry and medical imaging. We ask the following question: can the precision of the data representation be lowered for all inputs, with recovery guarantees and practical performance? Our first contribution is a theoretical analysis of the normalized Iterative Hard Thresholding (IHT) algorithm when all input data, meaning both the measurement matrix and the observation vector are quantized aggressively. We present a variant of low precision normalized {IHT} that, under mild conditions, can still provide recovery guarantees. The second contribution is the application of our quantization framework to radio astronomy and magnetic resonance imaging. We show that lowering the precision of the data can significantly accelerate image recovery. We evaluate our approach on telescope data and samples of brain images using CPU and FPGA implementations achieving up to a 9x speed-up with negligible loss of recovery quality.
△ Less
Submitted 22 December, 2020; v1 submitted 13 February, 2018;
originally announced February 2018.
-
Characterizing and Enumerating Walsh-Hadamard Transform Algorithms
Authors:
François Serre,
Markus Püschel
Abstract:
We propose a way of characterizing the algorithms computing a Walsh-Hadamard transform that consist of a sequence of arrays of butterflies ($I_{2^{n-1}}\otimes \text{DFT}_2$) interleaved by linear permutations. Linear permutations are those that map linearly the binary representation of its element indices. We also propose a method to enumerate these algorithms.
We propose a way of characterizing the algorithms computing a Walsh-Hadamard transform that consist of a sequence of arrays of butterflies ($I_{2^{n-1}}\otimes \text{DFT}_2$) interleaved by linear permutations. Linear permutations are those that map linearly the binary representation of its element indices. We also propose a method to enumerate these algorithms.
△ Less
Submitted 22 October, 2017;
originally announced October 2017.
-
Distributed Optimization With Local Domains: Applications in MPC and Network Flows
Authors:
João F. C. Mota,
João M. F. Xavier,
Pedro M. Q. Aguiar,
Markus Püschel
Abstract:
In this paper we consider a network with $P$ nodes, where each node has exclusive access to a local cost function. Our contribution is a communication-efficient distributed algorithm that finds a vector $x^\star$ minimizing the sum of all the functions. We make the additional assumption that the functions have intersecting local domains, i.e., each function depends only on some components of the v…
▽ More
In this paper we consider a network with $P$ nodes, where each node has exclusive access to a local cost function. Our contribution is a communication-efficient distributed algorithm that finds a vector $x^\star$ minimizing the sum of all the functions. We make the additional assumption that the functions have intersecting local domains, i.e., each function depends only on some components of the variable. Consequently, each node is interested in knowing only some components of $x^\star$, not the entire vector. This allows for improvement in communication-efficiency. We apply our algorithm to model predictive control (MPC) and to network flow problems and show, through experiments on large networks, that our proposed algorithm requires less communications to converge than prior algorithms.
△ Less
Submitted 8 May, 2013;
originally announced May 2013.
-
Distributed Basis Pursuit
Authors:
João F. C. Mota,
João M. F. Xavier,
Pedro M. Q. Aguiar,
Markus Püschel
Abstract:
We propose a distributed algorithm for solving the optimization problem Basis Pursuit (BP). BP finds the least L1-norm solution of the underdetermined linear system Ax = b and is used, for example, in compressed sensing for reconstruction. Our algorithm solves BP on a distributed platform such as a sensor network, and is designed to minimize the communication between nodes. The algorithm only requ…
▽ More
We propose a distributed algorithm for solving the optimization problem Basis Pursuit (BP). BP finds the least L1-norm solution of the underdetermined linear system Ax = b and is used, for example, in compressed sensing for reconstruction. Our algorithm solves BP on a distributed platform such as a sensor network, and is designed to minimize the communication between nodes. The algorithm only requires the network to be connected, has no notion of a central processing node, and no node has access to the entire matrix A at any time. We consider two scenarios in which either the columns or the rows of A are distributed among the compute nodes. Our algorithm, named D-ADMM, is a decentralized implementation of the alternating direction method of multipliers. We show through numerical simulation that our algorithm requires considerably less communications between the nodes than the state-of-the-art algorithms.
△ Less
Submitted 14 March, 2012; v1 submitted 6 September, 2010;
originally announced September 2010.
-
Algebraic Signal Processing Theory: Cooley-Tukey Type Algorithms for Polynomial Transforms Based on Induction
Authors:
Aliaksei Sandryhaila,
Jelena Kovacevic,
Markus Pueschel
Abstract:
A polynomial transform is the multiplication of an input vector $x\in\C^n$ by a matrix $\PT_{b,α}\in\C^{n\times n},$ whose $(k,\ell)$-th element is defined as $p_\ell(α_k)$ for polynomials $p_\ell(x)\in\C[x]$ from a list $b=\{p_0(x),\dots,p_{n-1}(x)\}$ and sample points $α_k\in\C$ from a list $α=\{α_0,\dots,α_{n-1}\}$. Such transforms find applications in the areas of signal processing, data compr…
▽ More
A polynomial transform is the multiplication of an input vector $x\in\C^n$ by a matrix $\PT_{b,α}\in\C^{n\times n},$ whose $(k,\ell)$-th element is defined as $p_\ell(α_k)$ for polynomials $p_\ell(x)\in\C[x]$ from a list $b=\{p_0(x),\dots,p_{n-1}(x)\}$ and sample points $α_k\in\C$ from a list $α=\{α_0,\dots,α_{n-1}\}$. Such transforms find applications in the areas of signal processing, data compression, and function interpolation. Important examples include the discrete Fourier and cosine transforms. In this paper we introduce a novel technique to derive fast algorithms for polynomial transforms. The technique uses the relationship between polynomial transforms and the representation theory of polynomial algebras. Specifically, we derive algorithms by decomposing the regular modules of these algebras as a stepwise induction. As an application, we derive novel $O(n\log{n})$ general-radix algorithms for the discrete Fourier transform and the discrete cosine transform of type 4.
△ Less
Submitted 17 August, 2010;
originally announced August 2010.
-
Algebraic Signal Processing Theory: Cooley-Tukey Type Algorithms for DCTs and DSTs
Authors:
Markus Pueschel,
Jose M. F. Moura
Abstract:
This paper presents a systematic methodology based on the algebraic theory of signal processing to classify and derive fast algorithms for linear transforms. Instead of manipulating the entries of transform matrices, our approach derives the algorithms by stepwise decomposition of the associated signal models, or polynomial algebras. This decomposition is based on two generic methods or algebrai…
▽ More
This paper presents a systematic methodology based on the algebraic theory of signal processing to classify and derive fast algorithms for linear transforms. Instead of manipulating the entries of transform matrices, our approach derives the algorithms by stepwise decomposition of the associated signal models, or polynomial algebras. This decomposition is based on two generic methods or algebraic principles that generalize the well-known Cooley-Tukey FFT and make the algorithms' derivations concise and transparent. Application to the 16 discrete cosine and sine transforms yields a large class of fast algorithms, many of which have not been found before.
△ Less
Submitted 4 February, 2007;
originally announced February 2007.
-
Algebraic Signal Processing Theory
Authors:
Markus Püschel,
José M. F. Moura
Abstract:
This paper presents an algebraic theory of linear signal processing. At the core of algebraic signal processing is the concept of a linear signal model defined as a triple (A, M, phi), where familiar concepts like the filter space and the signal space are cast as an algebra A and a module M, respectively, and phi generalizes the concept of the z-transform to bijective linear map**s from a vector…
▽ More
This paper presents an algebraic theory of linear signal processing. At the core of algebraic signal processing is the concept of a linear signal model defined as a triple (A, M, phi), where familiar concepts like the filter space and the signal space are cast as an algebra A and a module M, respectively, and phi generalizes the concept of the z-transform to bijective linear map**s from a vector space of, e.g., signal samples, into the module M. A signal model provides the structure for a particular linear signal processing application, such as infinite and finite discrete time, or infinite or finite discrete space, or the various forms of multidimensional linear signal processing. As soon as a signal model is chosen, basic ingredients follow, including the associated notions of filtering, spectrum, and Fourier transform. The shift operator is a key concept in the algebraic theory: it is the generator of the algebra of filters A. Once the shift is chosen, a well-defined methodology leads to the associated signal model. Different shifts correspond to infinite and finite time models with associated infinite and finite z-transforms, and to infinite and finite space models with associated infinite and finite C-transforms (that we introduce). In particular, we show that the 16 discrete cosine and sine transforms are Fourier transforms for the finite space models. Other definitions of the shift naturally lead to new signal models and to new transforms as associated Fourier transforms in one and higher dimensions, separable and non-separable. We explain in algebraic terms shift-invariance (the algebra of filters A is commutative), the role of boundary conditions and signal extensions, the connections between linear transforms and linear finite Gauss-Markov fields, and several other concepts and connections.
△ Less
Submitted 14 November, 2019; v1 submitted 15 December, 2006;
originally announced December 2006.
-
Fast Quantum Fourier Transforms for a Class of Non-abelian Groups
Authors:
Markus Pueschel,
Martin Roetteler,
Thomas Beth
Abstract:
An algorithm is presented allowing the construction of fast Fourier transforms for any solvable group on a classical computer. The special structure of the recursion formula being the core of this algorithm makes it a good starting point to obtain systematically fast Fourier transforms for solvable groups on a quantum computer. The inherent structure of the Hilbert space imposed by the qubit arc…
▽ More
An algorithm is presented allowing the construction of fast Fourier transforms for any solvable group on a classical computer. The special structure of the recursion formula being the core of this algorithm makes it a good starting point to obtain systematically fast Fourier transforms for solvable groups on a quantum computer. The inherent structure of the Hilbert space imposed by the qubit architecture suggests to consider groups of order 2^n first (where n is the number of qubits). As an example, fast quantum Fourier transforms for all 4 classes of non-abelian 2-groups with cyclic normal subgroup of index 2 are explicitly constructed in terms of quantum circuits. The (quantum) complexity of the Fourier transform for these groups of size 2^n is O(n^2) in all cases.
△ Less
Submitted 22 July, 1998;
originally announced July 1998.
