-
Distributed Optimization With Local Domains: Applications in MPC and Network Flows
Authors:
João F. C. Mota,
João M. F. Xavier,
Pedro M. Q. Aguiar,
Markus Püschel
Abstract:
In this paper we consider a network with $P$ nodes, where each node has exclusive access to a local cost function. Our contribution is a communication-efficient distributed algorithm that finds a vector $x^\star$ minimizing the sum of all the functions. We make the additional assumption that the functions have intersecting local domains, i.e., each function depends only on some components of the v…
▽ More
In this paper we consider a network with $P$ nodes, where each node has exclusive access to a local cost function. Our contribution is a communication-efficient distributed algorithm that finds a vector $x^\star$ minimizing the sum of all the functions. We make the additional assumption that the functions have intersecting local domains, i.e., each function depends only on some components of the variable. Consequently, each node is interested in knowing only some components of $x^\star$, not the entire vector. This allows for improvement in communication-efficiency. We apply our algorithm to model predictive control (MPC) and to network flow problems and show, through experiments on large networks, that our proposed algorithm requires less communications to converge than prior algorithms.
△ Less
Submitted 8 May, 2013;
originally announced May 2013.
-
D-ADMM: A Communication-Efficient Distributed Algorithm For Separable Optimization
Authors:
João F. C. Mota,
João M. F. Xavier,
Pedro M. Q. Aguiar,
Markus Püschel
Abstract:
We propose a distributed algorithm, named Distributed Alternating Direction Method of Multipliers (D-ADMM), for solving separable optimization problems in networks of interconnected nodes or agents. In a separable optimization problem there is a private cost function and a private constraint set at each node. The goal is to minimize the sum of all the cost functions, constraining the solution to b…
▽ More
We propose a distributed algorithm, named Distributed Alternating Direction Method of Multipliers (D-ADMM), for solving separable optimization problems in networks of interconnected nodes or agents. In a separable optimization problem there is a private cost function and a private constraint set at each node. The goal is to minimize the sum of all the cost functions, constraining the solution to be in the intersection of all the constraint sets. D-ADMM is proven to converge when the network is bipartite or when all the functions are strongly convex, although in practice, convergence is observed even when these conditions are not met. We use D-ADMM to solve the following problems from signal processing and control: average consensus, compressed sensing, and support vector machines. Our simulations show that D-ADMM requires less communications than state-of-the-art algorithms to achieve a given accuracy level. Algorithms with low communication requirements are important, for example, in sensor networks, where sensors are typically battery-operated and communicating is the most energy consuming operation.
△ Less
Submitted 22 April, 2013; v1 submitted 13 February, 2012;
originally announced February 2012.
-
A Proof of Convergence For the Alternating Direction Method of Multipliers Applied to Polyhedral-Constrained Functions
Authors:
João F. C. Mota,
João M. F. Xavier,
Pedro M. Q. Aguiar,
Markus Püschel
Abstract:
We give a general proof of convergence for the Alternating Direction Method of Multipliers (ADMM). ADMM is an optimization algorithm that has recently become very popular due to its capabilities to solve large-scale and/or distributed problems. We prove that the sequence generated by ADMM converges to an optimal primal-dual optimal solution. We assume the functions f and g, defining the cost f(x)…
▽ More
We give a general proof of convergence for the Alternating Direction Method of Multipliers (ADMM). ADMM is an optimization algorithm that has recently become very popular due to its capabilities to solve large-scale and/or distributed problems. We prove that the sequence generated by ADMM converges to an optimal primal-dual optimal solution. We assume the functions f and g, defining the cost f(x) + g(y), are real-valued, but constrained to lie on polyhedral sets X and Y. Our proof is an extension of the proofs from [Bertsekas97, Boyd11].
△ Less
Submitted 10 December, 2011;
originally announced December 2011.
-
ANSIG - An Analytic Signature for Arbitrary 2D Shapes (or Bags of Unlabeled Points)
Authors:
José J. Rodrigues,
João M. F. Xavier,
Pedro M. Q. Aguiar
Abstract:
In image analysis, many tasks require representing two-dimensional (2D) shape, often specified by a set of 2D points, for comparison purposes. The challenge of the representation is that it must not only capture the characteristics of the shape but also be invariant to relevant transformations. Invariance to geometric transformations, such as translation, rotation, and scale, has received attentio…
▽ More
In image analysis, many tasks require representing two-dimensional (2D) shape, often specified by a set of 2D points, for comparison purposes. The challenge of the representation is that it must not only capture the characteristics of the shape but also be invariant to relevant transformations. Invariance to geometric transformations, such as translation, rotation, and scale, has received attention in the past, usually under the assumption that the points are previously labeled, i.e., that the shape is characterized by an ordered set of landmarks. However, in many practical scenarios, the points describing the shape are obtained from automatic processes, e.g., edge or corner detection, thus without labels or natural ordering. Obviously, the combinatorial problem of computing the correspondences between the points of two shapes in the presence of the aforementioned geometrical distortions becomes a quagmire when the number of points is large. We circumvent this problem by representing shapes in a way that is invariant to the permutation of the landmarks, i.e., we represent bags of unlabeled 2D points. Within our framework, a shape is mapped to an analytic function on the complex plane, leading to what we call its analytic signature (ANSIG). To store an ANSIG, it suffices to sample it along a closed contour in the complex plane. We show that the ANSIG is a maximal invariant with respect to the permutation group, i.e., that different shapes have different ANSIGs and shapes that differ by a permutation (or re-labeling) of the landmarks have the same ANSIG. We further show how easy it is to factor out geometric transformations when comparing shapes using the ANSIG representation. Finally, we illustrate these capabilities with shape-based image classification experiments.
△ Less
Submitted 19 October, 2010;
originally announced October 2010.
-
Distributed Basis Pursuit
Authors:
João F. C. Mota,
João M. F. Xavier,
Pedro M. Q. Aguiar,
Markus Püschel
Abstract:
We propose a distributed algorithm for solving the optimization problem Basis Pursuit (BP). BP finds the least L1-norm solution of the underdetermined linear system Ax = b and is used, for example, in compressed sensing for reconstruction. Our algorithm solves BP on a distributed platform such as a sensor network, and is designed to minimize the communication between nodes. The algorithm only requ…
▽ More
We propose a distributed algorithm for solving the optimization problem Basis Pursuit (BP). BP finds the least L1-norm solution of the underdetermined linear system Ax = b and is used, for example, in compressed sensing for reconstruction. Our algorithm solves BP on a distributed platform such as a sensor network, and is designed to minimize the communication between nodes. The algorithm only requires the network to be connected, has no notion of a central processing node, and no node has access to the entire matrix A at any time. We consider two scenarios in which either the columns or the rows of A are distributed among the compute nodes. Our algorithm, named D-ADMM, is a decentralized implementation of the alternating direction method of multipliers. We show through numerical simulation that our algorithm requires considerably less communications between the nodes than the state-of-the-art algorithms.
△ Less
Submitted 14 March, 2012; v1 submitted 6 September, 2010;
originally announced September 2010.
