-
Parallel Algorithms for Equilevel Predicates
Authors:
Vijay K. Garg,
Robert P. Streit
Abstract:
We define a new class of predicates called equilevel predicates on a distributive lattice which eases the analysis of parallel algorithms. Many combinatorial problems such as the vertex cover problem, the bipartite matching problem, and the minimum spanning tree problem can be modeled as detecting an equilevel predicate. The problem of detecting an equilevel problem is NP-complete, but equilevel p…
▽ More
We define a new class of predicates called equilevel predicates on a distributive lattice which eases the analysis of parallel algorithms. Many combinatorial problems such as the vertex cover problem, the bipartite matching problem, and the minimum spanning tree problem can be modeled as detecting an equilevel predicate. The problem of detecting an equilevel problem is NP-complete, but equilevel predicates with the helpful property can be detected in polynomial time in an online manner. An equilevel predicate has the helpful property with a polynomial time algorithm if the algorithm can return a nonempty set of indices such that advancing on any of them can be used to detect the predicate. Furthermore, the refined independently helpful property allows online parallel detection of such predicates in NC. When the independently helpful property holds, advancing on all the specified indices in parallel can be used to detect the predicate in polylogarithmic time.
We also define a special class of equilevel predicates called solitary predicates. Unless NP = RP, this class of predicate also does not admit efficient algorithms. Earlier work has shown that solitary predicates with the efficient advancement can be detected in polynomial time. We introduce two properties called the antimonotone advancement and the efficient rejection which yield the detection of solitary predicates in NC. Finally, we identify the minimum spanning tree, the shortest path, and the conjunctive predicate detection as problems satisfying such properties, giving alternative certifications of their NC memberships as a result.
△ Less
Submitted 10 November, 2023;
originally announced November 2023.
-
Heterophased grain boundary-rich superparamagnetic Iron Oxides/carbon composite for Cationic and Anionic Dye Removal
Authors:
K Priyananda Singh,
Boris Wareppam,
Raghavendra K G,
N. Joseph Singh,
A. C. de Oliveira,
V. K. Garg,
Subrata Ghosh,
L. Herojit Singh
Abstract:
Iron oxide-based nanostructures receive significant attention as an efficient adsorbent for organic dyes removal. The removal properties have strong dependency on the stoichiometry, phases, reactive edges, defect states etc present in the iron-oxides nanostructures. Herein, iron oxide/carbon composite with well-defined heterophased grain boundaries is synthesized by simple precipitation method and…
▽ More
Iron oxide-based nanostructures receive significant attention as an efficient adsorbent for organic dyes removal. The removal properties have strong dependency on the stoichiometry, phases, reactive edges, defect states etc present in the iron-oxides nanostructures. Herein, iron oxide/carbon composite with well-defined heterophased grain boundaries is synthesized by simple precipitation method and followed by calcination. The local structure, spin dynamics and magnetic properties of heterophased iron oxides/carbon composite are thoroughly investigated to explore its cationic and anionic dye removal capability. To validate the effectivity of the presence of heterogeneous grain boundaries, iron oxide/carbon nanocomposite with homogeneous grain boundaries is also examined. It was found that the hetero-phased iron oxide/carbon showed removal capacity of 35.45 mg g-1 and 45.84 mg g-1 for cationic (Crystal Violet) and anionic (Congo Red) dyes, respectively as compared to that of as-synthesised imidazole-capped superparamagnetic α-Fe2O3 (25.11 mg g-1 and 40.44 mg g-1, respectively) and homophased iron oxide/carbon nanocomposite (9.41 mg g-1 and 5.43 mg g-1, respectively). The plausible mechanism on the local structural evolution of the heterophase in the course of calcination and increase of the removal capacity is discussed. A detailed dye adsorption investigation is presented including the adsorption kinetic study. The pseudo-second order kinetic model is found to be an appropriate one and suggests that the chemisorption is dominant factor leading to adsorption of dyes. Whereas Weber-Morris model indicate the strong influence of boundary layers of nanocomposite on the adsorption process.
△ Less
Submitted 5 July, 2023;
originally announced July 2023.
-
Lattice Linear Predicate Algorithms for the Constrained Stable Marriage Problem with Ties
Authors:
Vijay K. Garg
Abstract:
We apply Lattice-Linear Predicate Detection Technique to derive parallel and distributed algorithms for various variants of the stable matching problem. These problems are: (a) the constrained stable marriage problem (b) the super stable marriage problem in presence of ties, and (c) the strongly stable marriage in presence of ties. All these problems are solved using the Lattice-Linear Predicate (…
▽ More
We apply Lattice-Linear Predicate Detection Technique to derive parallel and distributed algorithms for various variants of the stable matching problem. These problems are: (a) the constrained stable marriage problem (b) the super stable marriage problem in presence of ties, and (c) the strongly stable marriage in presence of ties. All these problems are solved using the Lattice-Linear Predicate (LLP) algorithm showing its generality. The constrained stable marriage problem is a version of finding the stable marriage in presence of lattice-linear constraints such as ``Peter's regret is less than that of Paul.'' For the constrained stable marriage problem, we present a distributed algorithm that takes $O(n^2)$ messages each of size $O(\log n)$ where $n$ is the number of men in the problem. Our algorithm is completely asynchronous. Our algorithms for the stable marriage problem with ties are also parallel with no synchronization.
△ Less
Submitted 2 August, 2022;
originally announced August 2022.
-
Variation in defects and properties in composite of ZnO and α-Fe2O3 for methyl blue dye removal
Authors:
Boris Wareppam,
K. Priyananda Singh,
N. Joseph Singh,
Subrata Ghosh,
Ng. Aomoa,
V. K. Garg,
A. C. Oliveira,
L. Herojit Singh
Abstract:
The plasma deposition wall coated composite of ZnO and α-Fe2O3 (ZF-W) after exposure to ~ 2000 °C, mostly considered as waste-materials and cleaned out from the deposition unit, was subjected to anneal at 300, 500 and 1000 °C to manipulate the structural properties. An evolution of defect states along with the structural changes has been identified as annealing temperature was varied. As a consequ…
▽ More
The plasma deposition wall coated composite of ZnO and α-Fe2O3 (ZF-W) after exposure to ~ 2000 °C, mostly considered as waste-materials and cleaned out from the deposition unit, was subjected to anneal at 300, 500 and 1000 °C to manipulate the structural properties. An evolution of defect states along with the structural changes has been identified as annealing temperature was varied. As a consequence, an unstable state of ZnFe2O4 was found to be stabilized at 500 °C and migration of Zn from ZnO causes the phase transformation from the α-Fe2O3 to ZnFe2O4. While implemented for methyl blue adsorption/degradation without the effect of any external sources, the degradation for ZF-W annealed at 300 °C, 500 °C and 1000 °C were 84%, 68% and 82%, respectively. Compared to annealed structures, pristine ZF-W delivered the highest methyl blue adsorption efficiency of 86%. The changes in adsorption/degradation properties have been correlated with the simultaneous evolution of defects and structural properties of ZF-W as annealed at different temperatures. The plausible mechanism on the interaction of methyl blue with the composites on the adsorption/degradation is proposed. These findings give a clear indication on the importance of defects presence in the mixed metal oxide composite to obtain high-performance degradation/adsorption properties for sustainable wastewater treatment.
△ Less
Submitted 26 April, 2022;
originally announced April 2022.
-
Minimal Envy Matchings in the Hospitals/Residents Problem with Lower Quotas
Authors:
Changyong Hu,
Vijay K. Garg
Abstract:
In the Hospitals/Residents problem, every hospital has an upper quota that limits the number of residents assigned to it. While, in some applications, each hospital also has a lower quota for the number of residents it receives. In this setting, a stable matching may not exist. Envy-freeness is introduced as a relaxation of stability that allows blocking pairs involving a resident and an empty pos…
▽ More
In the Hospitals/Residents problem, every hospital has an upper quota that limits the number of residents assigned to it. While, in some applications, each hospital also has a lower quota for the number of residents it receives. In this setting, a stable matching may not exist. Envy-freeness is introduced as a relaxation of stability that allows blocking pairs involving a resident and an empty position of a hospital. While, envy-free matching might not exist either when lower quotas are introduced. We consider the problem of finding a feasible matching that satisfies lower quotas and upper quotas and minimizes envy in terms of envy-pairs and envy-residents in the Hospitals/Resident problem with Lower Quota. We show that the problem is NP-hard with both envy measurement. We also give a simple exponential-time algorithm for the Minimum-Envy-Pair HRLQ problem.
△ Less
Submitted 29 October, 2021;
originally announced October 2021.
-
Characterization of Super-stable Matchings
Authors:
Changyong Hu,
Vijay K. Garg
Abstract:
An instance of the super-stable matching problem with incomplete lists and ties is an undirected bipartite graph $G = (A \cup B, E)$, with an adjacency list being a linearly ordered list of ties. Ties are subsets of vertices equally good for a given vertex. An edge $(x,y) \in E \backslash M$ is a blocking edge for a matching $M$ if by getting matched to each other neither of the vertices $x$ and…
▽ More
An instance of the super-stable matching problem with incomplete lists and ties is an undirected bipartite graph $G = (A \cup B, E)$, with an adjacency list being a linearly ordered list of ties. Ties are subsets of vertices equally good for a given vertex. An edge $(x,y) \in E \backslash M$ is a blocking edge for a matching $M$ if by getting matched to each other neither of the vertices $x$ and $y$ would become worse off. Thus, there is no disadvantage if the two vertices would like to match up. A matching $M$ is super-stable if there is no blocking edge with respect to $M$. It has previously been shown that super-stable matchings form a distributive lattice and the number of super-stable matchings can be exponential in the number of vertices. We give two compact representations of size $O(m)$ that can be used to construct all super-stable matchings, where $m$ denotes the number of edges in the graph. The construction of the second representation takes $O(mn)$ time, where $n$ denotes the number of vertices in the graph, and gives an explicit rotation poset similar to the rotation poset in the classical stable marriage problem. We also give a polyhedral characterisation of the set of all super-stable matchings and prove that the super-stable matching polytope is integral, thus solving an open problem stated in the book by Gusfield and Irving .
△ Less
Submitted 20 May, 2021;
originally announced May 2021.
-
A Lattice Linear Predicate Parallel Algorithm for the Dynamic Programming Problems
Authors:
Vijay K. Garg
Abstract:
It has been shown that the parallel Lattice Linear Predicate (LLP) algorithm solves many combinatorial optimization problems such as the shortest path problem, the stable marriage problem and the market clearing price problem. In this paper, we give the parallel LLP algorithm for many dynamic programming problems. In particular, we show that the LLP algorithm solves the longest subsequence problem…
▽ More
It has been shown that the parallel Lattice Linear Predicate (LLP) algorithm solves many combinatorial optimization problems such as the shortest path problem, the stable marriage problem and the market clearing price problem. In this paper, we give the parallel LLP algorithm for many dynamic programming problems. In particular, we show that the LLP algorithm solves the longest subsequence problem, the optimal binary search tree problem, and the knapsack problem. Furthermore, the algorithm can be used to solve the constrained versions of these problems so long as the constraints are lattice linear. The parallel LLP algorithm requires only read-write atomicity and no higher-level atomic instructions.
△ Less
Submitted 10 March, 2021;
originally announced March 2021.
-
Improved Paths to Stability for the Stable Marriage Problem
Authors:
Vijay Kumar Garg,
Changyong Hu
Abstract:
The stable marriage problem requires one to find a marriage with no blocking pair. Given a matching that is not stable, Roth and Vande Vate have shown that there exists a sequence of matchings that leads to a stable matching in which each successive matching is obtained by satisfying a blocking pair. The sequence produced by Roth and Vande Vate's algorithm is of length $O(n^3)$ where $n$ is the nu…
▽ More
The stable marriage problem requires one to find a marriage with no blocking pair. Given a matching that is not stable, Roth and Vande Vate have shown that there exists a sequence of matchings that leads to a stable matching in which each successive matching is obtained by satisfying a blocking pair. The sequence produced by Roth and Vande Vate's algorithm is of length $O(n^3)$ where $n$ is the number of men (and women). In this paper, we present an algorithm that achieves stability in a sequence of matchings of length $O(n^2)$. We also give an efficient algorithm to find the stable matching closest to the given initial matching under an appropriate distance function between matchings.
△ Less
Submitted 16 May, 2023; v1 submitted 14 July, 2020;
originally announced July 2020.
-
Generalization and Representational Limits of Graph Neural Networks
Authors:
Vikas K. Garg,
Stefanie Jegelka,
Tommi Jaakkola
Abstract:
We address two fundamental questions about graph neural networks (GNNs). First, we prove that several important graph properties cannot be computed by GNNs that rely entirely on local information. Such GNNs include the standard message passing models, and more powerful spatial variants that exploit local graph structure (e.g., via relative orientation of messages, or local port ordering) to distin…
▽ More
We address two fundamental questions about graph neural networks (GNNs). First, we prove that several important graph properties cannot be computed by GNNs that rely entirely on local information. Such GNNs include the standard message passing models, and more powerful spatial variants that exploit local graph structure (e.g., via relative orientation of messages, or local port ordering) to distinguish neighbors of each node. Our treatment includes a novel graph-theoretic formalism. Second, we provide the first data dependent generalization bounds for message passing GNNs. This analysis explicitly accounts for the local permutation invariance of GNNs. Our bounds are much tighter than existing VC-dimension based guarantees for GNNs, and are comparable to Rademacher bounds for recurrent neural networks.
△ Less
Submitted 14 February, 2020;
originally announced February 2020.
-
Learn to Expect the Unexpected: Probably Approximately Correct Domain Generalization
Authors:
Vikas K. Garg,
Adam Kalai,
Katrina Ligett,
Zhiwei Steven Wu
Abstract:
Domain generalization is the problem of machine learning when the training data and the test data come from different data domains. We present a simple theoretical model of learning to generalize across domains in which there is a meta-distribution over data distributions, and those data distributions may even have different supports. In our model, the training data given to a learning algorithm c…
▽ More
Domain generalization is the problem of machine learning when the training data and the test data come from different data domains. We present a simple theoretical model of learning to generalize across domains in which there is a meta-distribution over data distributions, and those data distributions may even have different supports. In our model, the training data given to a learning algorithm consists of multiple datasets each from a single domain drawn in turn from the meta-distribution. We study this model in three different problem settings---a multi-domain Massart noise setting, a decision tree multi-dataset setting, and a feature selection setting, and find that computationally efficient, polynomial-sample domain generalization is possible in each. Experiments demonstrate that our feature selection algorithm indeed ignores spurious correlations and improves generalization.
△ Less
Submitted 13 February, 2020;
originally announced February 2020.
-
A Generalization of Teo and Sethuraman's Median Stable Marriage Theorem
Authors:
Vijay K. Garg
Abstract:
Let $L$ be any finite distributive lattice and $B$ be any boolean predicate defined on $L$ such that the set of elements satisfying $B$ is a sublattice of $L$. Consider any subset $M$ of $L$ of size $k$ of elements of $L$ that satisfy $B$. Then, we show that $k$ generalized median elements generated from $M$ also satisfy $B$. We call this result generalized median theorem on finite distributive la…
▽ More
Let $L$ be any finite distributive lattice and $B$ be any boolean predicate defined on $L$ such that the set of elements satisfying $B$ is a sublattice of $L$. Consider any subset $M$ of $L$ of size $k$ of elements of $L$ that satisfy $B$. Then, we show that $k$ generalized median elements generated from $M$ also satisfy $B$. We call this result generalized median theorem on finite distributive lattices. When this result is applied to the stable matching, we get Teo and Sethuraman's median stable matching theorem. Our proof is much simpler than that of Teo and Sethuraman. When the generalized median theorem is applied to the assignment problem, we get an analogous result for market clearing price vectors.
△ Less
Submitted 9 January, 2020;
originally announced January 2020.
-
NC Algorithms for Popular Matchings in One-Sided Preference Systems and Related Problems
Authors:
Changyong Hu,
Vijay K. Garg
Abstract:
The popular matching problem is of matching a set of applicants to a set of posts, where each applicant has a preference list, ranking a non-empty subset of posts in the order of preference, possibly with ties. A matching M is popular if there is no other matching M' such that more applicants prefer M' to M. We give the first NC algorithm to solve the popular matching problem without ties. We also…
▽ More
The popular matching problem is of matching a set of applicants to a set of posts, where each applicant has a preference list, ranking a non-empty subset of posts in the order of preference, possibly with ties. A matching M is popular if there is no other matching M' such that more applicants prefer M' to M. We give the first NC algorithm to solve the popular matching problem without ties. We also give an NC algorithm that solves the maximum-cardinality popular matching problem. No NC or RNC algorithms were known for the matching problem in preference systems prior to this work. Moreover, we give an NC algorithm for a weaker version of the stable matching problem, that is, the problem of finding the "next" stable matching given a stable matching.
△ Less
Submitted 20 December, 2019; v1 submitted 23 October, 2019;
originally announced October 2019.
-
Multiresolution Transformer Networks: Recurrence is Not Essential for Modeling Hierarchical Structure
Authors:
Vikas K. Garg,
Inderjit S. Dhillon,
Hsiang-Fu Yu
Abstract:
The architecture of Transformer is based entirely on self-attention, and has been shown to outperform models that employ recurrence on sequence transduction tasks such as machine translation. The superior performance of Transformer has been attributed to propagating signals over shorter distances, between positions in the input and the output, compared to the recurrent architectures. We establish…
▽ More
The architecture of Transformer is based entirely on self-attention, and has been shown to outperform models that employ recurrence on sequence transduction tasks such as machine translation. The superior performance of Transformer has been attributed to propagating signals over shorter distances, between positions in the input and the output, compared to the recurrent architectures. We establish connections between the dynamics in Transformer and recurrent networks to argue that several factors including gradient flow along an ensemble of multiple weakly dependent paths play a paramount role in the success of Transformer. We then leverage the dynamics to introduce {\em Multiresolution Transformer Networks} as the first architecture that exploits hierarchical structure in data via self-attention. Our models significantly outperform state-of-the-art recurrent and hierarchical recurrent models on two real-world datasets for query suggestion, namely, \aol and \amazon. In particular, on AOL data, our model registers at least 20\% improvement on each precision score, and over 25\% improvement on the BLEU score with respect to the best performing recurrent model. We thus provide strong evidence that recurrence is not essential for modeling hierarchical structure.
△ Less
Submitted 27 August, 2019;
originally announced August 2019.
-
Strategic Prediction with Latent Aggregative Games
Authors:
Vikas K. Garg,
Tommi Jaakkola
Abstract:
We introduce a new class of context dependent, incomplete information games to serve as structured prediction models for settings with significant strategic interactions. Our games map the input context to outcomes by first condensing the input into private player types that specify the utilities, weighted interactions, as well as the initial strategies for the players. The game is played over mul…
▽ More
We introduce a new class of context dependent, incomplete information games to serve as structured prediction models for settings with significant strategic interactions. Our games map the input context to outcomes by first condensing the input into private player types that specify the utilities, weighted interactions, as well as the initial strategies for the players. The game is played over multiple rounds where players respond to weighted aggregates of their neighbors' strategies. The predicted output from the model is a mixed strategy profile (a near-Nash equilibrium) and each observation is thought of as a sample from this strategy profile. We introduce two new aggregator paradigms with provably convergent game dynamics, and characterize the conditions under which our games are identifiable from data. Our games can be parameterized in a transferable manner so that the sets of players can change from one game to another. We demonstrate empirically that our games as models can recover meaningful strategic interactions from real voting data.
△ Less
Submitted 28 May, 2019;
originally announced May 2019.
-
Solving graph compression via optimal transport
Authors:
Vikas K. Garg,
Tommi Jaakkola
Abstract:
We propose a new approach to graph compression by appeal to optimal transport. The transport problem is seeded with prior information about node importance, attributes, and edges in the graph. The transport formulation can be setup for either directed or undirected graphs, and its dual characterization is cast in terms of distributions over the nodes. The compression pertains to the support of nod…
▽ More
We propose a new approach to graph compression by appeal to optimal transport. The transport problem is seeded with prior information about node importance, attributes, and edges in the graph. The transport formulation can be setup for either directed or undirected graphs, and its dual characterization is cast in terms of distributions over the nodes. The compression pertains to the support of node distributions and makes the problem challenging to solve directly. To this end, we introduce Boolean relaxations and specify conditions under which these relaxations are exact. The relaxations admit algorithms with provably fast convergence. Moreover, we provide an exact $O(d \log d)$ algorithm for the subproblem of projecting a $d$-dimensional vector to transformed simplex constraints. Our method outperforms state-of-the-art compression methods on graph classification.
△ Less
Submitted 28 May, 2019;
originally announced May 2019.
-
Removing Sequential Bottleneck of Dijkstra's Algorithm for the Shortest Path Problem
Authors:
Vijay K. Garg
Abstract:
All traditional methods of computing shortest paths depend upon edge-relaxation where the cost of reaching a vertex from a source vertex is possibly decreased if that edge is used. We introduce a method which maintains lower bounds as well as upper bounds for reaching a vertex. This method enables one to find the optimal cost for multiple vertices in one iteration and thereby reduces the sequentia…
▽ More
All traditional methods of computing shortest paths depend upon edge-relaxation where the cost of reaching a vertex from a source vertex is possibly decreased if that edge is used. We introduce a method which maintains lower bounds as well as upper bounds for reaching a vertex. This method enables one to find the optimal cost for multiple vertices in one iteration and thereby reduces the sequential bottleneck in Dijkstra's algorithm.
We present four algorithms in this paper --- $SP_1$, $SP_2$, $SP_3$ and $SP_4$. $SP_1$ and $SP_2$ reduce the number of heap operations in Dijkstra's algorithm. For directed acyclic graphs, or directed unweighted graphs they have the optimal complexity of $O(e)$ where $e$ is the number of edges in the graph which is better than that of Dijkstra's algorithm. For general graphs, their worst case complexity matches that of Dijkstra's algorithm for a sequential implementation but allows for greater parallelism. Algorithms $SP_3$ and $SP_4$ allow for even more parallelism but with higher work complexity. Algorithm $SP_3$ requires $O(n + e(\max(\log n, Δ)))$ work where $n$ is the number of vertices and $Δ$ is the maximum in-degree of a node. Algorithm $SP_4$ has the most parallelism. It requires $O(ne)$ work. These algorithms generalize the work by Crauser, Mehlhorn, Meyer, and Sanders on parallelizing Dijkstra's algorithm.
△ Less
Submitted 26 December, 2018;
originally announced December 2018.
-
Applying Predicate Detection to the Constrained Optimization Problems
Authors:
Vijay K. Garg
Abstract:
We present a method to design parallel algorithms for constrained combinatorial optimization problems. Our method solves and generalizes many classical combinatorial optimization problems including the stable marriage problem, the shortest path problem and the market clearing price problem. These three problems are solved in the literature using Gale-Shapley algorithm, Dijkstra's algorithm, and De…
▽ More
We present a method to design parallel algorithms for constrained combinatorial optimization problems. Our method solves and generalizes many classical combinatorial optimization problems including the stable marriage problem, the shortest path problem and the market clearing price problem. These three problems are solved in the literature using Gale-Shapley algorithm, Dijkstra's algorithm, and Demange, Gale, Sotomayor algorithm. Our method solves all these problems by casting them as searching for an element that satisfies an appropriate predicate in a distributive lattice. Moreover, it solves generalizations of all these problems - namely finding the optimal solution satisfying additional constraints called {\em lattice-linear} predicates. For stable marriage problems, an example of such a constraint is that Peter's regret is less than that of Paul. For shortest path problems, an example of such a constraint is that cost of reaching vertex $v_1$ is at least the cost of reaching vertex $v_2$. For the market clearing price problem, an example of such a constraint is that $item_1$ is priced at least as much as $item_2$. In addition to finding the optimal solution, our method is useful in enumerating all constrained stable matchings, and all constrained market clearing price vectors.
△ Less
Submitted 26 December, 2018;
originally announced December 2018.
-
Online Markov Decoding: Lower Bounds and Near-Optimal Approximation Algorithms
Authors:
Vikas K. Garg,
Tamar Pichkhadze
Abstract:
We resolve the fundamental problem of online decoding with general $n^{th}$ order ergodic Markov chain models. Specifically, we provide deterministic and randomized algorithms whose performance is close to that of the optimal offline algorithm even when latency is small. Our algorithms admit efficient implementation via dynamic programs, and readily extend to (adversarial) non-stationary or time-v…
▽ More
We resolve the fundamental problem of online decoding with general $n^{th}$ order ergodic Markov chain models. Specifically, we provide deterministic and randomized algorithms whose performance is close to that of the optimal offline algorithm even when latency is small. Our algorithms admit efficient implementation via dynamic programs, and readily extend to (adversarial) non-stationary or time-varying settings. We also establish lower bounds for online methods under latency constraints in both deterministic and randomized settings, and show that no online algorithm can perform significantly better than our algorithms. Empirically, just with latency one, our algorithm outperforms the online step algorithm by over 30\% in terms of decoding agreement with the optimal algorithm on genome sequence data.
△ Less
Submitted 30 May, 2019; v1 submitted 16 October, 2018;
originally announced October 2018.
-
Linearizable Replicated State Machines with Lattice Agreement
Authors:
Xiong Zheng,
Vijay K. Garg,
John Kaippallimalil
Abstract:
This paper studies the lattice agreement problem in asynchronous systems and explores its application to building linearizable replicated state machines (RSM). First, we propose an algorithm to solve the lattice agreement problem in $O(\log f)$ asynchronous rounds, where $f$ is the number of crash failures that the system can tolerate. This is an exponential improvement over the previous best uppe…
▽ More
This paper studies the lattice agreement problem in asynchronous systems and explores its application to building linearizable replicated state machines (RSM). First, we propose an algorithm to solve the lattice agreement problem in $O(\log f)$ asynchronous rounds, where $f$ is the number of crash failures that the system can tolerate. This is an exponential improvement over the previous best upper bound. Second, Faleiro et al have shown in [Faleiro et al. PODC, 2012] that combination of conflict-free data types and lattice agreement protocols can be applied to implement linearizable RSM. They give a Paxos style lattice agreement protocol, which can be adapted to implement linearizable RSM and guarantee that a command can be learned in at most $O(n)$ message delays, where $n$ is the number of proposers. Later on, Xiong et al in [Xiong et al. DISC, 2018] give a lattice agreement protocol which improves the $O(n)$ guarantee to be $O(f)$. However, neither protocols is practical for building a linearizable RSM. Thus, in the second part of the paper, we first give an improved protocol based on the one proposed by Xiong et al. Then, we implement a simple linearizable RSM using the our improved protocol and compare our implementation with an open source Java implementation of Paxos. Results show that better performance can be obtained by using lattice agreement based protocols to implement a linearizable RSM compared to traditional consensus based protocols.
△ Less
Submitted 13 October, 2018;
originally announced October 2018.
-
Lattice Agreement in Message Passing Systems
Authors:
Xiong Zheng,
Changyong Hu,
Vijay K. Garg
Abstract:
This paper studies the lattice agreement problem and the generalized lattice agreement problem in distributed message passing systems. In the lattice agreement problem, given input values from a lattice, processes have to non-trivially decide output values that lie on a chain. We consider the lattice agreement problem in both synchronous and asynchronous systems. For synchronous lattice agreement,…
▽ More
This paper studies the lattice agreement problem and the generalized lattice agreement problem in distributed message passing systems. In the lattice agreement problem, given input values from a lattice, processes have to non-trivially decide output values that lie on a chain. We consider the lattice agreement problem in both synchronous and asynchronous systems. For synchronous lattice agreement, we present two algorithms which run in $\log f$ and $\min \{O(\log^2 h(L)), O(\log^2 f)\}$ rounds, respectively, where $h(L)$ denotes the height of the {\em input sublattice} $L$, $f < n$ is the number of crash failures the system can tolerate, and $n$ is the number of processes in the system. These algorithms have significant better round complexity than previously known algorithms. The algorithm by Attiya et al. \cite{attiya1995atomic} takes $\log n$ synchronous rounds, and the algorithm by Mavronicolasa \cite{mavronicolasabound} takes $\min \{O(h(L)), O(\sqrt{f})\}$ rounds. For asynchronous lattice agreement, we propose an algorithm which has time complexity of $2 \cdot \min \{h(L), f + 1\}$ message delays which improves on the previously known time complexity of $O(n)$ message delays.
The generalized lattice agreement problem defined by Faleiro et al in \cite{faleiro2012generalized} is a generalization of the lattice agreement problem where it is applied for the replicated state machine. We propose an algorithm which guarantees liveness when a majority of the processes are correct in asynchronous systems. Our algorithm requires $\min \{O(h(L)), O(f)\}$ units of time in the worst case which is better than $O(n)$ units of time required by the algorithm of Faleiro et al. \cite{faleiro2012generalized}.
△ Less
Submitted 30 July, 2018;
originally announced July 2018.
-
Learning SMaLL Predictors
Authors:
Vikas K. Garg,
Ofer Dekel,
Lin Xiao
Abstract:
We present a new machine learning technique for training small resource-constrained predictors. Our algorithm, the Sparse Multiprototype Linear Learner (SMaLL), is inspired by the classic machine learning problem of learning $k$-DNF Boolean formulae. We present a formal derivation of our algorithm and demonstrate the benefits of our approach with a detailed empirical study.
We present a new machine learning technique for training small resource-constrained predictors. Our algorithm, the Sparse Multiprototype Linear Learner (SMaLL), is inspired by the classic machine learning problem of learning $k$-DNF Boolean formulae. We present a formal derivation of our algorithm and demonstrate the benefits of our approach with a detailed empirical study.
△ Less
Submitted 6 March, 2018;
originally announced March 2018.
-
Supervising Unsupervised Learning
Authors:
Vikas K. Garg,
Adam Kalai
Abstract:
We introduce a framework to leverage knowledge acquired from a repository of (heterogeneous) supervised datasets to new unsupervised datasets. Our perspective avoids the subjectivity inherent in unsupervised learning by reducing it to supervised learning, and provides a principled way to evaluate unsupervised algorithms. We demonstrate the versatility of our framework via simple agnostic bounds on…
▽ More
We introduce a framework to leverage knowledge acquired from a repository of (heterogeneous) supervised datasets to new unsupervised datasets. Our perspective avoids the subjectivity inherent in unsupervised learning by reducing it to supervised learning, and provides a principled way to evaluate unsupervised algorithms. We demonstrate the versatility of our framework via simple agnostic bounds on unsupervised problems. In the context of clustering, our approach helps choose the number of clusters and the clustering algorithm, remove the outliers, and provably circumvent the Kleinberg's impossibility result. Experimental results across hundreds of problems demonstrate improved performance on unsupervised data with simple algorithms, despite the fact that our problems come from heterogeneous domains. Additionally, our framework lets us leverage deep networks to learn common features from many such small datasets, and perform zero shot learning.
△ Less
Submitted 16 February, 2018; v1 submitted 14 September, 2017;
originally announced September 2017.
-
Meta-Unsupervised-Learning: A supervised approach to unsupervised learning
Authors:
Vikas K. Garg,
Adam Tauman Kalai
Abstract:
We introduce a new paradigm to investigate unsupervised learning, reducing unsupervised learning to supervised learning. Specifically, we mitigate the subjectivity in unsupervised decision-making by leveraging knowledge acquired from prior, possibly heterogeneous, supervised learning tasks. We demonstrate the versatility of our framework via comprehensive expositions and detailed experiments on se…
▽ More
We introduce a new paradigm to investigate unsupervised learning, reducing unsupervised learning to supervised learning. Specifically, we mitigate the subjectivity in unsupervised decision-making by leveraging knowledge acquired from prior, possibly heterogeneous, supervised learning tasks. We demonstrate the versatility of our framework via comprehensive expositions and detailed experiments on several unsupervised problems such as (a) clustering, (b) outlier detection, and (c) similarity prediction under a common umbrella of meta-unsupervised-learning. We also provide rigorous PAC-agnostic bounds to establish the theoretical foundations of our framework, and show that our framing of meta-clustering circumvents Kleinberg's impossibility theorem for clustering.
△ Less
Submitted 3 January, 2017; v1 submitted 28 December, 2016;
originally announced December 2016.
-
CRAFT: ClusteR-specific Assorted Feature selecTion
Authors:
Vikas K. Garg,
Cynthia Rudin,
Tommi Jaakkola
Abstract:
We present a framework for clustering with cluster-specific feature selection. The framework, CRAFT, is derived from asymptotic log posterior formulations of nonparametric MAP-based clustering models. CRAFT handles assorted data, i.e., both numeric and categorical data, and the underlying objective functions are intuitively appealing. The resulting algorithm is simple to implement and scales nicel…
▽ More
We present a framework for clustering with cluster-specific feature selection. The framework, CRAFT, is derived from asymptotic log posterior formulations of nonparametric MAP-based clustering models. CRAFT handles assorted data, i.e., both numeric and categorical data, and the underlying objective functions are intuitively appealing. The resulting algorithm is simple to implement and scales nicely, requires minimal parameter tuning, obviates the need to specify the number of clusters a priori, and compares favorably with other methods on real datasets.
△ Less
Submitted 25 June, 2015;
originally announced June 2015.
-
DEEP-CARVING: Discovering Visual Attributes by Carving Deep Neural Nets
Authors:
Sukrit Shankar,
Vikas K. Garg,
Roberto Cipolla
Abstract:
Most of the approaches for discovering visual attributes in images demand significant supervision, which is cumbersome to obtain. In this paper, we aim to discover visual attributes in a weakly supervised setting that is commonly encountered with contemporary image search engines. Deep Convolutional Neural Networks (CNNs) have enjoyed remarkable success in vision applications recently. However, in…
▽ More
Most of the approaches for discovering visual attributes in images demand significant supervision, which is cumbersome to obtain. In this paper, we aim to discover visual attributes in a weakly supervised setting that is commonly encountered with contemporary image search engines. Deep Convolutional Neural Networks (CNNs) have enjoyed remarkable success in vision applications recently. However, in a weakly supervised scenario, widely used CNN training procedures do not learn a robust model for predicting multiple attribute labels simultaneously. The primary reason is that the attributes highly co-occur within the training data. To ameliorate this limitation, we propose Deep-Carving, a novel training procedure with CNNs, that helps the net efficiently carve itself for the task of multiple attribute prediction. During training, the responses of the feature maps are exploited in an ingenious way to provide the net with multiple pseudo-labels (for training images) for subsequent iterations. The process is repeated periodically after a fixed number of iterations, and enables the net carve itself iteratively for efficiently disentangling features. Additionally, we contribute a noun-adjective pairing inspired Natural Scenes Attributes Dataset to the research community, CAMIT - NSAD, containing a number of co-occurring attributes within a noun category. We describe, in detail, salient aspects of this dataset. Our experiments on CAMIT-NSAD and the SUN Attributes Dataset, with weak supervision, clearly demonstrate that the Deep-Carved CNNs consistently achieve considerable improvement in the precision of attribute prediction over popular baseline methods.
△ Less
Submitted 19 April, 2015;
originally announced April 2015.
-
Necessary and Sufficient Conditions on Partial Orders for Modeling Concurrent Computations
Authors:
Himanshu Chauhan,
Vijay K. Garg
Abstract:
Partial orders are used extensively for modeling and analyzing concurrent computations. In this paper, we define two properties of partially ordered sets: width-extensibility and interleaving-consistency, and show that a partial order can be a valid state based model: (1) of some synchronous concurrent computation iff it is width-extensible, and (2) of some asynchronous concurrent computation iff…
▽ More
Partial orders are used extensively for modeling and analyzing concurrent computations. In this paper, we define two properties of partially ordered sets: width-extensibility and interleaving-consistency, and show that a partial order can be a valid state based model: (1) of some synchronous concurrent computation iff it is width-extensible, and (2) of some asynchronous concurrent computation iff it is width-extensible and interleaving-consistent. We also show a duality between the event based and state based models of concurrent computations, and give algorithms to convert models between the two domains. When applied to the problem of checkpointing, our theory leads to a better understanding of some existing results and algorithms in the field. It also leads to efficient detection algorithms for predicates whose evaluation requires knowledge of states from all the processes in the system.
△ Less
Submitted 5 October, 2014;
originally announced October 2014.
-
ActiveMonitor: Non-blocking Monitor Executions for Increased Parallelism
Authors:
Weil-Lun Hung,
Himanshu Chauhan,
Vijay K. Garg
Abstract:
We present a set of novel ideas on design and implementation of monitor objects for multi-threaded programs. Our approach has two main goals: (a) increase parallelism in monitor objects and thus provide performance gains (shorter runtimes) for multi-threaded programs, and (b) introduce constructs that allow programmers to easily write monitor-based multi-threaded programs that can achieve these pe…
▽ More
We present a set of novel ideas on design and implementation of monitor objects for multi-threaded programs. Our approach has two main goals: (a) increase parallelism in monitor objects and thus provide performance gains (shorter runtimes) for multi-threaded programs, and (b) introduce constructs that allow programmers to easily write monitor-based multi-threaded programs that can achieve these performance gains. We describe the concepts of our framework, called ActiveMonitor, and its prototype implementation using futures. We evaluate its performance in terms of runtimes of multi-threaded programs on linked-list, bounded-buffer, and other fundamental problems implemented in Java. We compare the runtimes of our implementation against implementations using Java's reentrant locks, recently proposed automatic signaling framework AutoSynch, and some other techniques from the literature. The results of of the evaluation indicate that monitors based on our framework provide significant gains in runtime performance in comparison to traditional monitors implemented using Java's reentrant locks.
△ Less
Submitted 4 August, 2014;
originally announced August 2014.
-
Distributed Abstraction Algorithm for Online Predicate Detection
Authors:
Himanshu Chauhan,
Vijay K. Garg,
Aravind Natarajan,
Neeraj Mittal
Abstract:
Analyzing a distributed computation is a hard problem in general due to the combinatorial explosion in the size of the state-space with the number of processes in the system. By abstracting the computation, unnecessary explorations can be avoided. Computation slicing is an approach for abstracting dis- tributed computations with respect to a given predicate. We focus on regular predicates, a famil…
▽ More
Analyzing a distributed computation is a hard problem in general due to the combinatorial explosion in the size of the state-space with the number of processes in the system. By abstracting the computation, unnecessary explorations can be avoided. Computation slicing is an approach for abstracting dis- tributed computations with respect to a given predicate. We focus on regular predicates, a family of predicates that covers a large number of commonly used predicates for runtime verification. The existing algorithms for computation slicing are centralized in nature in which a single process is responsible for computing the slice in either offline or online manner. In this paper, we present a distributed online algorithm for computing the slice of a distributed computation with respect to a regular predicate. Our algorithm distributes the work and storage requirements across the system, thus reducing the space and computation complexities per process. In addition, for conjunctive predicates, our algorithm also reduces the message load per process.
△ Less
Submitted 4 June, 2013; v1 submitted 15 April, 2013;
originally announced April 2013.
-
Fault Tolerance in Distributed Systems using Fused State Machines
Authors:
Bharath Balasubramanian,
Vijay K. Garg
Abstract:
Replication is a standard technique for fault tolerance in distributed systems modeled as deterministic finite state machines (DFSMs or machines). To correct f crash or f/2 Byzantine faults among n different machines, replication requires nf additional backup machines. We present a solution called fusion that requires just f additional backup machines. First, we build a framework for fault toleran…
▽ More
Replication is a standard technique for fault tolerance in distributed systems modeled as deterministic finite state machines (DFSMs or machines). To correct f crash or f/2 Byzantine faults among n different machines, replication requires nf additional backup machines. We present a solution called fusion that requires just f additional backup machines. First, we build a framework for fault tolerance in DFSMs based on the notion of Hamming distances. We introduce the concept of an (f,m)-fusion, which is a set of m backup machines that can correct f crash faults or f/2 Byzantine faults among a given set of machines. Second, we present an algorithm to generate an (f,f)-fusion for a given set of machines. We ensure that our backups are efficient in terms of the size of their state and event sets. Our evaluation of fusion on the widely used MCNC'91 benchmarks for DFSMs show that the average state space savings in fusion (over replication) is 38% (range 0-99%). To demonstrate the practical use of fusion, we describe its potential application to the MapReduce framework. Using a simple case study, we compare replication and fusion as applied to this framework. While a pure replication-based solution requires 1.8 million map tasks, our fusion-based solution requires only 1.4 million map tasks with minimal overhead during normal operation or recovery. Hence, fusion results in considerable savings in state space and other resources such as the power needed to run the backup tasks.
△ Less
Submitted 23 March, 2013;
originally announced March 2013.
-
AutoSynch: An Automatic-Signal Monitor Based on Predicate Tagging
Authors:
Wei-Lun Hung,
Vijay K. Garg
Abstract:
Most programming languages use monitors with explicit signals for synchronization in shared-memory programs. Requiring program- mers to signal threads explicitly results in many concurrency bugs due to missed notifications, or notifications on wrong condition variables. In this paper, we describe an implementation of an au- tomatic signaling monitor in Java called AutoSynch that eliminates such co…
▽ More
Most programming languages use monitors with explicit signals for synchronization in shared-memory programs. Requiring program- mers to signal threads explicitly results in many concurrency bugs due to missed notifications, or notifications on wrong condition variables. In this paper, we describe an implementation of an au- tomatic signaling monitor in Java called AutoSynch that eliminates such concurrency bugs by removing the burden of signaling from the programmer. We show that the belief that automatic signaling monitors are prohibitively expensive is wrong. For most problems, programs based on AutoSynch are almost as fast as those based on explicit signaling. For some, AutoSynch is even faster than explicit signaling because it never uses signalAll, whereas the programmers end up using signalAll with the explicit signal mechanism. AutoSynch achieves efficiency in synchronization based on three novel ideas. We introduce an operation called globalization that enables the predicate evaluation in every thread, thereby reducing context switches during the execution of the program. Secondly, AutoSynch avoids signalAll by using a property called relay invari- ance that guarantees that whenever possible there is always at least one thread whose condition is true which has been signaled. Finally, AutoSynch uses a technique called predicate tagging to efficiently determine a thread that should be signaled. To evaluate the effi- ciency of AutoSynch, we have implemented many different well- known synchronization problems such as the producers/consumers problem, the readers/writers problems, and the dining philosophers problem. The results show that AutoSynch is almost as efficient as the explicit-signal monitor and even more efficient for some cases.
△ Less
Submitted 1 March, 2013;
originally announced March 2013.
-
Byzantine Vector Consensus in Complete Graphs
Authors:
Nitin H. Vaidya,
Vijay K. Garg
Abstract:
Consider a network of n processes each of which has a d-dimensional vector of reals as its input. Each process can communicate directly with all the processes in the system; thus the communication network is a complete graph. All the communication channels are reliable and FIFO (first-in-first-out). The problem of Byzantine vector consensus (BVC) requires agreement on a d-dimensional vector that i…
▽ More
Consider a network of n processes each of which has a d-dimensional vector of reals as its input. Each process can communicate directly with all the processes in the system; thus the communication network is a complete graph. All the communication channels are reliable and FIFO (first-in-first-out). The problem of Byzantine vector consensus (BVC) requires agreement on a d-dimensional vector that is in the convex hull of the d-dimensional input vectors at the non-faulty processes. We obtain the following results for Byzantine vector consensus in complete graphs while tolerating up to f Byzantine failures:
* We prove that in a synchronous system, n >= max(3f+1, (d+1)f+1) is necessary and sufficient for achieving Byzantine vector consensus.
* In an asynchronous system, it is known that exact consensus is impossible in presence of faulty processes. For an asynchronous system, we prove that n >= (d+2)f+1 is necessary and sufficient to achieve approximate Byzantine vector consensus.
Our sufficiency proofs are constructive. We show sufficiency by providing explicit algorithms that solve exact BVC in synchronous systems, and approximate BVC in asynchronous systems.
We also obtain tight bounds on the number of processes for achieving BVC using algorithms that are restricted to a simpler communication pattern.
△ Less
Submitted 11 February, 2013;
originally announced February 2013.
-
Techniques and Applications of Computation Slicing
Authors:
Neeraj Mittal,
Vijay K. Garg
Abstract:
Writing correct distributed programs is hard. In spite of extensive testing and debugging, software faults persist even in commercial grade software. Many distributed systems, especially those employed in safety-critical environments, should be able to operate properly even in the presence of software faults. Monitoring the execution of a distributed system, and, on detecting a fault, initiating…
▽ More
Writing correct distributed programs is hard. In spite of extensive testing and debugging, software faults persist even in commercial grade software. Many distributed systems, especially those employed in safety-critical environments, should be able to operate properly even in the presence of software faults. Monitoring the execution of a distributed system, and, on detecting a fault, initiating the appropriate corrective action is an important way to tolerate such faults. This gives rise to the predicate detection problem which requires finding a consistent cut of a given computation that satisfies a given global predicate, if it exists.
Detecting a predicate in a computation is, however, an NP-complete problem. To ameliorate the associated combinatorial explosion problem, we introduce the notion of computation slice. Formally, the slice of a computation with respect to a predicate is a (sub)computation with the least number of consistent cuts that contains all consistent cuts of the computation satisfying the predicate. To detect a predicate, rather than searching the state-space of the computation, it is much more efficient to search the state-space of the slice.
We prove that the slice exists and is uniquely defined for all predicates. We present efficient slicing algorithms for several useful classes of predicates. We develop efficient heuristic algorithms for computing an approximate slice for predicates for which computing the slice is otherwise provably intractable. Our experimental results show that slicing can lead to an exponential improvement over existing techniques for predicate detection in terms of time and space.
△ Less
Submitted 15 March, 2003;
originally announced March 2003.
