-
Ranking by Lifts: A Cost-Benefit Approach to Large-Scale A/B Tests
Authors:
Pallavi Basu,
Ron Berman
Abstract:
A/B testers conducting large-scale tests prioritize lifts and want to be able to control false rejections of the null. This work develops a decision-theoretic framework for maximizing profits subject to false discovery rate (FDR) control. We build an empirical Bayes solution for the problem via the greedy knapsack approach. We derive an oracle rule based on ranking the ratio of expected lifts and…
▽ More
A/B testers conducting large-scale tests prioritize lifts and want to be able to control false rejections of the null. This work develops a decision-theoretic framework for maximizing profits subject to false discovery rate (FDR) control. We build an empirical Bayes solution for the problem via the greedy knapsack approach. We derive an oracle rule based on ranking the ratio of expected lifts and the cost of wrong rejections using the local false discovery rate (lfdr) statistic. Our oracle decision rule is valid and optimal for large-scale tests. Further, we establish asymptotic validity for the data-driven procedure and demonstrate finite-sample validity in experimental studies. We also demonstrate the merit of the proposed method over other FDR control methods. Finally, we discuss an application to actual Optimizely experiments.
△ Less
Submitted 1 July, 2024;
originally announced July 2024.
-
RankAug: Augmented data ranking for text classification
Authors:
Tiasa Singha Roy,
Priyam Basu
Abstract:
Research on data generation and augmentation has been focused majorly on enhancing generation models, leaving a notable gap in the exploration and refinement of methods for evaluating synthetic data. There are several text similarity metrics within the context of generated data filtering which can impact the performance of specific Natural Language Understanding (NLU) tasks, specifically focusing…
▽ More
Research on data generation and augmentation has been focused majorly on enhancing generation models, leaving a notable gap in the exploration and refinement of methods for evaluating synthetic data. There are several text similarity metrics within the context of generated data filtering which can impact the performance of specific Natural Language Understanding (NLU) tasks, specifically focusing on intent and sentiment classification. In this study, we propose RankAug, a text-ranking approach that detects and filters out the top augmented texts in terms of being most similar in meaning with lexical and syntactical diversity. Through experiments conducted on multiple datasets, we demonstrate that the judicious selection of filtering techniques can yield a substantial improvement of up to 35% in classification accuracy for under-represented classes.
△ Less
Submitted 8 November, 2023;
originally announced November 2023.
-
Efficient All-to-All Collective Communication Schedules for Direct-Connect Topologies
Authors:
Prithwish Basu,
Liangyu Zhao,
Jason Fantl,
Siddharth Pal,
Arvind Krishnamurthy,
Joud Khoury
Abstract:
The all-to-all collective communications primitive is widely used in machine learning (ML) and high performance computing (HPC) workloads, and optimizing its performance is of interest to both ML and HPC communities. All-to-all is a particularly challenging workload that can severely strain the underlying interconnect bandwidth at scale. This paper takes a holistic approach to optimize the perform…
▽ More
The all-to-all collective communications primitive is widely used in machine learning (ML) and high performance computing (HPC) workloads, and optimizing its performance is of interest to both ML and HPC communities. All-to-all is a particularly challenging workload that can severely strain the underlying interconnect bandwidth at scale. This paper takes a holistic approach to optimize the performance of all-to-all collective communications on supercomputer-scale direct-connect interconnects. We address several algorithmic and practical challenges in develo** efficient and bandwidth-optimal all-to-all schedules for any topology and lowering the schedules to various runtimes and interconnect technologies. We also propose a novel topology that delivers near-optimal all-to-all performance.
△ Less
Submitted 25 April, 2024; v1 submitted 23 September, 2023;
originally announced September 2023.
-
Interpretability of Fine-grained Classification of Sadness and Depression
Authors:
Tiasa Singha Roy,
Priyam Basu,
Aman Priyanshu,
Rakshit Naidu
Abstract:
While sadness is a human emotion that people experience at certain times throughout their lives, inflicting them with emotional disappointment and pain, depression is a longer term mental illness which impairs social, occupational, and other vital regions of functioning making it a much more serious issue and needs to be catered to at the earliest. NLP techniques can be utilized for the detection…
▽ More
While sadness is a human emotion that people experience at certain times throughout their lives, inflicting them with emotional disappointment and pain, depression is a longer term mental illness which impairs social, occupational, and other vital regions of functioning making it a much more serious issue and needs to be catered to at the earliest. NLP techniques can be utilized for the detection and subsequent diagnosis of these emotions. Most of the open sourced data on the web deal with sadness as a part of depression, as an emotion even though the difference in severity of both is huge. Thus, we create our own novel dataset illustrating the difference between the two. In this paper, we aim to highlight the difference between the two and highlight how interpretable our models are to distinctly label sadness and depression. Due to the sensitive nature of such information, privacy measures need to be taken for handling and training of such data. Hence, we also explore the effect of Federated Learning (FL) on contextualised language models.
△ Less
Submitted 19 March, 2022;
originally announced March 2022.
-
Efficient Direct-Connect Topologies for Collective Communications
Authors:
Liangyu Zhao,
Siddharth Pal,
Tapan Chugh,
Weiyang Wang,
Jason Fantl,
Prithwish Basu,
Joud Khoury,
Arvind Krishnamurthy
Abstract:
We consider the problem of distilling efficient network topologies for collective communications. We provide an algorithmic framework for constructing direct-connect topologies optimized for the latency vs. bandwidth trade-off associated with the workload. Our approach synthesizes many different topologies and schedules for a given cluster size and degree and then identifies the appropriate topolo…
▽ More
We consider the problem of distilling efficient network topologies for collective communications. We provide an algorithmic framework for constructing direct-connect topologies optimized for the latency vs. bandwidth trade-off associated with the workload. Our approach synthesizes many different topologies and schedules for a given cluster size and degree and then identifies the appropriate topology and schedule for a given workload. Our algorithms start from small, optimal base topologies and associated communication schedules and use techniques that can be iteratively applied to derive much larger topologies and schedules. Additionally, we incorporate well-studied large-scale graph topologies into our algorithmic framework by producing efficient collective schedules for them using a novel polynomial-time algorithm. Our evaluation uses multiple testbeds and large-scale simulations to demonstrate significant performance benefits from our derived topologies and schedules.
△ Less
Submitted 12 May, 2024; v1 submitted 7 February, 2022;
originally announced February 2022.
-
Privacy enabled Financial Text Classification using Differential Privacy and Federated Learning
Authors:
Priyam Basu,
Tiasa Singha Roy,
Rakshit Naidu,
Zumrut Muftuoglu
Abstract:
Privacy is important considering the financial Domain as such data is highly confidential and sensitive. Natural Language Processing (NLP) techniques can be applied for text classification and entity detection purposes in financial domains such as customer feedback sentiment analysis, invoice entity detection, categorisation of financial documents by type etc. Due to the sensitive nature of such d…
▽ More
Privacy is important considering the financial Domain as such data is highly confidential and sensitive. Natural Language Processing (NLP) techniques can be applied for text classification and entity detection purposes in financial domains such as customer feedback sentiment analysis, invoice entity detection, categorisation of financial documents by type etc. Due to the sensitive nature of such data, privacy measures need to be taken for handling and training large models with such data. In this work, we propose a contextualized transformer (BERT and RoBERTa) based text classification model integrated with privacy features such as Differential Privacy (DP) and Federated Learning (FL). We present how to privately train NLP models and desirable privacy-utility tradeoffs and evaluate them on the Financial Phrase Bank dataset.
△ Less
Submitted 4 October, 2021;
originally announced October 2021.
-
On Characterization of Finite Geometric Distributive Lattices
Authors:
Pranab Basu
Abstract:
A Lattice is a partially ordered set where both least upper bound and greatest lower bound of any pair of elements are unique and exist within the set. Kötter and Kschischang proved that codes in the linear lattice can be used for error and erasure-correction in random networks. Codes in the linear lattice have previously been shown to be special cases of codes in modular lattices. Two well known…
▽ More
A Lattice is a partially ordered set where both least upper bound and greatest lower bound of any pair of elements are unique and exist within the set. Kötter and Kschischang proved that codes in the linear lattice can be used for error and erasure-correction in random networks. Codes in the linear lattice have previously been shown to be special cases of codes in modular lattices. Two well known classifications of semimodular lattices are geometric and distributive lattices. Most of the frequently used coding spaces are examples of either or both. We have identified the unique criterion which makes a geometric lattice distributive, thus characterizing all finite geometric distributive lattices. Our characterization helps to prove a conjecture regarding the maximum size of a distributive sublattice of a finite geometric lattice and identify the maximal case. The Whitney numbers of the class of geometric distributive lattices are also calculated. We present a few other applications of this unique characterization to derive certain results regarding linearity and complements in the linear lattice.
△ Less
Submitted 29 September, 2021; v1 submitted 15 September, 2021;
originally announced September 2021.
-
Probabilistic Verification for Reliability of a Two-by-Two Network-on-Chip System
Authors:
Riley Roberts,
Benjamin Lewis,
Arnd Hartmanns,
Prabal Basu,
Sanghamitra Roy,
Koushik Chakraborty,
Zhen Zhang
Abstract:
Modern network-on-chip (NoC) systems face reliability issues due to process and environmental variations. The power supply noise (PSN) in the power delivery network of a NoC plays a key role in determining reliability. PSN leads to voltage droop, which can cause timing errors in the NoC. This paper makes a novel contribution towards formally analyzing PSN in NoC systems. We present a probabilistic…
▽ More
Modern network-on-chip (NoC) systems face reliability issues due to process and environmental variations. The power supply noise (PSN) in the power delivery network of a NoC plays a key role in determining reliability. PSN leads to voltage droop, which can cause timing errors in the NoC. This paper makes a novel contribution towards formally analyzing PSN in NoC systems. We present a probabilistic model checking approach to observe the PSN in a generic 2x2 mesh NoC with a uniform random traffic load. Key features of PSN are measured at the behavioral level. To tackle state explosion, we apply incremental abstraction techniques, including a novel probabilistic choice abstraction, based on observations of NoC behavior. The Modest Toolset is used for probabilistic modeling and verification. Results are obtained for several flit injection patterns to reveal their impacts on PSN. Our analysis finds an optimal flit pattern generation with zero probability of PSN events and suggests spreading flits rather than releasing them in consecutive cycles in order to minimize PSN.
△ Less
Submitted 28 May, 2021;
originally announced August 2021.
-
Equidistant Linear Codes in Projective Spaces
Authors:
Pranab Basu
Abstract:
Linear codes in the projective space $\mathbb{P}_q(n)$, the set of all subspaces of the vector space $\mathbb{F}_q^n$, were first considered by Braun, Etzion and Vardy. The Grassmannian $\mathbb{G}_q(n,k)$ is the collection of all subspaces of dimension $k$ in $\mathbb{P}_q(n)$. We study equidistant linear codes in $\mathbb{P}_q(n)$ in this paper and establish that the normalized minimum distance…
▽ More
Linear codes in the projective space $\mathbb{P}_q(n)$, the set of all subspaces of the vector space $\mathbb{F}_q^n$, were first considered by Braun, Etzion and Vardy. The Grassmannian $\mathbb{G}_q(n,k)$ is the collection of all subspaces of dimension $k$ in $\mathbb{P}_q(n)$. We study equidistant linear codes in $\mathbb{P}_q(n)$ in this paper and establish that the normalized minimum distance of a linear code is maximum if and only if it is equidistant. We prove that the upper bound on the size of such class of linear codes is $2^n$ when $q=2$ as conjectured by Braun et al. Moreover, the codes attaining this bound are shown to have structures akin to combinatorial objects, viz. \emph{Fano plane} and \emph{sunflower}. We also prove the existence of equidistant linear codes in $\mathbb{P}_q(n)$ for any prime power $q$ using \emph{Steiner triple system}. Thus we establish that the problem of finding equidistant linear codes of maximum size in $\mathbb{P}_q(n)$ with constant distance $2d$ is equivalent to the problem of finding the largest $d$-intersecting family of subspaces in $\mathbb{G}_q(n, 2d)$ for all $1 \le d \le \lfloor \frac{n}{2}\rfloor$. Our discovery proves that there exist equidistant linear codes of size more than $2^n$ for every prime power $q > 2$.
△ Less
Submitted 22 July, 2021;
originally announced July 2021.
-
Benchmarking Differential Privacy and Federated Learning for BERT Models
Authors:
Priyam Basu,
Tiasa Singha Roy,
Rakshit Naidu,
Zumrut Muftuoglu,
Sahib Singh,
Fatemehsadat Mireshghallah
Abstract:
Natural Language Processing (NLP) techniques can be applied to help with the diagnosis of medical conditions such as depression, using a collection of a person's utterances. Depression is a serious medical illness that can have adverse effects on how one feels, thinks, and acts, which can lead to emotional and physical problems. Due to the sensitive nature of such data, privacy measures need to be…
▽ More
Natural Language Processing (NLP) techniques can be applied to help with the diagnosis of medical conditions such as depression, using a collection of a person's utterances. Depression is a serious medical illness that can have adverse effects on how one feels, thinks, and acts, which can lead to emotional and physical problems. Due to the sensitive nature of such data, privacy measures need to be taken for handling and training models with such data. In this work, we study the effects that the application of Differential Privacy (DP) has, in both a centralized and a Federated Learning (FL) setup, on training contextualized language models (BERT, ALBERT, RoBERTa and DistilBERT). We offer insights on how to privately train NLP models and what architectures and setups provide more desirable privacy utility trade-offs. We envisage this work to be used in future healthcare and mental health studies to keep medical history private. Therefore, we provide an open-source implementation of this work.
△ Less
Submitted 16 June, 2022; v1 submitted 26 June, 2021;
originally announced June 2021.
-
FBGEMM: Enabling High-Performance Low-Precision Deep Learning Inference
Authors:
Daya Khudia,
Jianyu Huang,
Protonu Basu,
Summer Deng,
Haixin Liu,
Jongsoo Park,
Mikhail Smelyanskiy
Abstract:
Deep learning models typically use single-precision (FP32) floating point data types for representing activations and weights, but a slew of recent research work has shown that computations with reduced-precision data types (FP16, 16-bit integers, 8-bit integers or even 4- or 2-bit integers) are enough to achieve same accuracy as FP32 and are much more efficient. Therefore, we designed fbgemm, a h…
▽ More
Deep learning models typically use single-precision (FP32) floating point data types for representing activations and weights, but a slew of recent research work has shown that computations with reduced-precision data types (FP16, 16-bit integers, 8-bit integers or even 4- or 2-bit integers) are enough to achieve same accuracy as FP32 and are much more efficient. Therefore, we designed fbgemm, a high-performance kernel library, from ground up to perform high-performance quantized inference on current generation CPUs. fbgemm achieves efficiency by fusing common quantization operations with a high-performance gemm implementation and by shape- and size-specific kernel code generation at runtime. The library has been deployed at Facebook, where it delivers greater than 2x performance gains with respect to our current production baseline.
△ Less
Submitted 12 January, 2021;
originally announced January 2021.
-
Matching through Embedding in Dense Graphs
Authors:
Nitish K. Panigrahy,
Prithwish Basu,
Don Towsley
Abstract:
Finding optimal matchings in dense graphs is of general interest and of particular importance in social, transportation and biological networks. While develo** optimal solutions for various matching problems is important, the running times of the fastest available optimal matching algorithms are too costly. However, when the vertices of the graphs are point-sets in $R^d$ and edge weights corresp…
▽ More
Finding optimal matchings in dense graphs is of general interest and of particular importance in social, transportation and biological networks. While develo** optimal solutions for various matching problems is important, the running times of the fastest available optimal matching algorithms are too costly. However, when the vertices of the graphs are point-sets in $R^d$ and edge weights correspond to the euclidean distances, the available optimal matching algorithms are substantially faster. In this paper, we propose a novel network embedding based heuristic algorithm to solve various matching problems in dense graphs. In particular, using existing network embedding techniques, we first find a low dimensional representation of the graph vertices in $R^d$ and then run faster available matching algorithms on the embedded vertices. To the best of our knowledge, this is the first work that applies network embedding to solve various matching problems. Experimental results validate the efficacy of our proposed algorithm.
△ Less
Submitted 13 November, 2020;
originally announced November 2020.
-
Resource Allocation in One-dimensional Distributed Service Networks with Applications
Authors:
Nitish K. Panigrahy,
Prithwish Basu,
Philippe Nain,
Don Towsley,
Ananthram Swami,
Kevin S. Chan,
Kin K. Leung
Abstract:
We consider assignment policies that allocate resources to users, where both resources and users are located on a one-dimensional line. First, we consider unidirectional assignment policies that allocate resources only to users located to their left. We propose the Move to Right (MTR) policy, which scans from left to right assigning nearest rightmost available resource to a user, and contrast it t…
▽ More
We consider assignment policies that allocate resources to users, where both resources and users are located on a one-dimensional line. First, we consider unidirectional assignment policies that allocate resources only to users located to their left. We propose the Move to Right (MTR) policy, which scans from left to right assigning nearest rightmost available resource to a user, and contrast it to the Unidirectional Gale-Shapley (UGS) matching policy. While both policies among all unidirectional policies, minimize the expected distance traveled by a request (request distance), MTR is fairer. Moreover, we show that when user and resource locations are modeled by statistical point processes, and resources are allowed to satisfy more than one user, the spatial system under unidirectional policies can be mapped into bulk service queueing systems, thus allowing the application of many queueing theory results that yield closed form expressions. As we consider a case where different resources can satisfy different numbers of users, we also generate new results for bulk service queues. We also consider bidirectional policies where there are no directional restrictions on resource allocation and develop an algorithm for computing the optimal assignment which is more efficient than known algorithms in the literature when there are more resources than users. Numerical evaluation of performance of unidirectional and bidirectional allocation schemes yields design guidelines beneficial for resource placement. \np{Finally, we present a heuristic algorithm, which leverages the optimal dynamic programming scheme for one-dimensional inputs to obtain approximate solutions to the optimal assignment problem for the two-dimensional scenario and empirically yields request distances within a constant factor of the optimal solution.
△ Less
Submitted 8 November, 2020;
originally announced November 2020.
-
On the Analysis of Spatially Constrained Power of Two Choice Policies
Authors:
Nitish K. Panigrahy,
Prithwish Basu,
Don Towsley,
Ananthram Swami,
Kin K. Leung
Abstract:
We consider a class of power of two choice based assignment policies for allocating users to servers, where both users and servers are located on a two-dimensional Euclidean plane. In this framework, we investigate the inherent tradeoff between the communication cost, and load balancing performance of different allocation policies. To this end, we first design and evaluate a Spatial Power of two (…
▽ More
We consider a class of power of two choice based assignment policies for allocating users to servers, where both users and servers are located on a two-dimensional Euclidean plane. In this framework, we investigate the inherent tradeoff between the communication cost, and load balancing performance of different allocation policies. To this end, we first design and evaluate a Spatial Power of two (sPOT) policy in which each user is allocated to the least loaded server among its two geographically nearest servers sequentially. When servers are placed on a two-dimensional square grid, sPOT maps to the classical Power of two (POT) policy on the Delaunay graph associated with the Voronoi tessellation of the set of servers. We show that the associated Delaunay graph is 4-regular and provide expressions for asymptotic maximum load using results from the literature. For uniform placement of servers, we map sPOT to a classical balls and bins allocation policy with bins corresponding to the Voronoi regions associated with the second order Voronoi diagram of the set of servers. We provide expressions for the lower bound on the asymptotic expected maximum load on the servers and prove that sPOT does not achieve POT load balancing benefits. However, experimental results suggest the efficacy of sPOT with respect to expected communication cost. Finally, we propose two non-uniform server sampling based POT policies that achieve the best of both the performance metrics. Experimental results validate the effctiveness of our proposed policies.
△ Less
Submitted 4 November, 2020;
originally announced November 2020.
-
Proximity Based Load Balancing Policies on Graphs: A Simulation Study
Authors:
Nitish K. Panigrahy,
Thirupathaiah Vasantam,
Prithwish Basu,
Don Towsley
Abstract:
Distributed load balancing is the act of allocating jobs among a set of servers as evenly as possible. There are mainly two versions of the load balancing problem that have been studied in the literature: static and dynamic. The static interpretation leads to formulating the load balancing problem as a case with jobs (balls) never leaving the system and accumulating at the servers (bins) whereas t…
▽ More
Distributed load balancing is the act of allocating jobs among a set of servers as evenly as possible. There are mainly two versions of the load balancing problem that have been studied in the literature: static and dynamic. The static interpretation leads to formulating the load balancing problem as a case with jobs (balls) never leaving the system and accumulating at the servers (bins) whereas the dynamic setting deals with the case when jobs arrive and leave the system after service completion. This paper designs and evaluates server proximity aware job allocation policies for treating load balancing problems with a goal to reduce the communication cost associated with the jobs. We consider a class of proximity aware Power of Two (POT) choice based assignment policies for allocating jobs to servers, where servers are interconnected as an n-vertex graph G(V, E). For the static version, we assume each job arrives at one of the servers, u. For the dynamic setting, we assume G to be a circular graph and job arrival process at each server is described by a Poisson point process with the job service time exponentially distributed. For both settings, we then assign each job to the server with minimum load among servers u and v where v is chosen according to one of the following two policies: (i) Unif-POT(k): Sample a server v uniformly at random from k-hop neighborhood of u (ii) InvSq-POT(k): Sample a server v from k-hop neighborhood of u with probability proportional to the inverse square of the distance between u and v. Our simulation results show that both the policies consistently produce a load distribution which is much similar to that of a classical proximity oblivious POT policy.
△ Less
Submitted 3 November, 2020;
originally announced November 2020.
-
Amortized Probabilistic Detection of Communities in Graphs
Authors:
Yueqi Wang,
Yoonho Lee,
Pallab Basu,
Juho Lee,
Yee Whye Teh,
Liam Paninski,
Ari Pakman
Abstract:
Learning community structures in graphs has broad applications across scientific domains. While graph neural networks (GNNs) have been successful in encoding graph structures, existing GNN-based methods for community detection are limited by requiring knowledge of the number of communities in advance, in addition to lacking a proper probabilistic formulation to handle uncertainty. We propose a sim…
▽ More
Learning community structures in graphs has broad applications across scientific domains. While graph neural networks (GNNs) have been successful in encoding graph structures, existing GNN-based methods for community detection are limited by requiring knowledge of the number of communities in advance, in addition to lacking a proper probabilistic formulation to handle uncertainty. We propose a simple framework for amortized community detection, which addresses both of these issues by combining the expressive power of GNNs with recent methods for amortized clustering. Our models consist of a graph representation backbone that extracts structural information and an amortized clustering network that naturally handles variable numbers of clusters. Both components combine into well-defined models of the posterior distribution of graph communities and are jointly optimized given labeled graphs. At inference time, the models yield parallel samples from the posterior of community labels, quantifying uncertainty in a principled way. We evaluate several models from our framework on synthetic and real datasets and demonstrate superior performance to previous methods. As a separate contribution, we extend recent amortized probabilistic clustering architectures by adding attention modules, which yield further improvements on community detection tasks.
△ Less
Submitted 15 June, 2021; v1 submitted 29 October, 2020;
originally announced October 2020.
-
Resource Sharing in the Edge: A Distributed Bargaining-Theoretic Approach
Authors:
Faheem Zafari,
Prithwish Basu,
Kin K. Leung,
Jian Li,
Ananthram Swami,
Don Towsley
Abstract:
The growing demand for edge computing resources, particularly due to increasing popularity of Internet of Things (IoT), and distributed machine/deep learning applications poses a significant challenge. On the one hand, certain edge service providers (ESPs) may not have sufficient resources to satisfy their applications according to the associated service-level agreements. On the other hand, some E…
▽ More
The growing demand for edge computing resources, particularly due to increasing popularity of Internet of Things (IoT), and distributed machine/deep learning applications poses a significant challenge. On the one hand, certain edge service providers (ESPs) may not have sufficient resources to satisfy their applications according to the associated service-level agreements. On the other hand, some ESPs may have additional unused resources. In this paper, we propose a resource-sharing framework that allows different ESPs to optimally utilize their resources and improve the satisfaction level of applications subject to constraints such as communication cost for sharing resources across ESPs. Our framework considers that different ESPs have their own objectives for utilizing their resources, thus resulting in a multi-objective optimization problem. We present an $N$-person \emph{Nash Bargaining Solution} (NBS) for resource allocation and sharing among ESPs with \emph{Pareto} optimality guarantee. Furthermore, we propose a \emph{distributed}, primal-dual algorithm to obtain the NBS by proving that the strong-duality property holds for the resultant resource sharing optimization problem.
Using synthetic and real-world data traces, we show numerically that the proposed NBS based framework not only enhances the ability to satisfy applications' resource demands, but also improves utilities of different ESPs.
△ Less
Submitted 4 July, 2020; v1 submitted 13 January, 2020;
originally announced January 2020.
-
Let's Share: A Game-Theoretic Framework for Resource Sharing in Mobile Edge Clouds
Authors:
Faheem Zafari,
Kin K. Leung,
Don Towsley,
Prithwish Basu,
Ananthram Swami,
Jian Li
Abstract:
Mobile edge computing seeks to provide resources to different delay-sensitive applications. This is a challenging problem as an edge cloud-service provider may not have sufficient resources to satisfy all resource requests. Furthermore, allocating available resources optimally to different applications is also challenging. Resource sharing among different edge cloud-service providers can address t…
▽ More
Mobile edge computing seeks to provide resources to different delay-sensitive applications. This is a challenging problem as an edge cloud-service provider may not have sufficient resources to satisfy all resource requests. Furthermore, allocating available resources optimally to different applications is also challenging. Resource sharing among different edge cloud-service providers can address the aforementioned limitation as certain service providers may have resources available that can be ``rented'' by other service providers. However, edge cloud service providers can have different objectives or \emph{utilities}. Therefore, there is a need for an efficient and effective mechanism to share resources among service providers, while considering the different objectives of various providers. We model resource sharing as a multi-objective optimization problem and present a solution framework based on \emph{Cooperative Game Theory} (CGT). We consider the strategy where each service provider allocates resources to its native applications first and shares the remaining resources with applications from other service providers. We prove that for a monotonic, non-decreasing utility function, the game is canonical and convex. Hence, the \emph{core} is not empty and the grand coalition is stable. We propose two algorithms \emph{Game-theoretic Pareto optimal allocation} (GPOA) and \emph{Polyandrous-Polygamous Matching based Pareto Optimal Allocation} (PPMPOA) that provide allocations from the core. Hence the obtained allocations are \emph{Pareto} optimal and the grand coalition of all the service providers is stable. Experimental results confirm that our proposed resource sharing framework improves utilities of edge cloud-service providers and application request satisfaction.
△ Less
Submitted 2 January, 2020;
originally announced January 2020.
-
The Lattice Structure of Linear Subspace Codes
Authors:
Pranab Basu,
Navin Kashyap
Abstract:
The projective space $\mathbb{P}_q(n)$, i.e. the set of all subspaces of the vector space $\mathbb{F}_q^n$, is a metric space endowed with the subspace distance metric. Braun, Etzion and Vardy argued that codes in a projective space are analogous to binary block codes in $\mathbb{F}_2^n$ using a framework of lattices. They defined linear codes in $\mathbb{P}_q(n)$ by mimicking key features of line…
▽ More
The projective space $\mathbb{P}_q(n)$, i.e. the set of all subspaces of the vector space $\mathbb{F}_q^n$, is a metric space endowed with the subspace distance metric. Braun, Etzion and Vardy argued that codes in a projective space are analogous to binary block codes in $\mathbb{F}_2^n$ using a framework of lattices. They defined linear codes in $\mathbb{P}_q(n)$ by mimicking key features of linear codes in the Hamming space $\mathbb{F}_2^n$. In this paper, we prove that a linear code in a projective space forms a sublattice of the corresponding projective lattice if and only if the code is closed under intersection. The sublattice thus formed is geometric distributive. We also present an application of this lattice-theoretic characterization.
△ Less
Submitted 2 November, 2019;
originally announced November 2019.
-
A Game-Theoretic Framework for Resource Sharing in Clouds
Authors:
Faheem Zafari,
Kin K. Leung,
Don Towsley,
Prithwish Basu,
Ananthram Swami
Abstract:
Providing resources to different users or applications is fundamental to cloud computing. This is a challenging problem as a cloud service provider may have insufficient resources to satisfy all user requests. Furthermore, allocating available resources optimally to different applications is also challenging. Resource sharing among different cloud service providers can improve resource availabilit…
▽ More
Providing resources to different users or applications is fundamental to cloud computing. This is a challenging problem as a cloud service provider may have insufficient resources to satisfy all user requests. Furthermore, allocating available resources optimally to different applications is also challenging. Resource sharing among different cloud service providers can improve resource availability and resource utilization as certain cloud service providers may have free resources available that can be ``rented'' by other service providers. However, different cloud service providers can have different objectives or \emph{utilities}. Therefore, there is a need for a framework that can share and allocate resources in an efficient and effective way, while taking into account the objectives of various service providers that results in a \emph{multi-objective optimization} problem. In this paper, we present a \emph{Cooperative Game Theory} (CGT) based framework for resource sharing and allocation among different service providers with varying objectives that form a coalition. We show that the resource sharing problem can be modeled as an $N-$player \emph{canonical} cooperative game with \emph{non-transferable utility} (NTU) and prove that the game is convex for monotonic non-decreasing utilities. We propose an $\mathcal{O}({N})$ algorithm that provides an allocation from the \emph{core}, hence guaranteeing \emph{Pareto optimality}. We evaluate the performance of our proposed resource sharing framework in a number of simulation settings and show that our proposed framework improves user satisfaction and utility of service providers.
△ Less
Submitted 28 May, 2019; v1 submitted 1 April, 2019;
originally announced April 2019.
-
An information theoretic model for summarization, and some basic results
Authors:
Eric Graves,
Qiang Ning,
Prithwish Basu
Abstract:
A basic information theoretic model for summarization is formulated. Here summarization is considered as the process of taking a report of $v$ binary objects, and producing from it a $j$ element subset that captures most of the important features of the original report, with importance being defined via an arbitrary set function endemic to the model. The loss of information is then measured by a w…
▽ More
A basic information theoretic model for summarization is formulated. Here summarization is considered as the process of taking a report of $v$ binary objects, and producing from it a $j$ element subset that captures most of the important features of the original report, with importance being defined via an arbitrary set function endemic to the model. The loss of information is then measured by a weight average of variational distances, which we term the semantic loss.
Our results include both cases where the probability distribution generating the $v$-length reports are known and unknown. In the case where it is known, our results demonstrate how to construct summarizers which minimize the semantic loss. For the case where the probability distribution is unknown, we show how to construct summarizers whose semantic loss when averaged uniformly over all possible distribution converges to the minimum.
△ Less
Submitted 18 January, 2019;
originally announced January 2019.
-
Resource Allocation in One-dimensional Distributed Service Networks
Authors:
Nitish K. Panigrahy,
Prithwish Basu,
Philippe Nain,
Don Towsley,
Ananthram Swami,
Kevin S. Chan,
Kin K. Leung
Abstract:
We consider assignment policies that allocate resources to users, where both resources and users are located on a one-dimensional line. First, we consider unidirectional assignment policies that allocate resources only to users located to their left. We propose the Move to Right (MTR) policy, which scans from left to right assigning nearest rightmost available resource to a user, and contrast it t…
▽ More
We consider assignment policies that allocate resources to users, where both resources and users are located on a one-dimensional line. First, we consider unidirectional assignment policies that allocate resources only to users located to their left. We propose the Move to Right (MTR) policy, which scans from left to right assigning nearest rightmost available resource to a user, and contrast it to the Unidirectional Gale-Shapley (UGS) matching policy. While both these policies are optimal among all unidirectional policies, we show that they are equivalent with respect to the expected distance traveled by a request (request distance), although MTR is fairer. Moreover, we show that when user and resource locations are modeled by statistical point processes, and resources are allowed to satisfy more than one user, the spatial system under unidirectional policies can be mapped into bulk service queuing systems, thus allowing the application of a plethora of queuing theory results that yield closed form expressions. As we consider a case where different resources can satisfy different numbers of users, we also generate new results for bulk service queues. We also consider bidirectional policies where there are no directional restrictions on resource allocation and develop an algorithm for computing the optimal assignment which is more efficient than known algorithms in the literature when there are more resources than users. Finally, numerical evaluation of performance of unidirectional and bidirectional allocation schemes yields design guidelines beneficial for resource placement.
△ Less
Submitted 11 February, 2020; v1 submitted 8 January, 2019;
originally announced January 2019.
-
Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications
Authors:
Jongsoo Park,
Maxim Naumov,
Protonu Basu,
Summer Deng,
Aravind Kalaiah,
Daya Khudia,
James Law,
Parth Malani,
Andrey Malevich,
Satish Nadathur,
Juan Pino,
Martin Schatz,
Alexander Sidorov,
Viswanath Sivakumar,
Andrew Tulloch,
Xiaodong Wang,
Yiming Wu,
Hector Yuen,
Utku Diril,
Dmytro Dzhulgakov,
Kim Hazelwood,
Bill Jia,
Yangqing Jia,
Lin Qiao,
Vijay Rao
, et al. (3 additional authors not shown)
Abstract:
The application of deep learning techniques resulted in remarkable improvement of machine learning models. In this paper provides detailed characterizations of deep learning models used in many Facebook social network services. We present computational characteristics of our models, describe high performance optimizations targeting existing systems, point out their limitations and make suggestions…
▽ More
The application of deep learning techniques resulted in remarkable improvement of machine learning models. In this paper provides detailed characterizations of deep learning models used in many Facebook social network services. We present computational characteristics of our models, describe high performance optimizations targeting existing systems, point out their limitations and make suggestions for the future general-purpose/accelerated inference hardware. Also, we highlight the need for better co-design of algorithms, numerics and computing platforms to address the challenges of workloads often run in data centers.
△ Less
Submitted 29 November, 2018; v1 submitted 24 November, 2018;
originally announced November 2018.
-
Maximizing Coverage Centrality via Network Design: Extended Version
Authors:
Sourav Medya,
Arlei Silva,
Ambuj Singh,
Prithwish Basu,
Ananthram Swami
Abstract:
Network centrality plays an important role in many applications. Central nodes in social networks can be influential, driving opinions and spreading news or rumors.In hyperlinked environments, such as the Web, where users navigate via clicks, central content receives high traffic, becoming targets for advertising campaigns. While there is an extensive amount of work on centrality measures and thei…
▽ More
Network centrality plays an important role in many applications. Central nodes in social networks can be influential, driving opinions and spreading news or rumors.In hyperlinked environments, such as the Web, where users navigate via clicks, central content receives high traffic, becoming targets for advertising campaigns. While there is an extensive amount of work on centrality measures and their efficient computation, controlling nodes' centrality via network updates is a more recent and challenging problem. Performing minimal modifications to a network to achieve a desired property falls under the umbrella of network design problems. This paper is focused on improving the coverage centrality of a set of nodes, which is the number of pairs of nodes that have a shortest path passing through the set, by adding edges to the network. We prove strong inapproximability results and propose a greedy algorithm for maximizing coverage centrality. To ensure scalability to large networks, we also design an efficient sampling algorithm for the problem. In addition to providing an extensive empirical evaluation of our algorithms, we also show that, under some realistic constraints, the proposed solutions achieve almost-optimal approximation for coverage centrality maximization.
△ Less
Submitted 9 October, 2017; v1 submitted 14 February, 2017;
originally announced February 2017.
-
Outlier Detection from Network Data with Subnetwork Interpretation
Authors:
Xuan-Hong Dang,
Arlei Silva,
Ambuj Singh,
Ananthram Swami,
Prithwish Basu
Abstract:
Detecting a small number of outliers from a set of data observations is always challenging. This problem is more difficult in the setting of multiple network samples, where computing the anomalous degree of a network sample is generally not sufficient. In fact, explaining why the network is exceptional, expressed in the form of subnetwork, is also equally important. In this paper, we develop a nov…
▽ More
Detecting a small number of outliers from a set of data observations is always challenging. This problem is more difficult in the setting of multiple network samples, where computing the anomalous degree of a network sample is generally not sufficient. In fact, explaining why the network is exceptional, expressed in the form of subnetwork, is also equally important. In this paper, we develop a novel algorithm to address these two key problems. We treat each network sample as a potential outlier and identify subnetworks that mostly discriminate it from nearby regular samples. The algorithm is developed in the framework of network regression combined with the constraints on both network topology and L1-norm shrinkage to perform subnetwork discovery. Our method thus goes beyond subspace/subgraph discovery and we show that it converges to a global optimum. Evaluation on various real-world network datasets demonstrates that our algorithm not only outperforms baselines in both network and high dimensional setting, but also discovers highly relevant and interpretable local subnetworks, further enhancing our understanding of anomalous networks.
△ Less
Submitted 30 September, 2016;
originally announced October 2016.
-
Generative Models for Global Collaboration Relationships
Authors:
Ertugrul N. Ciftcioglu,
Ram Ramanathan,
Prithwish Basu
Abstract:
When individuals interact with each other and meaningfully contribute toward a common goal, it results in a collaboration, as can be seen in many walks of life such as scientific research, motion picture production, or team sports. The artifacts resulting from a collaboration (e.g. papers, movies) are best captured using a hypergraph model, whereas the relation of who has collaborated with whom is…
▽ More
When individuals interact with each other and meaningfully contribute toward a common goal, it results in a collaboration, as can be seen in many walks of life such as scientific research, motion picture production, or team sports. The artifacts resulting from a collaboration (e.g. papers, movies) are best captured using a hypergraph model, whereas the relation of who has collaborated with whom is best captured via an abstract simplicial complex (SC). In this paper, we propose a generative algorithm GeneSCs for SCs modeling fundamental collaboration relations, primarily based on preferential attachment. The proposed network growth process favors attachment that is preferential not to an individual's degree, i.e., how many people has he/she collaborated with, but to his/her facet degree, i.e., how many maximal groups or facets has he/she collaborated within. Unlike graphs, in SCs, both facet degrees (of nodes) and facet sizes are important to capture connectivity properties. Based on our observation that several real-world facet size distributions have significant deviation from power law-mainly due to the fact that larger facets tend to subsume smaller ones-we adopt a data-driven approach. We seed GeneSCs with a facet size distribution informed by collaboration network data and randomly grow the SC facet-by-facet to generate a final SC whose facet degree distribution matches real data. We prove that the facet degree distribution yielded by GeneSCs is power law distributed for large SCs and show that it is in agreement with real world co-authorship data. Finally, based on our intuition of collaboration formation in domains such as collaborative scientific experiments and movie production, we propose two variants of GeneSCs based on clamped and hybrid preferential attachment schemes, and show that they perform well in these domains.
△ Less
Submitted 1 August, 2016; v1 submitted 26 June, 2016;
originally announced June 2016.
-
Graph Wavelets via Sparse Cuts: Extended Version
Authors:
Arlei Silva,
Xuan-Hong Dang,
Prithwish Basu,
Ambuj K Singh,
Ananthram Swami
Abstract:
Modeling information that resides on vertices of large graphs is a key problem in several real-life applications, ranging from social networks to the Internet-of-things. Signal Processing on Graphs and, in particular, graph wavelets can exploit the intrinsic smoothness of these datasets in order to represent them in a both compact and accurate manner. However, how to discover wavelet bases that ca…
▽ More
Modeling information that resides on vertices of large graphs is a key problem in several real-life applications, ranging from social networks to the Internet-of-things. Signal Processing on Graphs and, in particular, graph wavelets can exploit the intrinsic smoothness of these datasets in order to represent them in a both compact and accurate manner. However, how to discover wavelet bases that capture the geometry of the data with respect to the signal as well as the graph structure remains an open question. In this paper, we study the problem of computing graph wavelet bases via sparse cuts in order to produce low-dimensional encodings of data-driven bases. This problem is connected to known hard problems in graph theory (e.g. multiway cuts) and thus requires an efficient heuristic. We formulate the basis discovery task as a relaxation of a vector optimization problem, which leads to an elegant solution as a regularized eigenvalue computation. Moreover, we propose several strategies in order to scale our algorithm to large graphs. Experimental results show that the proposed algorithm can effectively encode both the graph structure and signal, producing compressed and accurate representations for vertex values in a wide range of datasets (e.g. sensor and gene networks) and significantly outperforming the best baseline.
△ Less
Submitted 12 June, 2016; v1 submitted 10 February, 2016;
originally announced February 2016.
-
Computing Traversal Times on Dynamic Markovian Paths
Authors:
Philippe Nain,
Don Towsley,
Matthew P. Johnson,
Prithwish Basu,
Amotz Bar-Noy,
Feng Yu
Abstract:
In source routing, a complete path is chosen for a packet to travel from source to destination. While computing the time to traverse such a path may be straightforward in a fixed, static graph, doing so becomes much more challenging in dynamic graphs, in which the state of an edge in one time slot (i.e., its presence or absence) is random, and may depend on its state in the previous time step. The…
▽ More
In source routing, a complete path is chosen for a packet to travel from source to destination. While computing the time to traverse such a path may be straightforward in a fixed, static graph, doing so becomes much more challenging in dynamic graphs, in which the state of an edge in one time slot (i.e., its presence or absence) is random, and may depend on its state in the previous time step. The traversal time is due to both time spent waiting for edges to appear and time spent crossing them once they become available. We compute the expected traversal time (ETT) for a dynamic path in a number of special cases of stochastic edge dynamics models, and for three edge failure models, culminating in a surprisingly challenging yet realistic setting in which the initial configuration of edge states for the entire path is known. We show that the ETT for this "initial configuration" setting can be computed in quadratic time, by an algorithm based on probability generating functions. We also give several linear-time upper and lower bounds on the ETT.
△ Less
Submitted 14 March, 2013;
originally announced March 2013.
-
Online Myopic Network Covering
Authors:
Konstantin Avrachenkov,
Prithwish Basu,
Giovanni Neglia,
Bruno Ribeiro,
Don Towsley
Abstract:
Efficient marketing or awareness-raising campaigns seek to recruit $n$ influential individuals -- where $n$ is the campaign budget -- that are able to cover a large target audience through their social connections. So far most of the related literature on maximizing this network cover assumes that the social network topology is known. Even in such a case the optimal solution is NP-hard. In practic…
▽ More
Efficient marketing or awareness-raising campaigns seek to recruit $n$ influential individuals -- where $n$ is the campaign budget -- that are able to cover a large target audience through their social connections. So far most of the related literature on maximizing this network cover assumes that the social network topology is known. Even in such a case the optimal solution is NP-hard. In practice, however, the network topology is generally unknown and needs to be discovered on-the-fly. In this work we consider an unknown topology where recruited individuals disclose their social connections (a feature known as {\em one-hop lookahead}). The goal of this work is to provide an efficient greedy online algorithm that recruits individuals as to maximize the size of target audience covered by the campaign.
We propose a new greedy online algorithm, Maximum Expected $d$-Excess Degree (MEED), and provide, to the best of our knowledge, the first detailed theoretical analysis of the cover size of a variety of well known network sampling algorithms on finite networks. Our proposed algorithm greedily maximizes the expected size of the cover. For a class of random power law networks we show that MEED simplifies into a straightforward procedure, which we denote MOD (Maximum Observed Degree). We substantiate our analytical results with extensive simulations and show that MOD significantly outperforms all analyzed myopic algorithms. We note that performance may be further improved if the node degree distribution is known or can be estimated online during the campaign.
△ Less
Submitted 20 December, 2012;
originally announced December 2012.
-
Multiple Random Walks to Uncover Short Paths in Power Law Networks
Authors:
Bruno Ribeiro,
Prithwish Basu,
Don Towsley
Abstract:
Consider the following routing problem in the context of a large scale network $G$, with particular interest paid to power law networks, although our results do not assume a particular degree distribution. A small number of nodes want to exchange messages and are looking for short paths on $G$. These nodes do not have access to the topology of $G$ but are allowed to crawl the network within a limi…
▽ More
Consider the following routing problem in the context of a large scale network $G$, with particular interest paid to power law networks, although our results do not assume a particular degree distribution. A small number of nodes want to exchange messages and are looking for short paths on $G$. These nodes do not have access to the topology of $G$ but are allowed to crawl the network within a limited budget. Only crawlers whose sample paths cross are allowed to exchange topological information. In this work we study the use of random walks (RWs) to crawl $G$. We show that the ability of RWs to find short paths bears no relation to the paths that they take. Instead, it relies on two properties of RWs on power law networks: 1) RW's ability observe a sizable fraction of the network edges; and 2) an almost certainty that two distinct RW sample paths cross after a small percentage of the nodes have been visited. We show promising simulation results on several real world networks.
△ Less
Submitted 26 May, 2012;
originally announced May 2012.
-
Cologne: A Declarative Distributed Constraint Optimization Platform
Authors:
Changbin Liu,
Lu Ren,
Boon Thau Loo,
Yun Mao,
Prithwish Basu
Abstract:
This paper presents Cologne, a declarative optimization platform that enables constraint optimization problems (COPs) to be declaratively specified and incrementally executed in distributed systems. Cologne integrates a declarative networking engine with an off-the-shelf constraint solver. We have developed the Colog language that combines distributed Datalog used in declarative networking with la…
▽ More
This paper presents Cologne, a declarative optimization platform that enables constraint optimization problems (COPs) to be declaratively specified and incrementally executed in distributed systems. Cologne integrates a declarative networking engine with an off-the-shelf constraint solver. We have developed the Colog language that combines distributed Datalog used in declarative networking with language constructs for specifying goals and constraints used in COPs. Cologne uses novel query processing strategies for processing Colog programs, by combining the use of bottom-up distributed Datalog evaluation with top-down goal-oriented constraint solving. Using case studies based on cloud and wireless network optimizations, we demonstrate that Cologne (1) can flexibly support a wide range of policy-based optimizations in distributed systems, (2) results in orders of magnitude less code compared to imperative implementations, and (3) is highly efficient with low overhead and fast convergence times.
△ Less
Submitted 26 April, 2012;
originally announced April 2012.
-
Modeling and Analysis of Time-Varying Graphs
Authors:
Prithwish Basu,
Amotz Bar-Noy,
Ram Ramanathan,
Matthew P. Johnson
Abstract:
We live in a world increasingly dominated by networks -- communications, social, information, biological etc. A central attribute of many of these networks is that they are dynamic, that is, they exhibit structural changes over time. While the practice of dynamic networks has proliferated, we lag behind in the fundamental, mathematical understanding of network dynamism. Existing research on time-v…
▽ More
We live in a world increasingly dominated by networks -- communications, social, information, biological etc. A central attribute of many of these networks is that they are dynamic, that is, they exhibit structural changes over time. While the practice of dynamic networks has proliferated, we lag behind in the fundamental, mathematical understanding of network dynamism. Existing research on time-varying graphs ranges from preliminary algorithmic studies (e.g., Ferreira's work on evolving graphs) to analysis of specific properties such as flooding time in dynamic random graphs. A popular model for studying dynamic graphs is a sequence of graphs arranged by increasing snapshots of time. In this paper, we study the fundamental property of reachability in a time-varying graph over time and characterize the latency with respect to two metrics, namely store-or-advance latency and cut-through latency. Instead of expected value analysis, we concentrate on characterizing the exact probability distribution of routing latency along a randomly intermittent path in two popular dynamic random graph models. Using this analysis, we characterize the loss of accuracy (in a probabilistic setting) between multiple temporal graph models, ranging from one that preserves all the temporal ordering information for the purpose of computing temporal graph properties to one that collapses various snapshots into one graph (an operation called smashing), with multiple intermediate variants. We also show how some other traditional graph theoretic properties can be extended to the temporal domain. Finally, we propose algorithms for controlling the progress of a packet in single-copy adaptive routing schemes in various dynamic random graphs.
△ Less
Submitted 1 December, 2010;
originally announced December 2010.
-
Superposition frames for adaptive time-frequency analysis and fast reconstruction
Authors:
Daniel Rudoy,
Prabahan Basu,
Patrick J. Wolfe
Abstract:
In this article we introduce a broad family of adaptive, linear time-frequency representations termed superposition frames, and show that they admit desirable fast overlap-add reconstruction properties akin to standard short-time Fourier techniques. This approach stands in contrast to many adaptive time-frequency representations in the extant literature, which, while more flexible than standard…
▽ More
In this article we introduce a broad family of adaptive, linear time-frequency representations termed superposition frames, and show that they admit desirable fast overlap-add reconstruction properties akin to standard short-time Fourier techniques. This approach stands in contrast to many adaptive time-frequency representations in the extant literature, which, while more flexible than standard fixed-resolution approaches, typically fail to provide efficient reconstruction and often lack the regular structure necessary for precise frame-theoretic analysis. Our main technical contributions come through the development of properties which ensure that this construction provides for a numerically stable, invertible signal representation. Our primary algorithmic contributions come via the introduction and discussion of specific signal adaptation criteria in deterministic and stochastic settings, based respectively on time-frequency concentration and nonstationarity detection. We conclude with a short speech enhancement example that serves to highlight potential applications of our approach.
△ Less
Submitted 3 November, 2009; v1 submitted 29 June, 2009;
originally announced June 2009.
