-
TsSHAP: Robust model agnostic feature-based explainability for time series forecasting
Authors:
Vikas C. Raykar,
Arindam Jati,
Sumanta Mukherjee,
Nupur Aggarwal,
Kanthi Sarpatwar,
Giridhar Ganapavarapu,
Roman Vaculin
Abstract:
A trustworthy machine learning model should be accurate as well as explainable. Understanding why a model makes a certain decision defines the notion of explainability. While various flavors of explainability have been well-studied in supervised learning paradigms like classification and regression, literature on explainability for time series forecasting is relatively scarce.
In this paper, we…
▽ More
A trustworthy machine learning model should be accurate as well as explainable. Understanding why a model makes a certain decision defines the notion of explainability. While various flavors of explainability have been well-studied in supervised learning paradigms like classification and regression, literature on explainability for time series forecasting is relatively scarce.
In this paper, we propose a feature-based explainability algorithm, TsSHAP, that can explain the forecast of any black-box forecasting model. The method is agnostic of the forecasting model and can provide explanations for a forecast in terms of interpretable features defined by the user a prior.
The explanations are in terms of the SHAP values obtained by applying the TreeSHAP algorithm on a surrogate model that learns a map** between the interpretable feature space and the forecast of the black-box model.
Moreover, we formalize the notion of local, semi-local, and global explanations in the context of time series forecasting, which can be useful in several scenarios. We validate the efficacy and robustness of TsSHAP through extensive experiments on multiple datasets.
△ Less
Submitted 22 March, 2023;
originally announced March 2023.
-
HE-PEx: Efficient Machine Learning under Homomorphic Encryption using Pruning, Permutation and Expansion
Authors:
Ehud Aharoni,
Moran Baruch,
Pradip Bose,
Alper Buyuktosunoglu,
Nir Drucker,
Subhankar Pal,
Tomer Pelleg,
Kanthi Sarpatwar,
Hayim Shaul,
Omri Soceanu,
Roman Vaculin
Abstract:
Privacy-preserving neural network (NN) inference solutions have recently gained significant traction with several solutions that provide different latency-bandwidth trade-offs. Of these, many rely on homomorphic encryption (HE), a method of performing computations over encrypted data. However, HE operations even with state-of-the-art schemes are still considerably slow compared to their plaintext…
▽ More
Privacy-preserving neural network (NN) inference solutions have recently gained significant traction with several solutions that provide different latency-bandwidth trade-offs. Of these, many rely on homomorphic encryption (HE), a method of performing computations over encrypted data. However, HE operations even with state-of-the-art schemes are still considerably slow compared to their plaintext counterparts. Pruning the parameters of a NN model is a well-known approach to improving inference latency. However, pruning methods that are useful in the plaintext context may lend nearly negligible improvement in the HE case, as has also been demonstrated in recent work.
In this work, we propose a novel set of pruning methods that reduce the latency and memory requirement, thus bringing the effectiveness of plaintext pruning methods to HE. Crucially, our proposal employs two key techniques, viz. permutation and expansion of the packed model weights, that enable pruning significantly more ciphertexts and recuperating most of the accuracy loss, respectively. We demonstrate the advantage of our method on fully connected layers where the weights are packed using a recently proposed packing technique called tile tensors, which allows executing deep NN inference in a non-interactive mode. We evaluate our methods on various autoencoder architectures and demonstrate that for a small mean-square reconstruction loss of 1.5*10^{-5} on MNIST, we reduce the memory requirement and latency of HE-enabled inference by 60%.
△ Less
Submitted 7 July, 2022;
originally announced July 2022.
-
Efficient Encrypted Inference on Ensembles of Decision Trees
Authors:
Kanthi Sarpatwar,
Karthik Nandakumar,
Nalini Ratha,
James Rayfield,
Karthikeyan Shanmugam,
Sharath Pankanti,
Roman Vaculin
Abstract:
Data privacy concerns often prevent the use of cloud-based machine learning services for sensitive personal data. While homomorphic encryption (HE) offers a potential solution by enabling computations on encrypted data, the challenge is to obtain accurate machine learning models that work within the multiplicative depth constraints of a leveled HE scheme. Existing approaches for encrypted inferenc…
▽ More
Data privacy concerns often prevent the use of cloud-based machine learning services for sensitive personal data. While homomorphic encryption (HE) offers a potential solution by enabling computations on encrypted data, the challenge is to obtain accurate machine learning models that work within the multiplicative depth constraints of a leveled HE scheme. Existing approaches for encrypted inference either make ad-hoc simplifications to a pre-trained model (e.g., replace hard comparisons in a decision tree with soft comparators) at the cost of accuracy or directly train a new depth-constrained model using the original training set. In this work, we propose a framework to transfer knowledge extracted by complex decision tree ensembles to shallow neural networks (referred to as DTNets) that are highly conducive to encrypted inference. Our approach minimizes the accuracy loss by searching for the best DTNet architecture that operates within the given depth constraints and training this DTNet using only synthetic data sampled from the training data distribution. Extensive experiments on real-world datasets demonstrate that these characteristics are critical in ensuring that DTNet accuracy approaches that of the original tree ensemble. Our system is highly scalable and can perform efficient inference on batched encrypted (134 bits of security) data with amortized time in milliseconds. This is approximately three orders of magnitude faster than the standard approach of applying soft comparison at the internal nodes of the ensemble trees.
△ Less
Submitted 4 March, 2021;
originally announced March 2021.
-
Differentially Private Distributed Data Summarization under Covariate Shift
Authors:
Kanthi Sarpatwar,
Karthikeyan Shanmugam,
Venkata Sitaramagiridharganesh Ganapavarapu,
Ashish Jagmohan,
Roman Vaculin
Abstract:
We envision AI marketplaces to be platforms where consumers, with very less data for a target task, can obtain a relevant model by accessing many private data sources with vast number of data samples. One of the key challenges is to construct a training dataset that matches a target task without compromising on privacy of the data sources. To this end, we consider the following distributed data su…
▽ More
We envision AI marketplaces to be platforms where consumers, with very less data for a target task, can obtain a relevant model by accessing many private data sources with vast number of data samples. One of the key challenges is to construct a training dataset that matches a target task without compromising on privacy of the data sources. To this end, we consider the following distributed data summarizataion problem. Given K private source datasets denoted by $[D_i]_{i\in [K]}$ and a small target validation set $D_v$, which may involve a considerable covariate shift with respect to the sources, compute a summary dataset $D_s\subseteq \bigcup_{i\in [K]} D_i$ such that its statistical distance from the validation dataset $D_v$ is minimized. We use the popular Maximum Mean Discrepancy as the measure of statistical distance. The non-private problem has received considerable attention in prior art, for example in prototype selection (Kim et al., NIPS 2016). Our work is the first to obtain strong differential privacy guarantees while ensuring the quality guarantees of the non-private version. We study this problem in a Parsimonious Curator Privacy Model, where a trusted curator coordinates the summarization process while minimizing the amount of private information accessed. Our central result is a novel protocol that (a) ensures the curator accesses at most $O(K^{\frac{1}{3}}|D_s| + |D_v|)$ points (b) has formal privacy guarantees on the leakage of information between the data owners and (c) closely matches the best known non-private greedy algorithm. Our protocol uses two hash functions, one inspired by the Rahimi-Recht random features method and the second leverages state of the art differential privacy mechanisms. We introduce a novel "noiseless" differentially private auctioning protocol for winner notification and demonstrate the efficacy of our protocol using real-world datasets.
△ Less
Submitted 9 January, 2020; v1 submitted 28 October, 2019;
originally announced October 2019.
-
Generalized Assignment via Submodular Optimization with Reserved Capacity
Authors:
Ariel Kulik,
Kanthi Sarpatwar,
Baruch Schieber,
Hadas Shachnai
Abstract:
We study a variant of the \emph{generalized assignment problem} ({\sf GAP}) with group constraints. An instance of {\sf Group GAP} is a set $I$ of items, partitioned into $L$ groups, and a set of $m$ uniform (unit-sized) bins. Each item $i \in I$ has a size $s_i >0$, and a profit $p_{i,j} \geq 0$ if packed in bin $j$. A group of items is \emph{satisfied} if all of its items are packed. The goal is…
▽ More
We study a variant of the \emph{generalized assignment problem} ({\sf GAP}) with group constraints. An instance of {\sf Group GAP} is a set $I$ of items, partitioned into $L$ groups, and a set of $m$ uniform (unit-sized) bins. Each item $i \in I$ has a size $s_i >0$, and a profit $p_{i,j} \geq 0$ if packed in bin $j$. A group of items is \emph{satisfied} if all of its items are packed. The goal is to find a feasible packing of a subset of the items in the bins such that the total profit from satisfied groups is maximized. We point to central applications of {\sf Group GAP} in Video-on-Demand services, mobile Device-to-Device network caching and base station cooperation in 5G networks.
Our main result is a $\frac{1}{6}$-approximation algorithm for {\sf Group GAP} instances where the total size of each group is at most $\frac{m}{2}$. At the heart of our algorithm lies an interesting derivation of a submodular function from the classic LP formulation of {\sf GAP}, which facilitates the construction of a high profit solution utilizing at most half the total bin capacity, while the other half is \emph{reserved} for later use. In particular, we give an algorithm for submodular maximization subject to a knapsack constraint, which finds a solution of profit at least $\frac{1}{3}$ of the optimum, using at most half the knapsack capacity, under mild restrictions on element sizes. Our novel approach of submodular optimization subject to a knapsack \emph{with reserved capacity} constraint may find applications in solving other group assignment problems.
△ Less
Submitted 16 September, 2019; v1 submitted 3 July, 2019;
originally announced July 2019.
-
The Preemptive Resource Allocation Problem
Authors:
Kanthi Sarpatwar,
Baruch Schieber,
Hadas Shachnai
Abstract:
We revisit a classical scheduling model to incorporate modern trends in data center networks and cloud services. Addressing some key challenges in the allocation of shared resources to user requests (jobs) in such settings, we consider the following variants of the classic {\em resource allocation problem} (\textsf{RAP}). The input to our problems is a set $J$ of jobs and a set $M$ of homogeneous…
▽ More
We revisit a classical scheduling model to incorporate modern trends in data center networks and cloud services. Addressing some key challenges in the allocation of shared resources to user requests (jobs) in such settings, we consider the following variants of the classic {\em resource allocation problem} (\textsf{RAP}). The input to our problems is a set $J$ of jobs and a set $M$ of homogeneous hosts, each has an available amount of some resource. A job is associated with a release time, a due date, a weight, and a given length, as well as its resource requirement. A \emph{feasible} schedule is an allocation of the resource to a subset of the jobs, satisfying the job release times/due dates as well as the resource constraints. A crucial distinction between classic {\textsf{RAP}} and our problems is that we allow preemption and migration of jobs, motivated by virtualization techniques.
We consider two natural objectives: {\em throughput maximization} (\textsf{MaxT}), which seeks a maximum weight subset of the jobs that can be feasibly scheduled on the hosts in $M$, and {\em resource minimization} (\textsf{MinR}), that is finding the minimum number of (homogeneous) hosts needed to feasibly schedule all jobs. Both problems are known to be NP-hard.
We first present a $Ω(1)$-approximation algorithm for \textsf{MaxT} instances where time-windows form a laminar family of intervals. We then extend the algorithm to handle instances with arbitrary time-windows, assuming there is sufficient slack for each job to be completed.
For \textsf{MinR} we study a more general setting with $d$ resources and derive an $O(\log d)$-approximation for any fixed $d \geq 1$, under the assumption that time-windows are not too small. This assumption can be removed leading to a slightly worse ratio of $O(\log d\log^* T)$, where $T$ is the maximum due date of any job.
△ Less
Submitted 18 November, 2018;
originally announced November 2018.
-
Constrained Submodular Maximization via Greedy Local Search
Authors:
Kanthi K. Sarpatwar,
Baruch Schieber,
Hadas Shachnai
Abstract:
We present a simple combinatorial $\frac{1 -e^{-2}}{2}$-approximation algorithm for maximizing a monotone submodular function subject to a knapsack and a matroid constraint.
This classic problem is known to be hard to approximate within factor better than $1 - 1/e$. We show that the algorithm can be extended to yield a ratio of $\frac{1 - e^{-(k+1)}}{k+1}$ for the problem with a single knapsack…
▽ More
We present a simple combinatorial $\frac{1 -e^{-2}}{2}$-approximation algorithm for maximizing a monotone submodular function subject to a knapsack and a matroid constraint.
This classic problem is known to be hard to approximate within factor better than $1 - 1/e$. We show that the algorithm can be extended to yield a ratio of $\frac{1 - e^{-(k+1)}}{k+1}$ for the problem with a single knapsack and the intersection of $k$ matroid constraints, for any fixed $k > 1$.
Our algorithms, which combine the greedy algorithm of [Khuller, Moss and Naor, 1999] and [Sviridenko, 2004] with local search, show the power of this natural framework in submodular maximization with combined constraints.
△ Less
Submitted 14 January, 2018; v1 submitted 17 May, 2017;
originally announced May 2017.
-
Approximation Algorithms for Connected Maximum Cut and Related Problems
Authors:
MohammadTaghi Hajiaghayi,
Guy Kortsarz,
Robert MacDavid,
Manish Purohit,
Kanthi Sarpatwar
Abstract:
An instance of the Connected Maximum Cut problem consists of an undirected graph G = (V, E) and the goal is to find a subset of vertices S $\subseteq$ V that maximizes the number of edges in the cut δ(S) such that the induced graph G[S] is connected. We present the first non-trivial Ω(1/log n) approximation algorithm for the connected maximum cut problem in general graphs using novel techniques. W…
▽ More
An instance of the Connected Maximum Cut problem consists of an undirected graph G = (V, E) and the goal is to find a subset of vertices S $\subseteq$ V that maximizes the number of edges in the cut δ(S) such that the induced graph G[S] is connected. We present the first non-trivial Ω(1/log n) approximation algorithm for the connected maximum cut problem in general graphs using novel techniques. We then extend our algorithm to an edge weighted case and obtain a poly-logarithmic approximation algorithm. Interestingly, in stark contrast to the classical max-cut problem, we show that the connected maximum cut problem remains NP-hard even on unweighted, planar graphs. On the positive side, we obtain a polynomial time approximation scheme for the connected maximum cut problem on planar graphs and more generally on graphs with bounded genus.
△ Less
Submitted 2 July, 2015;
originally announced July 2015.
-
Analyzing the Optimal Neighborhood: Algorithms for Budgeted and Partial Connected Dominating Set Problems
Authors:
Samir Khuller,
Manish Purohit,
Kanthi Sarpatwar
Abstract:
We study partial and budgeted versions of the well studied connected dominating set problem. In the partial connected dominating set problem, we are given an undirected graph G = (V,E) and an integer n', and the goal is to find a minimum subset of vertices that induces a connected subgraph of G and dominates at least n' vertices. We obtain the first polynomial time algorithm with an O(\ln Δ) appro…
▽ More
We study partial and budgeted versions of the well studied connected dominating set problem. In the partial connected dominating set problem, we are given an undirected graph G = (V,E) and an integer n', and the goal is to find a minimum subset of vertices that induces a connected subgraph of G and dominates at least n' vertices. We obtain the first polynomial time algorithm with an O(\ln Δ) approximation factor for this problem, thereby significantly extending the results of Guha and Khuller (Algorithmica, Vol. 20(4), Pages 374-387, 1998) for the connected dominating set problem. We note that none of the methods developed earlier can be applied directly to solve this problem. In the budgeted connected dominating set problem, there is a budget on the number of vertices we can select, and the goal is to dominate as many vertices as possible. We obtain a (1/13)(1 - 1/e) approximation algorithm for this problem. Finally, we show that our techniques extend to a more general setting where the profit function associated with a subset of vertices is a monotone "special" submodular function. This generalization captures the connected dominating set problem with capacities and/or weighted profits as special cases. This implies a O(\ln q) approximation (where q denotes the quota) and an O(1) approximation algorithms for the partial and budgeted versions of these problems. While the algorithms are simple, the results make a surprising use of the greedy set cover framework in defining a useful profit function.
△ Less
Submitted 10 November, 2013;
originally announced November 2013.
-
Hardness and Parameterized Algorithms on Rainbow Connectivity problem
Authors:
Prabhanjan Ananth,
Meghana Nasre,
Kanthi K Sarpatwar
Abstract:
A path in an edge colored graph is said to be a rainbow path if no two edges on the path have the same color. An edge colored graph is (strongly) rainbow connected if there exists a (geodesic) rainbow path between every pair of vertices. The (strong) rainbow connectivity of a graph G, denoted by (src(G), respectively) rc(G) is the smallest number of colors required to edge color the graph such tha…
▽ More
A path in an edge colored graph is said to be a rainbow path if no two edges on the path have the same color. An edge colored graph is (strongly) rainbow connected if there exists a (geodesic) rainbow path between every pair of vertices. The (strong) rainbow connectivity of a graph G, denoted by (src(G), respectively) rc(G) is the smallest number of colors required to edge color the graph such that G is (strongly) rainbow connected. In this paper we study the rainbow connectivity problem and the strong rainbow connectivity problem from a computational point of view. Our main results can be summarised as below:
1) For every fixed k >= 3, it is NP-Complete to decide whether src(G) <= k even when the graph G is bipartite.
2) For every fixed odd k >= 3, it is NP-Complete to decide whether rc(G) <= k. This resolves one of the open problems posed by Chakraborty et al. (J. Comb. Opt., 2011) where they prove the hardness for the even case.
3) The following problem is fixed parameter tractable: Given a graph G, determine the maximum number of pairs of vertices that can be rainbow connected using two colors.
4) For a directed graph G, it is NP-Complete to decide whether rc(G) <= 2.
△ Less
Submitted 7 October, 2011; v1 submitted 5 May, 2011;
originally announced May 2011.
