-
OpenAirLink: Reproducible Wireless Channel Emulation using Software Defined Radios
Authors:
Yash Deshpande,
Xianglong Wang,
Wolfgang Kellerer
Abstract:
This paper presents OpenAirLink(OAL), an open-source channel emulator for reproducible testing of wireless scenarios. OAL is implemented on off-the-shelf software-defined radios (SDR) and presents a smaller-scale alternative to expensive commercially available channel emulators. Path loss and propagation delay are the fundamental aspects of emulating a wireless channel. OAL provides a simple metho…
▽ More
This paper presents OpenAirLink(OAL), an open-source channel emulator for reproducible testing of wireless scenarios. OAL is implemented on off-the-shelf software-defined radios (SDR) and presents a smaller-scale alternative to expensive commercially available channel emulators. Path loss and propagation delay are the fundamental aspects of emulating a wireless channel. OAL provides a simple method to change these aspects in real-time. The emulator is implemented using a finite impulse response (FIR) filter. The FIR filter is written in Verilog and flashed on the SDRs Field Programmable Gate Array (FPGA). Most processing transpires on the FPGA, so OAL does not require high-performance computing hardware and SDRs. We validate the performance of OAL and demonstrate the utility of such a channel emulation tool using two examples. We believe that open-source channel emulators such as OAL can make reproducible wireless experiments accessible to many researchers in the scientific community.
△ Less
Submitted 15 April, 2024;
originally announced April 2024.
-
Evaluation of NR-Sidelink for Cooperative Industrial AGVs
Authors:
Shubhangi Bhadauria,
Klea Plaku,
Yash Deshpande,
Wolfgang Kellerer
Abstract:
Industry 4.0 has brought to attention the need for a connected, flexible, and autonomous production environment. The New Radio (NR)-sidelink, which was introduced by the third-generation partnership project (3GPP) in Release 16, can be particularly helpful for factories that need to facilitate cooperative and close-range communication. Automated Guided Vehicles (AGVs) are important for material ha…
▽ More
Industry 4.0 has brought to attention the need for a connected, flexible, and autonomous production environment. The New Radio (NR)-sidelink, which was introduced by the third-generation partnership project (3GPP) in Release 16, can be particularly helpful for factories that need to facilitate cooperative and close-range communication. Automated Guided Vehicles (AGVs) are important for material handling and carriage within these environments, and using NR-sidelink communication can further enhance their performance. An efficient resource allocation mechanism is required to ensure reliable communication and avoid interference between AGVs and other wireless systems in the factory using NR-sidelink. This work evaluates the 3GPP standardized resource allocation algorithm for NR-sidelink for a use case of cooperative carrying AGVs. We suggest further improvements that are tailored to the quality of service (QoS) requirements of an indoor factory communication scenario with cooperative AGVs.The use of NR-sidelink communication has the potential to help meet the QoS requirements for different Industry 4.0 use cases. This work can be a foundation for further improvements in NR-sidelink in 3GPP Release 18 and beyond.
△ Less
Submitted 6 September, 2023;
originally announced September 2023.
-
An Advanced Tree Algorithm with Interference Cancellation in Uplink and Downlink
Authors:
Quirin Vogel,
Yash Deshpande,
Čedomir Stefanović,
Wolfgang Kellerer
Abstract:
In this paper, we propose Advanced Tree-algorithm with Interference Cancellation (ATIC), a variant of binary tree-algorithm with successive interference cancellation (SICTA) introduced by Yu and Giannakis. ATIC assumes that Interference Cancellation (IC) can be performed both by the access point (AP), as in SICTA, but also by the users. Specifically, after every collision slot, the AP broadcasts t…
▽ More
In this paper, we propose Advanced Tree-algorithm with Interference Cancellation (ATIC), a variant of binary tree-algorithm with successive interference cancellation (SICTA) introduced by Yu and Giannakis. ATIC assumes that Interference Cancellation (IC) can be performed both by the access point (AP), as in SICTA, but also by the users. Specifically, after every collision slot, the AP broadcasts the observed collision as feedback. Users who participated in the collision then attempt to perform IC by subtracting their transmissions from the collision signal. This way, the users can resolve collisions of degree 2 and, using a simple distributed arbitration algorithm based on user IDs, ensure that the next slot will contain just a single transmission. We show that ATIC reaches the asymptotic throughput of 0.924 as the number of initially collided users tends to infinity and reduces the number of collisions and packet delay. We also compare ATIC with other tree algorithms and indicate the extra feedback resources it requires.
△ Less
Submitted 22 November, 2023; v1 submitted 29 August, 2023;
originally announced August 2023.
-
Analysis of d-ary Tree Algorithms with Successive Interference Cancellation
Authors:
Quirin Vogel,
Yash Deshpande,
Cedomir Stefanović,
Wolfgang Kellerer
Abstract:
In this article, we calculate the mean throughput, number of collisions, successes, and idle slots for random tree algorithms with successive interference cancellation. Except for the case of the throughput for the binary tree, all the results are new. We furthermore disprove the claim that only the binary tree maximises throughput. Our method works with many observables and can be used as a bluep…
▽ More
In this article, we calculate the mean throughput, number of collisions, successes, and idle slots for random tree algorithms with successive interference cancellation. Except for the case of the throughput for the binary tree, all the results are new. We furthermore disprove the claim that only the binary tree maximises throughput. Our method works with many observables and can be used as a blueprint for further analysis.
△ Less
Submitted 16 February, 2023;
originally announced February 2023.
-
On d-ary tree algorithms with successive interference cancellation
Authors:
Yash Deshpande,
Cedomir Stefanovic,
H. Murat Gürsu,
Wolfgang Kellerer
Abstract:
In this paper, we outline the approach for the derivation of the length of the collision resolution interval for d-ary tree algorithms (TA) with gated access and successive interference cancellation (SIC), conditioned on the number of the contending users. This is the basic performance parameter for TA with gated access. We identify the deficiencies of the analysis performed in the seminal paper o…
▽ More
In this paper, we outline the approach for the derivation of the length of the collision resolution interval for d-ary tree algorithms (TA) with gated access and successive interference cancellation (SIC), conditioned on the number of the contending users. This is the basic performance parameter for TA with gated access. We identify the deficiencies of the analysis performed in the seminal paper on TA with SIC by Yu and Giannakis, showing that their analysis is correct only for binary splitting, i.e. for d=2. We also provide some insightful results on the stable throughput that can be achieved for different values of d.
△ Less
Submitted 8 August, 2022;
originally announced August 2022.
-
Endowing Language Models with Multimodal Knowledge Graph Representations
Authors:
Ningyuan Huang,
Yash R. Deshpande,
Yibo Liu,
Houda Alberts,
Kyunghyun Cho,
Clara Vania,
Iacer Calixto
Abstract:
We propose a method to make natural language understanding models more parameter efficient by storing knowledge in an external knowledge graph (KG) and retrieving from this KG using a dense index. Given (possibly multilingual) downstream task data, e.g., sentences in German, we retrieve entities from the KG and use their multimodal representations to improve downstream task performance. We use the…
▽ More
We propose a method to make natural language understanding models more parameter efficient by storing knowledge in an external knowledge graph (KG) and retrieving from this KG using a dense index. Given (possibly multilingual) downstream task data, e.g., sentences in German, we retrieve entities from the KG and use their multimodal representations to improve downstream task performance. We use the recently released VisualSem KG as our external knowledge repository, which covers a subset of Wikipedia and WordNet entities, and compare a mix of tuple-based and graph-based algorithms to learn entity and relation representations that are grounded on the KG multimodal information. We demonstrate the usefulness of the learned entity representations on two downstream tasks, and show improved performance on the multilingual named entity recognition task by $0.3\%$--$0.7\%$ F1, while we achieve up to $2.5\%$ improvement in accuracy on the visual sense disambiguation task. All our code and data are available in: \url{https://github.com/iacercalixto/visualsem-kg}.
△ Less
Submitted 27 June, 2022;
originally announced June 2022.
-
Agreement and Statistical Efficiency in Bayesian Perception Models
Authors:
Yash Deshpande,
Elchanan Mossel,
Youngtak Sohn
Abstract:
Bayesian models of group learning are studied in Economics since the 1970s. and more recently in computational linguistics. The models from Economics postulate that agents maximize utility in their communication and actions. The Economics models do not explain the ``probability matching" phenomena that are observed in many experimental studies. To address these observations, Bayesian models that d…
▽ More
Bayesian models of group learning are studied in Economics since the 1970s. and more recently in computational linguistics. The models from Economics postulate that agents maximize utility in their communication and actions. The Economics models do not explain the ``probability matching" phenomena that are observed in many experimental studies. To address these observations, Bayesian models that do not formally fit into the economic utility maximization framework were introduced. In these models individuals sample from their posteriors in communication. In this work we study the asymptotic behavior of such models on connected networks with repeated communication. Perhaps surprisingly, despite the fact that individual agents are not utility maximizers in the classical sense, we establish that the individuals ultimately agree and furthermore show that the limiting posterior is Bayes optimal.
We explore the interpretation of our results in terms of Large Language Models (LLMs). In the positive direction our results can be interpreted as stating that interaction between different LLMs can lead to optimal learning. However, we provide an example showing how misspecification may lead LLM agents to be overconfident in their estimates.
△ Less
Submitted 9 August, 2023; v1 submitted 23 May, 2022;
originally announced May 2022.
-
Improving AoI via Learning-based Distributed MAC in Wireless Networks
Authors:
Yash Deshpande,
Onur Ayan,
Wolfgang Kellerer
Abstract:
In this work, we consider a remote monitoring scenario in which multiple sensors share a wireless channel to deliver their status updates to a process monitor via an access point (AP). Moreover, we consider that the sensors randomly arrive and depart from the network as they become active and inactive. The goal of the sensors is to devise a medium access strategy to collectively minimize the long-…
▽ More
In this work, we consider a remote monitoring scenario in which multiple sensors share a wireless channel to deliver their status updates to a process monitor via an access point (AP). Moreover, we consider that the sensors randomly arrive and depart from the network as they become active and inactive. The goal of the sensors is to devise a medium access strategy to collectively minimize the long-term mean network \ac{AoI} of their respective processes at the remote monitor. For this purpose, we propose specific modifications to ALOHA-QT algorithm, a distributed medium access algorithm that employs a policy tree (PT) and reinforcement learning (RL) to achieve high throughput. We provide the upper bound on the mean network Age of Information (AoI) for the proposed algorithm along with pointers for selecting its key parameter. The results reveal that the proposed algorithm reduces mean network \ac{AoI} by more than 50 percent for state of the art stationary randomized policies while successfully adjusting to a changing number of active users in the network. The algorithm needs less memory and computation than ALOHA-QT while performing better in terms of AoI.
△ Less
Submitted 18 February, 2022;
originally announced February 2022.
-
Tree-Algorithms with Multi-Packet Reception and Successive Interference Cancellation
Authors:
Cedomir Stefanović,
Yash Deshpande,
H. Murat Gürsu,
Wolfgang Kellerer
Abstract:
In this paper, we perform a thorough analysis of tree-algorithms with multi-packet reception (MPR) and successive interference cancellation (SIC), showing a number of novel results. We first derive the basic performance parameters, which are the expected length of the collision resolution interval and the normalized throughput, conditioned on the number of contending users. We then study their asy…
▽ More
In this paper, we perform a thorough analysis of tree-algorithms with multi-packet reception (MPR) and successive interference cancellation (SIC), showing a number of novel results. We first derive the basic performance parameters, which are the expected length of the collision resolution interval and the normalized throughput, conditioned on the number of contending users. We then study their asymptotic behaviour, identifying an oscillatory component that amplifies with the increase in MPR. In the next step, we derive the throughput for the gated and windowed access, assuming Poisson arrivals. We show that for windowed access, the bound on maximum stable normalized throughput increases with the increase in MPR. his implies that investing in advanced physical capabilities, i.e., MPR and SIC pays off from the perspective of the medium access control algorithm.
△ Less
Submitted 10 August, 2022; v1 submitted 2 August, 2021;
originally announced August 2021.
-
Near-optimal inference in adaptive linear regression
Authors:
Koulik Khamaru,
Yash Deshpande,
Tor Lattimore,
Lester Mackey,
Martin J. Wainwright
Abstract:
When data is collected in an adaptive manner, even simple methods like ordinary least squares can exhibit non-normal asymptotic behavior. As an undesirable consequence, hypothesis tests and confidence intervals based on asymptotic normality can lead to erroneous results. We propose a family of online debiasing estimators to correct these distributional anomalies in least squares estimation. Our pr…
▽ More
When data is collected in an adaptive manner, even simple methods like ordinary least squares can exhibit non-normal asymptotic behavior. As an undesirable consequence, hypothesis tests and confidence intervals based on asymptotic normality can lead to erroneous results. We propose a family of online debiasing estimators to correct these distributional anomalies in least squares estimation. Our proposed methods take advantage of the covariance structure present in the dataset and provide sharper estimates in directions for which more information has accrued. We establish an asymptotic normality property for our proposed online debiasing estimators under mild conditions on the data collection process and provide asymptotically exact confidence intervals. We additionally prove a minimax lower bound for the adaptive linear regression problem, thereby providing a baseline by which to compare estimators. There are various conditions under which our proposed estimators achieve the minimax lower bound. We demonstrate the usefulness of our theory via applications to multi-armed bandit, autoregressive time series estimation, and active learning with exploration.
△ Less
Submitted 21 March, 2023; v1 submitted 5 July, 2021;
originally announced July 2021.
-
VisualSem: A High-quality Knowledge Graph for Vision and Language
Authors:
Houda Alberts,
Teresa Huang,
Yash Deshpande,
Yibo Liu,
Kyunghyun Cho,
Clara Vania,
Iacer Calixto
Abstract:
An exciting frontier in natural language understanding (NLU) and generation (NLG) calls for (vision-and-) language models that can efficiently access external structured knowledge repositories. However, many existing knowledge bases only cover limited domains, or suffer from noisy data, and most of all are typically hard to integrate into neural language pipelines. To fill this gap, we release Vis…
▽ More
An exciting frontier in natural language understanding (NLU) and generation (NLG) calls for (vision-and-) language models that can efficiently access external structured knowledge repositories. However, many existing knowledge bases only cover limited domains, or suffer from noisy data, and most of all are typically hard to integrate into neural language pipelines. To fill this gap, we release VisualSem: a high-quality knowledge graph (KG) which includes nodes with multilingual glosses, multiple illustrative images, and visually relevant relations. We also release a neural multi-modal retrieval model that can use images or sentences as inputs and retrieves entities in the KG. This multi-modal retrieval model can be integrated into any (neural network) model pipeline. We encourage the research community to use VisualSem for data augmentation and/or as a source of grounding, among other possible uses. VisualSem as well as the multi-modal retrieval models are publicly available and can be downloaded in this URL: https://github.com/iacercalixto/visualsem
△ Less
Submitted 20 October, 2021; v1 submitted 20 August, 2020;
originally announced August 2020.
-
Analysis of Tree-Algorithms with Multi-Packet Reception
Authors:
Cedomir Stefanović,
H. Murat Gürsu,
Yash Deshpande,
Wolfgang Kellerer
Abstract:
In this paper, we analyze binary-tree algorithms in a setup in which the receiver can perform multi-packet reception (MPR) of up to and including K packets simultaneously. The analysis addresses both traffic-independent performance as well as performance under Poisson arrivals. For the former case, we show that the throughput, when normalized with respect to the assumed linear increase in resource…
▽ More
In this paper, we analyze binary-tree algorithms in a setup in which the receiver can perform multi-packet reception (MPR) of up to and including K packets simultaneously. The analysis addresses both traffic-independent performance as well as performance under Poisson arrivals. For the former case, we show that the throughput, when normalized with respect to the assumed linear increase in resources required to achieve K-MPR capability, tends to the same value that holds for the single-reception setup. However, when coupled with Poisson arrivals in the windowed access scheme, the normalized throughput increases with K, and we present evidence that it asymptotically tends to 1. We also provide performance results for the modified tree algorithm with K-MPR in the clipped access scheme. To the best of our knowledge, this is the first paper that provides an analytical treatment and a number of fundamental insights in the performance of tree-algorithms with MPR.
△ Less
Submitted 4 February, 2021; v1 submitted 28 May, 2020;
originally announced May 2020.
-
Online Debiasing for Adaptively Collected High-dimensional Data with Applications to Time Series Analysis
Authors:
Yash Deshpande,
Adel Javanmard,
Mohammad Mehrabi
Abstract:
Adaptive collection of data is commonplace in applications throughout science and engineering. From the point of view of statistical inference however, adaptive data collection induces memory and correlation in the samples, and poses significant challenge. We consider the high-dimensional linear regression, where the samples are collected adaptively, and the sample size $n$ can be smaller than…
▽ More
Adaptive collection of data is commonplace in applications throughout science and engineering. From the point of view of statistical inference however, adaptive data collection induces memory and correlation in the samples, and poses significant challenge. We consider the high-dimensional linear regression, where the samples are collected adaptively, and the sample size $n$ can be smaller than $p$, the number of covariates. In this setting, there are two distinct sources of bias: the first due to regularization imposed for consistent estimation, e.g. using the LASSO, and the second due to adaptivity in collecting the samples. We propose "online debiasing", a general procedure for estimators such as the LASSO, which addresses both sources of bias. In two concrete contexts $(i)$ time series analysis and $(ii)$ batched data collection, we demonstrate that online debiasing optimally debiases the LASSO estimate when the underlying parameter $θ_0$ has sparsity of order $o(\sqrt{n}/\log p)$. In this regime, the debiased estimator can be used to compute $p$-values and confidence intervals of optimal size.
△ Less
Submitted 5 May, 2020; v1 submitted 4 November, 2019;
originally announced November 2019.
-
Contextual Stochastic Block Models
Authors:
Yash Deshpande,
Andrea Montanari,
Elchanan Mossel,
Subhabrata Sen
Abstract:
We provide the first information theoretic tight analysis for inference of latent community structure given a sparse graph along with high dimensional node covariates, correlated with the same latent communities. Our work bridges recent theoretical breakthroughs in the detection of latent community structure without nodes covariates and a large body of empirical work using diverse heuristics for c…
▽ More
We provide the first information theoretic tight analysis for inference of latent community structure given a sparse graph along with high dimensional node covariates, correlated with the same latent communities. Our work bridges recent theoretical breakthroughs in the detection of latent community structure without nodes covariates and a large body of empirical work using diverse heuristics for combining node covariates with graphs for inference. The tightness of our analysis implies in particular, the information theoretical necessity of combining the different sources of information. Our analysis holds for networks of large degrees as well as for a Gaussian version of the model.
△ Less
Submitted 22 July, 2018;
originally announced July 2018.
-
The threshold for SDP-refutation of random regular NAE-3SAT
Authors:
Yash Deshpande,
Andrea Montanari,
Ryan O'Donnell,
Tselil Schramm,
Subhabrata Sen
Abstract:
Unlike its cousin 3SAT, the NAE-3SAT (not-all-equal-3SAT) problem has the property that spectral/SDP algorithms can efficiently refute random instances when the constraint density is a large constant (with high probability). But do these methods work immediately above the "satisfiability threshold", or is there still a range of constraint densities for which random NAE-3SAT instances are unsatisfi…
▽ More
Unlike its cousin 3SAT, the NAE-3SAT (not-all-equal-3SAT) problem has the property that spectral/SDP algorithms can efficiently refute random instances when the constraint density is a large constant (with high probability). But do these methods work immediately above the "satisfiability threshold", or is there still a range of constraint densities for which random NAE-3SAT instances are unsatisfiable but hard to refute?
We show that the latter situation prevails, at least in the context of random regular instances and SDP-based refutation. More precisely, whereas a random $d$-regular instance of NAE-3SAT is easily shown to be unsatisfiable (whp) once $d \geq 8$, we establish the following sharp threshold result regarding efficient refutation: If $d < 13.5$ then the basic SDP, even augmented with triangle inequalities, fails to refute satisfiability (whp), if $d > 13.5$ then even the most basic spectral algorithm refutes satisfiability~(whp).
△ Less
Submitted 14 April, 2018;
originally announced April 2018.
-
Accurate Inference for Adaptive Linear Models
Authors:
Yash Deshpande,
Lester Mackey,
Vasilis Syrgkanis,
Matt Taddy
Abstract:
Estimators computed from adaptively collected data do not behave like their non-adaptive brethren. Rather, the sequential dependence of the collection policy can lead to severe distributional biases that persist even in the infinite data limit. We develop a general method -- $\mathbf{W}$-decorrelation -- for transforming the bias of adaptive linear regression estimators into variance. The method u…
▽ More
Estimators computed from adaptively collected data do not behave like their non-adaptive brethren. Rather, the sequential dependence of the collection policy can lead to severe distributional biases that persist even in the infinite data limit. We develop a general method -- $\mathbf{W}$-decorrelation -- for transforming the bias of adaptive linear regression estimators into variance. The method uses only coarse-grained information about the data collection policy and does not need access to propensity scores or exact knowledge of the policy. We bound the finite-sample bias and variance of the $\mathbf{W}$-estimator and develop asymptotically correct confidence intervals based on a novel martingale central limit theorem. We then demonstrate the empirical benefits of the generic $\mathbf{W}$-decorrelation procedure in two different adaptive data settings: the multi-armed bandit and the autoregressive time series.
△ Less
Submitted 2 January, 2020; v1 submitted 18 December, 2017;
originally announced December 2017.
-
Inference in Graphical Models via Semidefinite Programming Hierarchies
Authors:
Murat A. Erdogdu,
Yash Deshpande,
Andrea Montanari
Abstract:
Maximum A posteriori Probability (MAP) inference in graphical models amounts to solving a graph-structured combinatorial optimization problem. Popular inference algorithms such as belief propagation (BP) and generalized belief propagation (GBP) are intimately related to linear programming (LP) relaxation within the Sherali-Adams hierarchy. Despite the popularity of these algorithms, it is well und…
▽ More
Maximum A posteriori Probability (MAP) inference in graphical models amounts to solving a graph-structured combinatorial optimization problem. Popular inference algorithms such as belief propagation (BP) and generalized belief propagation (GBP) are intimately related to linear programming (LP) relaxation within the Sherali-Adams hierarchy. Despite the popularity of these algorithms, it is well understood that the Sum-of-Squares (SOS) hierarchy based on semidefinite programming (SDP) can provide superior guarantees. Unfortunately, SOS relaxations for a graph with $n$ vertices require solving an SDP with $n^{Θ(d)}$ variables where $d$ is the degree in the hierarchy. In practice, for $d\ge 4$, this approach does not scale beyond a few tens of variables. In this paper, we propose binary SDP relaxations for MAP inference using the SOS hierarchy with two innovations focused on computational efficiency. Firstly, in analogy to BP and its variants, we only introduce decision variables corresponding to contiguous regions in the graphical model. Secondly, we solve the resulting SDP using a non-convex Burer-Monteiro style method, and develop a sequential rounding procedure. We demonstrate that the resulting algorithm can solve problems with tens of thousands of variables within minutes, and outperforms BP and GBP on practical problems such as image denoising and Ising spin glasses. Finally, for specific graph types, we establish a sufficient condition for the tightness of the proposed partial SOS relaxation.
△ Less
Submitted 19 September, 2017;
originally announced September 2017.
-
Asymptotic Mutual Information for the Two-Groups Stochastic Block Model
Authors:
Yash Deshpande,
Emmanuel Abbe,
Andrea Montanari
Abstract:
We develop an information-theoretic view of the stochastic block model, a popular statistical model for the large-scale structure of complex networks. A graph $G$ from such a model is generated by first assigning vertex labels at random from a finite alphabet, and then connecting vertices with edge probabilities depending on the labels of the endpoints. In the case of the symmetric two-group model…
▽ More
We develop an information-theoretic view of the stochastic block model, a popular statistical model for the large-scale structure of complex networks. A graph $G$ from such a model is generated by first assigning vertex labels at random from a finite alphabet, and then connecting vertices with edge probabilities depending on the labels of the endpoints. In the case of the symmetric two-group model, we establish an explicit `single-letter' characterization of the per-vertex mutual information between the vertex labels and the graph.
The explicit expression of the mutual information is intimately related to estimation-theoretic quantities, and --in particular-- reveals a phase transition at the critical point for community detection. Below the critical point the per-vertex mutual information is asymptotically the same as if edges were independent. Correspondingly, no algorithm can estimate the partition better than random guessing. Conversely, above the threshold, the per-vertex mutual information is strictly smaller than the independent-edges upper bound. In this regime there exists a procedure that estimates the vertex labels better than random guessing.
△ Less
Submitted 30 July, 2015;
originally announced July 2015.
-
Improved Sum-of-Squares Lower Bounds for Hidden Clique and Hidden Submatrix Problems
Authors:
Yash Deshpande,
Andrea Montanari
Abstract:
Given a large data matrix $A\in\mathbb{R}^{n\times n}$, we consider the problem of determining whether its entries are i.i.d. with some known marginal distribution $A_{ij}\sim P_0$, or instead $A$ contains a principal submatrix $A_{{\sf Q},{\sf Q}}$ whose entries have marginal distribution $A_{ij}\sim P_1\neq P_0$. As a special case, the hidden (or planted) clique problem requires to find a plante…
▽ More
Given a large data matrix $A\in\mathbb{R}^{n\times n}$, we consider the problem of determining whether its entries are i.i.d. with some known marginal distribution $A_{ij}\sim P_0$, or instead $A$ contains a principal submatrix $A_{{\sf Q},{\sf Q}}$ whose entries have marginal distribution $A_{ij}\sim P_1\neq P_0$. As a special case, the hidden (or planted) clique problem requires to find a planted clique in an otherwise uniformly random graph.
Assuming unbounded computational resources, this hypothesis testing problem is statistically solvable provided $|{\sf Q}|\ge C \log n$ for a suitable constant $C$. However, despite substantial effort, no polynomial time algorithm is known that succeeds with high probability when $|{\sf Q}| = o(\sqrt{n})$. Recently Meka and Wigderson \cite{meka2013association}, proposed a method to establish lower bounds within the Sum of Squares (SOS) semidefinite hierarchy.
Here we consider the degree-$4$ SOS relaxation, and study the construction of \cite{meka2013association} to prove that SOS fails unless $k\ge C\, n^{1/3}/\log n$. An argument presented by Barak implies that this lower bound cannot be substantially improved unless the witness construction is changed in the proof. Our proof uses the moments method to bound the spectrum of a certain random association scheme, i.e. a symmetric random matrix whose rows and columns are indexed by the edges of an Erdös-Renyi random graph.
△ Less
Submitted 23 February, 2015;
originally announced February 2015.
-
Information-theoretically Optimal Sparse PCA
Authors:
Yash Deshpande,
Andrea Montanari
Abstract:
Sparse Principal Component Analysis (PCA) is a dimensionality reduction technique wherein one seeks a low-rank representation of a data matrix with additional sparsity constraints on the obtained representation. We consider two probabilistic formulations of sparse PCA: a spiked Wigner and spiked Wishart (or spiked covariance) model. We analyze an Approximate Message Passing (AMP) algorithm to esti…
▽ More
Sparse Principal Component Analysis (PCA) is a dimensionality reduction technique wherein one seeks a low-rank representation of a data matrix with additional sparsity constraints on the obtained representation. We consider two probabilistic formulations of sparse PCA: a spiked Wigner and spiked Wishart (or spiked covariance) model. We analyze an Approximate Message Passing (AMP) algorithm to estimate the underlying signal and show, in the high dimensional limit, that the AMP estimates are information-theoretically optimal. As an immediate corollary, our results demonstrate that the posterior expectation of the underlying signal, which is often intractable to compute, can be obtained using a polynomial-time scheme. Our results also effectively provide a single-letter characterization of the sparse PCA problem.
△ Less
Submitted 3 May, 2014; v1 submitted 10 February, 2014;
originally announced February 2014.
-
Sparse PCA via Covariance Thresholding
Authors:
Yash Deshpande,
Andrea Montanari
Abstract:
In sparse principal component analysis we are given noisy observations of a low-rank matrix of dimension $n\times p$ and seek to reconstruct it under additional sparsity assumptions. In particular, we assume here each of the principal components $\mathbf{v}_1,\dots,\mathbf{v}_r$ has at most $s_0$ non-zero entries. We are particularly interested in the high dimensional regime wherein $p$ is compara…
▽ More
In sparse principal component analysis we are given noisy observations of a low-rank matrix of dimension $n\times p$ and seek to reconstruct it under additional sparsity assumptions. In particular, we assume here each of the principal components $\mathbf{v}_1,\dots,\mathbf{v}_r$ has at most $s_0$ non-zero entries. We are particularly interested in the high dimensional regime wherein $p$ is comparable to, or even much larger than $n$. In an influential paper, \cite{johnstone2004sparse} introduced a simple algorithm that estimates the support of the principal vectors $\mathbf{v}_1,\dots,\mathbf{v}_r$ by the largest entries in the diagonal of the empirical covariance. This method can be shown to identify the correct support with high probability if $s_0\le K_1\sqrt{n/\log p}$, and to fail with high probability if $s_0\ge K_2 \sqrt{n/\log p}$ for two constants $0<K_1,K_2<\infty$. Despite a considerable amount of work over the last ten years, no practical algorithm exists with provably better support recovery guarantees.
Here we analyze a covariance thresholding algorithm that was recently proposed by \cite{KrauthgamerSPCA}. On the basis of numerical simulations (for the rank-one case), these authors conjectured that covariance thresholding correctly recover the support with high probability for $s_0\le K\sqrt{n}$ (assuming $n$ of the same order as $p$). We prove this conjecture, and in fact establish a more general guarantee including higher-rank as well as $n$ much smaller than $p$. Recent lower bounds \cite{berthet2013computational, ma2015sum} suggest that no polynomial time algorithm can do significantly better. The key technical component of our analysis develops new bounds on the norm of kernel random matrices, in regimes that were not considered before.
△ Less
Submitted 25 April, 2016; v1 submitted 20 November, 2013;
originally announced November 2013.
-
Finding Hidden Cliques of Size \sqrt{N/e} in Nearly Linear Time
Authors:
Yash Deshpande,
Andrea Montanari
Abstract:
Consider an Erdös-Renyi random graph in which each edge is present independently with probability 1/2, except for a subset $\sC_N$ of the vertices that form a clique (a completely connected subgraph). We consider the problem of identifying the clique, given a realization of such a random graph.
The best known algorithm provably finds the clique in linear time with high probability, provided…
▽ More
Consider an Erdös-Renyi random graph in which each edge is present independently with probability 1/2, except for a subset $\sC_N$ of the vertices that form a clique (a completely connected subgraph). We consider the problem of identifying the clique, given a realization of such a random graph.
The best known algorithm provably finds the clique in linear time with high probability, provided $|\sC_N|\ge 1.261\sqrt{N}$ \cite{dekel2011finding}. Spectral methods can be shown to fail on cliques smaller than $\sqrt{N}$. In this paper we describe a nearly linear time algorithm that succeeds with high probability for $|\sC_N|\ge (1+\eps)\sqrt{N/e}$ for any $\eps>0$. This is the first algorithm that provably improves over spectral methods.
We further generalize the hidden clique problem to other background graphs (the standard case corresponding to the complete graph on $N$ vertices). For large girth regular graphs of degree $(Δ+1)$ we prove that `local' algorithms succeed if $|\sC_N|\ge (1+\eps)N/\sqrt{eΔ}$ and fail if $|\sC_N|\le(1-\eps)N/\sqrt{eΔ}$.
△ Less
Submitted 25 April, 2013;
originally announced April 2013.
-
Linear Bandits in High Dimension and Recommendation Systems
Authors:
Yash Deshpande,
Andrea Montanari
Abstract:
A large number of online services provide automated recommendations to help users to navigate through a large collection of items. New items (products, videos, songs, advertisements) are suggested on the basis of the user's past history and --when available-- her demographic profile. Recommendations have to satisfy the dual goal of hel** the user to explore the space of available items, while al…
▽ More
A large number of online services provide automated recommendations to help users to navigate through a large collection of items. New items (products, videos, songs, advertisements) are suggested on the basis of the user's past history and --when available-- her demographic profile. Recommendations have to satisfy the dual goal of hel** the user to explore the space of available items, while allowing the system to probe the user's preferences.
We model this trade-off using linearly parametrized multi-armed bandits, propose a policy and prove upper and lower bounds on the cumulative "reward" that coincide up to constants in the data poor (high-dimensional) regime. Prior work on linear bandits has focused on the data rich (low-dimensional) regime and used cumulative "risk" as the figure of merit. For this data rich regime, we provide a simple modification for our policy that achieves near-optimal risk performance under more restrictive assumptions on the geometry of the problem. We test (a variation of) the scheme used for establishing achievability on the Netflix and MovieLens datasets and obtain good agreement with the qualitative predictions of the theory we develop.
△ Less
Submitted 8 January, 2013;
originally announced January 2013.
-
Power Controlled Adaptive Sum-Capacity of Fading MACs with Distributed CSI
Authors:
Sibi Raj B. Pillai,
Bikash K. Dey,
Yash Deshpande,
Krishnamoorthy Iyer
Abstract:
We consider the problem of finding optimal, fair and distributed power-rate strategies to achieve the sum capacity of the Gaussian multiple-access block-fading channel. In here, the transmitters have access to only their own fading coefficients, while the receiver has global access to all the fading coefficients. Outage is not permitted in any communication block. The resulting average sum-through…
▽ More
We consider the problem of finding optimal, fair and distributed power-rate strategies to achieve the sum capacity of the Gaussian multiple-access block-fading channel. In here, the transmitters have access to only their own fading coefficients, while the receiver has global access to all the fading coefficients. Outage is not permitted in any communication block. The resulting average sum-throughput is also known as `power-controlled adaptive sum-capacity', which appears as an open problem in literature.
This paper presents the power-controlled adaptive sum-capacity of a wide-class of popular MAC models. In particular, we propose a power-rate strategy in the presence of distributed channel state information (CSI), which is throughput optimal when all the users have identical channel statistics. The proposed scheme also has an efficient implementation using successive cancellation and rate-splitting. We propose an upperbound when the channel laws are not identical. Furthermore, the optimal schemes are extended to situations in which each transmitter has additional finite-rate partial CSI on the link quality of others.
△ Less
Submitted 23 August, 2012;
originally announced August 2012.
-
Web Engineering
Authors:
Yogesh Deshpande,
San Murugesan,
Athula Ginige,
Steve Hansen,
Daniel Schwabe,
Martin Gaedke,
Bebo White
Abstract:
Web Engineering is the application of systematic, disciplined and quantifiable approaches to development, operation, and maintenance of Web-based applications. It is both a pro-active approach and a growing collection of theoretical and empirical research in Web application development. This paper gives an overview of Web Engineering by addressing the questions: a) why is it needed? b) what is i…
▽ More
Web Engineering is the application of systematic, disciplined and quantifiable approaches to development, operation, and maintenance of Web-based applications. It is both a pro-active approach and a growing collection of theoretical and empirical research in Web application development. This paper gives an overview of Web Engineering by addressing the questions: a) why is it needed? b) what is its domain of operation? c) how does it help and what should it do to improve Web application development? and d) how should it be incorporated in education and training? The paper discusses the significant differences that exist between Web applications and conventional software, the taxonomy of Web applications, the progress made so far and the research issues and experience of creating a specialisation at the master's level. The paper reaches a conclusion that Web Engineering at this stage is a moving target since Web technologies are constantly evolving, making new types of applications possible, which in turn may require innovations in how they are built, deployed and maintained.
△ Less
Submitted 17 June, 2003;
originally announced June 2003.
-
BaBar - A Community Web Site in an Organizational Setting
Authors:
Ray Cowan,
Yogesh Deshpande,
Bebo White
Abstract:
The BABAR Web site was established in 1993 at the Stanford Linear Accelerator Center (SLAC) to support the BABAR experiment, to report its results, and to facilitate communication among its scientific and engineering collaborators, currently numbering about 600 individuals from 75 collaborating institutions in 10 countries. The BABAR Web site is, therefore, a community Web site. At the same time…
▽ More
The BABAR Web site was established in 1993 at the Stanford Linear Accelerator Center (SLAC) to support the BABAR experiment, to report its results, and to facilitate communication among its scientific and engineering collaborators, currently numbering about 600 individuals from 75 collaborating institutions in 10 countries. The BABAR Web site is, therefore, a community Web site. At the same time it is hosted at SLAC and funded by agencies that demand adherence to policies decided under different priorities. Additionally, the BABAR Web administrators deal with the problems that arise during the course of managing users, content, policies, standards, and changing technologies. Desired solutions to some of these problems may be incompatible with the overall administration of the SLAC Web sites and/or the SLAC policies and concerns. There are thus different perspectives of the same Web site and differing expectations in segments of the SLAC population which act as constraints and challenges in any review or re-engineering activities. Web Engineering, which post-dates the BABAR Web, has aimed to provide a comprehensive understanding of all aspects of Web development. This paper reports on the first part of a recent review of application of Web Engineering methods to the BABAR Web site, which has led to explicit user and information models of the BABAR community and how SLAC and the BABAR community relate and react to each other. The paper identifies the issues of a community Web site in a hierarchical, semi-governmental sector and formulates a strategy for periodic reviews of BABAR and similar sites.
△ Less
Submitted 16 June, 2003;
originally announced June 2003.
