Showing 1–50 of 97 results for author: Jamieson, K

Optimizing Configuration Selection in Reconfigurable-Antenna MIMO Systems: Physics-Inspired Heuristic Solvers

Authors: I. Krikidis, C. Psomas, A. K. Singh, K. Jamieson

Abstract: Reconfigurable antenna multiple-input multiple-output (MIMO) is a foundational technology for the continuing evolution of cellular systems, including upcoming 6G communication systems. In this paper, we address the problem of flexible/reconfigurable antenna configuration selection for point-to-point MIMO antenna systems by using physics-inspired heuristics. Firstly, we optimize the antenna configu… ▽ More Reconfigurable antenna multiple-input multiple-output (MIMO) is a foundational technology for the continuing evolution of cellular systems, including upcoming 6G communication systems. In this paper, we address the problem of flexible/reconfigurable antenna configuration selection for point-to-point MIMO antenna systems by using physics-inspired heuristics. Firstly, we optimize the antenna configuration to maximize the signal-to-noise ratio (SNR) at the receiver by leveraging two basic heuristic solvers, i.e., coherent Ising machines (CIMs), that mimic quantum mechanical dynamics, and quantum annealing (QA), where a real-world QA architecture is considered (D-Wave). A mathematical framework that converts the configuration selection problem into CIM- and QA- compatible unconstrained quadratic formulations is investigated. Numerical and experimental results show that the proposed designs outperform classical counterparts and achieve near-optimal performance (similar to exhaustive search with exponential complexity) while ensuring polynomial complexity. Moreover, we study the optimal antenna configuration that maximizes the end-to-end Shannon capacity. A simulated annealing (SA) heuristic which achieves near-optimal performance through appropriate parameterization is adopted. A modified version of the basic SA that exploits parallel tempering to avoid local maxima is also studied, which provides additional performance gains. Extended numerical studies show that the SA solutions outperform conventional heuristics (which are also developed for comparison purposes), while the employment of the SNR-based solutions is highly sub-optimal. △ Less

Submitted 25 June, 2024; originally announced June 2024.

Comments: arXiv admin note: text overlap with arXiv:2403.12571

Journal ref: IEEE Transactions on Communications, 2004

Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioning

Authors: Jifan Zhang, Lalit Jain, Yang Guo, Jiayi Chen, Kuan Lok Zhou, Siddharth Suresh, Andrew Wagenmaker, Scott Sievert, Timothy Rogers, Kevin Jamieson, Robert Mankoff, Robert Nowak

Abstract: We present a novel multimodal preference dataset for creative tasks, consisting of over 250 million human ratings on more than 2.2 million captions, collected through crowdsourcing rating data for The New Yorker's weekly cartoon caption contest over the past eight years. This unique dataset supports the development and evaluation of multimodal large language models and preference-based fine-tuning… ▽ More We present a novel multimodal preference dataset for creative tasks, consisting of over 250 million human ratings on more than 2.2 million captions, collected through crowdsourcing rating data for The New Yorker's weekly cartoon caption contest over the past eight years. This unique dataset supports the development and evaluation of multimodal large language models and preference-based fine-tuning algorithms for humorous caption generation. We propose novel benchmarks for judging the quality of model-generated captions, utilizing both GPT4 and human judgments to establish ranking-based evaluation strategies. Our experimental results highlight the limitations of current fine-tuning methods, such as RLHF and DPO, when applied to creative tasks. Furthermore, we demonstrate that even state-of-the-art models like GPT4 and Claude currently underperform top human contestants in generating humorous captions. As we conclude this extensive data collection effort, we release the entire preference dataset to the research community, fostering further advancements in AI humor generation and evaluation. △ Less

Submitted 15 June, 2024; originally announced June 2024.

Sample Complexity Reduction via Policy Difference Estimation in Tabular Reinforcement Learning

Authors: Adhyyan Narang, Andrew Wagenmaker, Lillian Ratliff, Kevin Jamieson

Abstract: In this paper, we study the non-asymptotic sample complexity for the pure exploration problem in contextual bandits and tabular reinforcement learning (RL): identifying an epsilon-optimal policy from a set of policies with high probability. Existing work in bandits has shown that it is possible to identify the best policy by estimating only the difference between the behaviors of individual polici… ▽ More In this paper, we study the non-asymptotic sample complexity for the pure exploration problem in contextual bandits and tabular reinforcement learning (RL): identifying an epsilon-optimal policy from a set of policies with high probability. Existing work in bandits has shown that it is possible to identify the best policy by estimating only the difference between the behaviors of individual policies, which can be substantially cheaper than estimating the behavior of each policy directly. However, the best-known complexities in RL fail to take advantage of this and instead estimate the behavior of each policy directly. Does it suffice to estimate only the differences in the behaviors of policies in RL? We answer this question positively for contextual bandits but in the negative for tabular RL, showing a separation between contextual bandits and RL. However, inspired by this, we show that it almost suffices to estimate only the differences in RL: if we can estimate the behavior of a single reference policy, it suffices to only estimate how any other policy deviates from this reference policy. We develop an algorithm which instantiates this principle and obtains, to the best of our knowledge, the tightest known bound on the sample complexity of tabular RL. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Comments: 59 pages, 2 Figures

CLIPLoss and Norm-Based Data Selection Methods for Multimodal Contrastive Learning

Authors: Yi** Wang, Yifang Chen, Wendan Yan, Alex Fang, Wen**g Zhou, Kevin Jamieson, Simon Shaolei Du

Abstract: Data selection has emerged as a core issue for large-scale visual-language model pretaining (e.g., CLIP), particularly with noisy web-curated datasets. Three main data selection approaches are: (1) leveraging external non-CLIP models to aid data selection, (2) training new CLIP-style embedding models that are more effective at selecting high-quality data than the original OpenAI CLIP model, and (3… ▽ More Data selection has emerged as a core issue for large-scale visual-language model pretaining (e.g., CLIP), particularly with noisy web-curated datasets. Three main data selection approaches are: (1) leveraging external non-CLIP models to aid data selection, (2) training new CLIP-style embedding models that are more effective at selecting high-quality data than the original OpenAI CLIP model, and (3) designing better metrics or strategies universally applicable to any CLIP embedding without requiring specific model properties (e.g., CLIPScore is one popular metric). While the first two approaches have been extensively studied, the third remains under-explored. In this paper, we advance the third approach by proposing two new methods. Firstly, instead of classical CLIP scores that only consider the alignment between two modalities from a single sample, we introduce negCLIPLoss, a CLIP loss-inspired method that adds the alignment between one sample and its contrastive pairs as an extra normalization term for better quality measurement. Secondly, when downstream tasks are known, we propose a new norm-based metric, NormSim, to measure the similarity between pretraining data and target data. We test our methods on the data selection benchmark, DataComp~\cite{gadre2023datacomp}. Compared to the best baseline using only OpenAI's CLIP-L/14, our methods achieve a 5.3\% improvement on ImageNet-1k and a 2.8\% improvement on 38 downstream evaluation tasks. Moreover, both negCLIPLoss and NormSim are compatible with existing techniques. By combining our methods with the current best methods DFN~\cite{fang2023data} and HYPE~\cite{kim2024hype}, we can boost average performance on downstream tasks by 0.9\%, achieving a new state-of-the-art. △ Less

Submitted 29 May, 2024; originally announced May 2024.

Comments: This paper supercedes our previous VAS paper (arXiv:2402.02055)

Wall-Street: Smart Surface-Enabled 5G mmWave for Roadside Networking

Authors: Kun Woo Cho, Prasanthi Maddala, Ivan Seskar, Kyle Jamieson

Abstract: 5G mmWave roadside networks promise high-speed wireless connectivity, but face significant challenges in maintaining reliable connections for users moving at high speed. Frequent handovers, complex beam alignment, and signal attenuation due to obstacles like car bodies lead to service interruptions and degraded performance. We present Wall-Street, a smart surface installed on vehicles to enhance 5… ▽ More 5G mmWave roadside networks promise high-speed wireless connectivity, but face significant challenges in maintaining reliable connections for users moving at high speed. Frequent handovers, complex beam alignment, and signal attenuation due to obstacles like car bodies lead to service interruptions and degraded performance. We present Wall-Street, a smart surface installed on vehicles to enhance 5G mmWave connectivity for users inside. Wall-Street improves mobility management by (1) steering outdoor mmWave signals into the vehicle, ensuring coverage for all users; (2) enabling simultaneous serving cell data transfer and candidate handover cell measurement, allowing seamless handovers without service interruption; and (3) combining beams from source and target cells during a handover to increase reliability. Through its flexible and diverse signal manipulation capabilities, Wall-Street provides uninterrupted high-speed connectivity for latency-sensitive applications in challenging mobile environments. We have implemented and integrated Wall-Street in the COSMOS testbed and evaluated its real-time performance with four gNBs and a mobile client inside a surface-enabled vehicle, driving on a nearby road. Wall-Street achieves a 2.5-3.4x TCP throughput improvement and a 0.4-0.8x reduction in delay over a baseline 5G Standalone handover protocol. △ Less

Submitted 10 May, 2024; originally announced May 2024.

Comments: 15 pages, 22 figures, under submission

Multi Digit Ising Map** for Low Precision Ising Solvers

Authors: Abhishek Kumar Singh, Kyle Jamieson

Abstract: The last couple of years have seen an ever-increasing interest in using different Ising solvers, like Quantum annealers, Coherent Ising machines, and Oscillator-based Ising machines, for solving tough computational problems in various domains. Although the simulations predict massive performance improvements for several tough computational problems, the real implementations of the Ising solvers te… ▽ More The last couple of years have seen an ever-increasing interest in using different Ising solvers, like Quantum annealers, Coherent Ising machines, and Oscillator-based Ising machines, for solving tough computational problems in various domains. Although the simulations predict massive performance improvements for several tough computational problems, the real implementations of the Ising solvers tend to have limited precision, which can cause significant performance deterioration. This paper presents a novel methodology for map** the problem on the Ising solvers to artificially increase the effective precision. We further evaluate our method for the Multiple-Input-Multiple-Output signal detection problem. △ Less

Submitted 8 April, 2024; originally announced April 2024.

Comments: version 1.0

Optimizing Reconfigurable Antenna MIMO Systems with Coherent Ising Machines

Authors: Ioannis Krikidis, Abhishek Kumar Singh, Kyle Jamieson

Abstract: Reconfigurable antenna multiple-input multiple-output (MIMO) is a promising technology for upcoming 6G communication systems. In this paper, we deal with the problem of configuration selection for reconfigurable antenna MIMO by leveraging Coherent Ising Machines (CIMs). By adopting the CIM as a heuristic solver for the Ising problem, the optimal antenna configuration that maximizes the received si… ▽ More Reconfigurable antenna multiple-input multiple-output (MIMO) is a promising technology for upcoming 6G communication systems. In this paper, we deal with the problem of configuration selection for reconfigurable antenna MIMO by leveraging Coherent Ising Machines (CIMs). By adopting the CIM as a heuristic solver for the Ising problem, the optimal antenna configuration that maximizes the received signal-to-noise ratio is investigated. A mathematical framework that converts the selection problem into a CIM-compatible unconstrained quadratic formulation is presented. Numerical studies show that the proposed CIM-based design outperforms classical counterparts and achieves near-optimal performance (similar to exponentially complex exhaustive searching) while ensuring polynomial complexity. △ Less

Submitted 19 March, 2024; originally announced March 2024.

Journal ref: IEEE International Conference on Communications (ICC), June 2024

X-ResQ: Reverse Annealing for Quantum MIMO Detection with Flexible Parallelism

Authors: Minsung Kim, Abhishek Kumar Singh, Davide Venturelli, John Kaewell, Kyle Jamieson

Abstract: Quantum Annealing (QA)-accelerated MIMO detection is an emerging research approach in the context of NextG wireless networks. The opportunity is to enable large MIMO systems and thus improve wireless performance. The approach aims to leverage QA to expedite the computation required for theoretically optimal but computationally-demanding Maximum Likelihood detection to overcome the limitations of t… ▽ More Quantum Annealing (QA)-accelerated MIMO detection is an emerging research approach in the context of NextG wireless networks. The opportunity is to enable large MIMO systems and thus improve wireless performance. The approach aims to leverage QA to expedite the computation required for theoretically optimal but computationally-demanding Maximum Likelihood detection to overcome the limitations of the currently deployed linear detectors. This paper presents X-ResQ, a QA-based MIMO detector system featuring fine-grained quantum task parallelism that is uniquely enabled by the Reverse Annealing (RA) protocol. Unlike prior designs, X-ResQ has many desirable system properties for a parallel QA detector and has effectively improved detection performance as more qubits are assigned. In our evaluations on a state-of-the-art quantum annealer, fully parallel X-ResQ achieves near-optimal throughput (over 10 bits/s/Hz) for $4\times6$ MIMO with 16-QAM using six levels of parallelism with 240 qubits and $220~μ$s QA compute time, achieving 2.5--5$\times$ gains compared against other tested detectors. For more comprehensive evaluations, we implement and evaluate X-ResQ in the non-quantum digital setting. This non-quantum X-ResQ demonstration showcases the potential to realize ultra-large $1024\times1024$ MIMO, significantly outperforming other MIMO detectors, including the state-of-the-art RA detector classically implemented in the same way. △ Less

Submitted 9 March, 2024; v1 submitted 28 February, 2024; originally announced February 2024.

Comments: 22 pages

Evolving Mobile Cloud Gaming with 5G Standalone Network Telemetry

Authors: Haoran Wan, Kyle Jamieson

Abstract: Mobile cloud gaming places the simultaneous demands of high capacity and low latency on the wireless network, demands that Private and Metropolitan-Area Standalone 5G networks are poised to meet. However, lacking introspection into the 5G Radio Access Network (RAN), cloud gaming servers are ill-poised to cope with the vagaries of the wireless last hop to a mobile client, while 5G network operators… ▽ More Mobile cloud gaming places the simultaneous demands of high capacity and low latency on the wireless network, demands that Private and Metropolitan-Area Standalone 5G networks are poised to meet. However, lacking introspection into the 5G Radio Access Network (RAN), cloud gaming servers are ill-poised to cope with the vagaries of the wireless last hop to a mobile client, while 5G network operators run mostly closed networks, limiting their potential for co-design with the wider internet and user applications. This paper presents Telesa, a passive, incrementally-deployable, and independently-deployable Standalone 5G network telemetry system that streams fine-grained RAN capacity, latency, and retransmission information to application servers to enable better millisecond scale, application-level decisions on offered load and bit rate adaptation than end-to-end latency measurements or end-to-end packet losses currently permit. We design, implement, and evaluate a Telesa telemetry-enhanced game streaming platform, demonstrating exact congestion-control that can better adapt game video bitrate while simultaneously controlling end-to-end latency, thus maximizing game quality of experience. Our experimental evaluation on a production 5G Standalone network demonstrates a 178-249% Quality of Experience improvement versus two state-of-the-art cloud gaming applications. △ Less

Submitted 6 February, 2024; originally announced February 2024.

Variance Alignment Score: A Simple But Tough-to-Beat Data Selection Method for Multimodal Contrastive Learning

Authors: Yi** Wang, Yifang Chen, Wendan Yan, Kevin Jamieson, Simon Shaolei Du

Abstract: In recent years, data selection has emerged as a core issue for large-scale visual-language model pretraining, especially on noisy web-curated datasets. One widely adopted strategy assigns quality scores such as CLIP similarity for each sample and retains the data pairs with the highest scores. However, these approaches are agnostic of data distribution and always fail to select the most informati… ▽ More In recent years, data selection has emerged as a core issue for large-scale visual-language model pretraining, especially on noisy web-curated datasets. One widely adopted strategy assigns quality scores such as CLIP similarity for each sample and retains the data pairs with the highest scores. However, these approaches are agnostic of data distribution and always fail to select the most informative samples. To solve this problem, we propose a simple yet theoretically principled metric named Variance Alignment Score (VAS), which has the form $\langle Σ_{\text{test}}, Σ_i\rangle$. Here, $Σ_{\text{test}}$ represents the target (cross-)covariance matrix we aim to align, potentially based on prior knowledge, while $Σ_i$ denotes the tensor product of single or multi-modal representations for the $i$-th sample. We further design a new data selection method that maximizes the total VAS. We provide theoretical analysis in a simplified setting to demonstrate the theoretical advantage of VAS over random or other existing data selection. Experimentally, applying VAS and CLIP scores together can outperform baselines by a margin of $1.3\%$ average on 38 evaluation sets for noisy dataset DataComp and $2.5\%$ on VTAB for high-quality dataset CC12M. Additionally, our ablation study also shows visual features are better than text for calculating VAS, and the related classical experimental design methods may fail under this context. △ Less

Submitted 3 February, 2024; originally announced February 2024.

Comments: 17 pages, 4 figures

An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models

Authors: Gantavya Bhatt, Yifang Chen, Arnav M. Das, Jifan Zhang, Sang T. Truong, Stephen Mussmann, Yinglun Zhu, Jeffrey Bilmes, Simon S. Du, Kevin Jamieson, Jordan T. Ash, Robert D. Nowak

Abstract: Supervised finetuning (SFT) on instruction datasets has played a crucial role in achieving the remarkable zero-shot generalization capabilities observed in modern large language models (LLMs). However, the annotation efforts required to produce high quality responses for instructions are becoming prohibitively expensive, especially as the number of tasks spanned by instruction datasets continues t… ▽ More Supervised finetuning (SFT) on instruction datasets has played a crucial role in achieving the remarkable zero-shot generalization capabilities observed in modern large language models (LLMs). However, the annotation efforts required to produce high quality responses for instructions are becoming prohibitively expensive, especially as the number of tasks spanned by instruction datasets continues to increase. Active learning is effective in identifying useful subsets of samples to annotate from an unlabeled pool, but its high computational cost remains a barrier to its widespread applicability in the context of LLMs. To mitigate the annotation cost of SFT and circumvent the computational bottlenecks of active learning, we propose using experimental design. Experimental design techniques select the most informative samples to label, and typically maximize some notion of uncertainty and/or diversity. In our work, we implement a framework that evaluates several existing and novel experimental design techniques and find that these methods consistently yield significant gains in label efficiency with little computational overhead. On generative tasks, our methods achieve the same generalization performance with only $50\%$ of annotation cost required by random sampling. △ Less

Submitted 6 May, 2024; v1 submitted 12 January, 2024; originally announced January 2024.

LoLa: Low-Latency Realtime Video Conferencing over Multiple Cellular Carriers

Authors: Sara Ayoubi, Giulio Grassi, Giovanni Pau, Kyle Jamieson, Renata Teixeira

Abstract: LoLa is a novel multi-path system for video conferencing applications over cellular networks. It provides significant gains over single link solutions when the link quality over different cellular networks fluctuate dramatically and independently over time, or when aggregating the throughput across different cellular links improves the perceived video quality. LoLa achieves this by continuously es… ▽ More LoLa is a novel multi-path system for video conferencing applications over cellular networks. It provides significant gains over single link solutions when the link quality over different cellular networks fluctuate dramatically and independently over time, or when aggregating the throughput across different cellular links improves the perceived video quality. LoLa achieves this by continuously estimating the quality of available cellular links to decide how to strip video packets across them without inducing delays or packet drops. It is also tightly coupled with state-of-the-art video codec to dynamically adapt video frame size to respond quickly to changing network conditions. Using multiple traces collected over 4 different cellular operators in a large metropolitan city, we demonstrate that LoLa provides significant gains in terms of throughput and delays compared to state-of-the-art real-time video conferencing solution. △ Less

Submitted 19 December, 2023; originally announced December 2023.

Comments: 9 pages, 9 figures

Fair Active Learning in Low-Data Regimes

Authors: Romain Camilleri, Andrew Wagenmaker, Jamie Morgenstern, Lalit Jain, Kevin Jamieson

Abstract: In critical machine learning applications, ensuring fairness is essential to avoid perpetuating social inequities. In this work, we address the challenges of reducing bias and improving accuracy in data-scarce environments, where the cost of collecting labeled data prohibits the use of large, labeled datasets. In such settings, active learning promises to maximize marginal accuracy gains of small… ▽ More In critical machine learning applications, ensuring fairness is essential to avoid perpetuating social inequities. In this work, we address the challenges of reducing bias and improving accuracy in data-scarce environments, where the cost of collecting labeled data prohibits the use of large, labeled datasets. In such settings, active learning promises to maximize marginal accuracy gains of small amounts of labeled data. However, existing applications of active learning for fairness fail to deliver on this, typically requiring large labeled datasets, or failing to ensure the desired fairness tolerance is met on the population distribution. To address such limitations, we introduce an innovative active learning framework that combines an exploration procedure inspired by posterior sampling with a fair classification subroutine. We demonstrate that this framework performs effectively in very data-scarce regimes, maximizing accuracy while satisfying fairness constraints with high probability. We evaluate our proposed approach using well-established real-world benchmark datasets and compare it against state-of-the-art methods, demonstrating its effectiveness in producing fair models, and improvement over existing methods. △ Less

Submitted 13 December, 2023; originally announced December 2023.

Physics-Inspired Discrete-Phase Optimization for 3D Beamforming with PIN-Diode Extra-Large Antenna Arrays

Authors: Minsung Kim, Annalise Stockley, Keith Briggs, Kyle Jamieson

Abstract: Large antenna arrays can steer narrow beams towards a target area, and thus improve the communications capacity of wireless channels and the fidelity of radio sensing. Hardware that is capable of continuously-variable phase shifts is expensive, presenting scaling challenges. PIN diodes that apply only discrete phase shifts are promising and cost-effective; however, unlike continuous phase shifters… ▽ More Large antenna arrays can steer narrow beams towards a target area, and thus improve the communications capacity of wireless channels and the fidelity of radio sensing. Hardware that is capable of continuously-variable phase shifts is expensive, presenting scaling challenges. PIN diodes that apply only discrete phase shifts are promising and cost-effective; however, unlike continuous phase shifters, finding the best phase configuration across elements is an NP-hard optimization problem. Thus, the complexity of optimization becomes a new bottleneck for large-antenna arrays. To address this challenge, this paper suggests a procedure for converting the optimization objective function from a ratio of quadratic functions to a sequence of more easily solvable quadratic unconstrained binary optimization (QUBO) sub-problems. This conversion is an exact equivalence, and the resulting QUBO forms are standard input formats for various physics-inspired optimization methods. We demonstrate that a simulated annealing approach is very effective for solving these sub-problems, and we give performance metrics for several large array types optimized by this technique. Through numerical experiments, we report 3D beamforming performance for extra-large arrays with up to 10,000 elements. △ Less

Submitted 3 January, 2024; v1 submitted 30 October, 2023; originally announced November 2023.

Minimax Optimal Submodular Optimization with Bandit Feedback

Authors: Artin Tajdini, Lalit Jain, Kevin Jamieson

Abstract: We consider maximizing a monotonic, submodular set function $f: 2^{[n]} \rightarrow [0,1]$ under stochastic bandit feedback. Specifically, $f$ is unknown to the learner but at each time $t=1,\dots,T$ the learner chooses a set $S_t \subset [n]$ with $|S_t| \leq k$ and receives reward $f(S_t) + η_t$ where $η_t$ is mean-zero sub-Gaussian noise. The objective is to minimize the learner's regret over… ▽ More We consider maximizing a monotonic, submodular set function $f: 2^{[n]} \rightarrow [0,1]$ under stochastic bandit feedback. Specifically, $f$ is unknown to the learner but at each time $t=1,\dots,T$ the learner chooses a set $S_t \subset [n]$ with $|S_t| \leq k$ and receives reward $f(S_t) + η_t$ where $η_t$ is mean-zero sub-Gaussian noise. The objective is to minimize the learner's regret over $T$ times with respect to ($1-e^{-1}$)-approximation of maximum $f(S_*)$ with $|S_*| = k$, obtained through greedy maximization of $f$. To date, the best regret bound in the literature scales as $k n^{1/3} T^{2/3}$. And by trivially treating every set as a unique arm one deduces that $\sqrt{ {n \choose k} T }$ is also achievable. In this work, we establish the first minimax lower bound for this setting that scales like $\mathcal{O}(\min_{i \le k}(in^{1/3}T^{2/3} + \sqrt{n^{k-i}T}))$. Moreover, we propose an algorithm that is capable of matching the lower bound regret. △ Less

Submitted 27 October, 2023; originally announced October 2023.

Near-Optimal Pure Exploration in Matrix Games: A Generalization of Stochastic Bandits & Dueling Bandits

Authors: Arnab Maiti, Ross Boczar, Kevin Jamieson, Lillian J. Ratliff

Abstract: We study the sample complexity of identifying the pure strategy Nash equilibrium (PSNE) in a two-player zero-sum matrix game with noise. Formally, we are given a stochastic model where any learner can sample an entry $(i,j)$ of the input matrix $A\in[-1,1]^{n\times m}$ and observe $A_{i,j}+η$ where $η$ is a zero-mean 1-sub-Gaussian noise. The aim of the learner is to identify the PSNE of $A$, when… ▽ More We study the sample complexity of identifying the pure strategy Nash equilibrium (PSNE) in a two-player zero-sum matrix game with noise. Formally, we are given a stochastic model where any learner can sample an entry $(i,j)$ of the input matrix $A\in[-1,1]^{n\times m}$ and observe $A_{i,j}+η$ where $η$ is a zero-mean 1-sub-Gaussian noise. The aim of the learner is to identify the PSNE of $A$, whenever it exists, with high probability while taking as few samples as possible. Zhou et al. (2017) presents an instance-dependent sample complexity lower bound that depends only on the entries in the row and column in which the PSNE lies. We design a near-optimal algorithm whose sample complexity matches the lower bound, up to log factors. The problem of identifying the PSNE also generalizes the problem of pure exploration in stochastic multi-armed bandits and dueling bandits, and our result matches the optimal bounds, up to log factors, in both the settings. △ Less

Submitted 27 November, 2023; v1 submitted 24 October, 2023; originally announced October 2023.

Comments: 22 pages, 5 figures

Query-Efficient Algorithms to Find the Unique Nash Equilibrium in a Two-Player Zero-Sum Matrix Game

Authors: Arnab Maiti, Ross Boczar, Kevin Jamieson, Lillian J. Ratliff

Abstract: We study the query complexity of identifying Nash equilibria in two-player zero-sum matrix games. Grigoriadis and Khachiyan (1995) showed that any deterministic algorithm needs to query $Ω(n^2)$ entries in worst case from an $n\times n$ input matrix in order to compute an $\varepsilon$-approximate Nash equilibrium, where $\varepsilon<\frac{1}{2}$. Moreover, they designed a randomized algorithm tha… ▽ More We study the query complexity of identifying Nash equilibria in two-player zero-sum matrix games. Grigoriadis and Khachiyan (1995) showed that any deterministic algorithm needs to query $Ω(n^2)$ entries in worst case from an $n\times n$ input matrix in order to compute an $\varepsilon$-approximate Nash equilibrium, where $\varepsilon<\frac{1}{2}$. Moreover, they designed a randomized algorithm that queries $\mathcal O(\frac{n\log n}{\varepsilon^2})$ entries from the input matrix in expectation and returns an $\varepsilon$-approximate Nash equilibrium when the entries of the matrix are bounded between $-1$ and $1$. However, these two results do not completely characterize the query complexity of finding an exact Nash equilibrium in two-player zero-sum matrix games. In this work, we characterize the query complexity of finding an exact Nash equilibrium for two-player zero-sum matrix games that have a unique Nash equilibrium $(x_\star,y_\star)$. We first show that any randomized algorithm needs to query $Ω(nk)$ entries of the input matrix $A\in\mathbb{R}^{n\times n}$ in expectation in order to find the unique Nash equilibrium where $k=|\text{supp}(x_\star)|$. We complement this lower bound by presenting a simple randomized algorithm that, with probability $1-δ$, returns the unique Nash equilibrium by querying at most $\mathcal O(nk^4\cdot \text{polylog}(\frac{n}δ))$ entries of the input matrix $A\in\mathbb{R}^{n\times n}$. In the special case when the unique Nash Equilibrium is a pure-strategy Nash equilibrium (PSNE), we design a simple deterministic algorithm that finds the PSNE by querying at most $\mathcal O(n)$ entries of the input matrix. △ Less

Submitted 27 November, 2023; v1 submitted 24 October, 2023; originally announced October 2023.

Comments: 17 pages

WaveFlex: A Smart Surface for Private CBRS Wireless Cellular Networks

Authors: Fan Yi, Kun Woo Cho, Yaxiong Xie, Kyle Jamieson

Abstract: We present the design and implementation of WaveFlex, the first smart surface that enhances Private LTE/5G networks operating under the shared-license framework in the Citizens Broadband Radio Service frequency band. WaveFlex works in the presence of frequency diversity: multiple nearby base stations operating on different frequencies, as dictated by a Spectrum Access System coordinator. It also h… ▽ More We present the design and implementation of WaveFlex, the first smart surface that enhances Private LTE/5G networks operating under the shared-license framework in the Citizens Broadband Radio Service frequency band. WaveFlex works in the presence of frequency diversity: multiple nearby base stations operating on different frequencies, as dictated by a Spectrum Access System coordinator. It also handles time dynamism: due to the dynamic sharing rules of the band, base stations occasionally switch channels, especially when priority users enter the network. Finally, WaveFlex operates independently of the network itself, not requiring access to nor modification of the base station or mobile users, yet it remain compliant with and effective on prevailing cellular protocols. We have designed and fabricated WaveFlex on a custom multi-layer PCB, software defined radio-based network monitor, and supporting control software and hardware. Our experimental evaluation benchmarks an operational Private LTE network running at full line rate. Results demonstrate an 8.50 dB average SNR gain, and an average throughput gain of 4.36 Mbps for a single small cell, and 3.19 Mbps for four small cells, in a realistic indoor office scenario. △ Less

Submitted 17 October, 2023; originally announced October 2023.

Comments: 15 pages

Optimal Exploration is no harder than Thompson Sampling

Authors: Zhaoqi Li, Kevin Jamieson, Lalit Jain

Abstract: Given a set of arms $\mathcal{Z}\subset \mathbb{R}^d$ and an unknown parameter vector $θ_\ast\in\mathbb{R}^d$, the pure exploration linear bandit problem aims to return $\arg\max_{z\in \mathcal{Z}} z^{\top}θ_{\ast}$, with high probability through noisy measurements of $x^{\top}θ_{\ast}$ with $x\in \mathcal{X}\subset \mathbb{R}^d$. Existing (asymptotically) optimal methods require either a) potenti… ▽ More Given a set of arms $\mathcal{Z}\subset \mathbb{R}^d$ and an unknown parameter vector $θ_\ast\in\mathbb{R}^d$, the pure exploration linear bandit problem aims to return $\arg\max_{z\in \mathcal{Z}} z^{\top}θ_{\ast}$, with high probability through noisy measurements of $x^{\top}θ_{\ast}$ with $x\in \mathcal{X}\subset \mathbb{R}^d$. Existing (asymptotically) optimal methods require either a) potentially costly projections for each arm $z\in \mathcal{Z}$ or b) explicitly maintaining a subset of $\mathcal{Z}$ under consideration at each time. This complexity is at odds with the popular and simple Thompson Sampling algorithm for regret minimization, which just requires access to a posterior sampling and argmax oracle, and does not need to enumerate $\mathcal{Z}$ at any point. Unfortunately, Thompson sampling is known to be sub-optimal for pure exploration. In this work, we pose a natural question: is there an algorithm that can explore optimally and only needs the same computational primitives as Thompson Sampling? We answer the question in the affirmative. We provide an algorithm that leverages only sampling and argmax oracles and achieves an exponential convergence rate, with the exponent being the optimal among all possible allocations asymptotically. In addition, we show that our algorithm can be easily implemented and performs as well empirically as existing asymptotically optimal methods. △ Less

Submitted 24 October, 2023; v1 submitted 9 October, 2023; originally announced October 2023.

Pick Planning Strategies for Large-Scale Package Manipulation

Authors: Shuai Li, Azarakhsh Keipour, Kevin Jamieson, Nicolas Hudson, Sicong Zhao, Charles Swan, Kostas Bekris

Abstract: Automating warehouse operations can reduce logistics overhead costs, ultimately driving down the final price for consumers, increasing the speed of delivery, and enhancing the resiliency to market fluctuations. This extended abstract showcases a large-scale package manipulation from unstructured piles in Amazon Robotics' Robot Induction (Robin) fleet, which is used for picking and singulating up… ▽ More Automating warehouse operations can reduce logistics overhead costs, ultimately driving down the final price for consumers, increasing the speed of delivery, and enhancing the resiliency to market fluctuations. This extended abstract showcases a large-scale package manipulation from unstructured piles in Amazon Robotics' Robot Induction (Robin) fleet, which is used for picking and singulating up to 6 million packages per day and so far has manipulated over 2 billion packages. It describes the various heuristic methods developed over time and their successor, which utilizes a pick success predictor trained on real production data. To the best of the authors' knowledge, this work is the first large-scale deployment of learned pick quality estimation methods in a real production system. △ Less

Submitted 8 October, 2023; v1 submitted 22 September, 2023; originally announced September 2023.

Comments: 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Learning Meets Model-based Methods for Manipulation and Gras** Workshop

A/B Testing and Best-arm Identification for Linear Bandits with Robustness to Non-stationarity

Authors: Zhihan Xiong, Romain Camilleri, Maryam Fazel, Lalit Jain, Kevin Jamieson

Abstract: We investigate the fixed-budget best-arm identification (BAI) problem for linear bandits in a potentially non-stationary environment. Given a finite arm set $\mathcal{X}\subset\mathbb{R}^d$, a fixed budget $T$, and an unpredictable sequence of parameters $\left\lbraceθ_t\right\rbrace_{t=1}^{T}$, an algorithm will aim to correctly identify the best arm… ▽ More We investigate the fixed-budget best-arm identification (BAI) problem for linear bandits in a potentially non-stationary environment. Given a finite arm set $\mathcal{X}\subset\mathbb{R}^d$, a fixed budget $T$, and an unpredictable sequence of parameters $\left\lbraceθ_t\right\rbrace_{t=1}^{T}$, an algorithm will aim to correctly identify the best arm $x^* := \arg\max_{x\in\mathcal{X}}x^\top\sum_{t=1}^{T}θ_t$ with probability as high as possible. Prior work has addressed the stationary setting where $θ_t = θ_1$ for all $t$ and demonstrated that the error probability decreases as $\exp(-T /ρ^*)$ for a problem-dependent constant $ρ^*$. But in many real-world $A/B/n$ multivariate testing scenarios that motivate our work, the environment is non-stationary and an algorithm expecting a stationary setting can easily fail. For robust identification, it is well-known that if arms are chosen randomly and non-adaptively from a G-optimal design over $\mathcal{X}$ at each time then the error probability decreases as $\exp(-TΔ^2_{(1)}/d)$, where $Δ_{(1)} = \min_{x \neq x^*} (x^* - x)^\top \frac{1}{T}\sum_{t=1}^T θ_t$. As there exist environments where $Δ_{(1)}^2/ d \ll 1/ ρ^*$, we are motivated to propose a novel algorithm $\mathsf{P1}$-$\mathsf{RAGE}$ that aims to obtain the best of both worlds: robustness to non-stationarity and fast rates of identification in benign settings. We characterize the error probability of $\mathsf{P1}$-$\mathsf{RAGE}$ and demonstrate empirically that the algorithm indeed never performs worse than G-optimal design but compares favorably to the best algorithms in the stationary setting. △ Less

Submitted 15 February, 2024; v1 submitted 27 July, 2023; originally announced July 2023.

Comments: 25 pages, 6 figures

Logarithmic Regret for Matrix Games against an Adversary with Noisy Bandit Feedback

Authors: Arnab Maiti, Kevin Jamieson, Lillian J. Ratliff

Abstract: This paper considers a variant of zero-sum matrix games where at each timestep the row player chooses row $i$, the column player chooses column $j$, and the row player receives a noisy reward with mean $A_{i,j}$. The objective of the row player is to accumulate as much reward as possible, even against an adversarial column player. If the row player uses the EXP3 strategy, an algorithm known for ob… ▽ More This paper considers a variant of zero-sum matrix games where at each timestep the row player chooses row $i$, the column player chooses column $j$, and the row player receives a noisy reward with mean $A_{i,j}$. The objective of the row player is to accumulate as much reward as possible, even against an adversarial column player. If the row player uses the EXP3 strategy, an algorithm known for obtaining $\sqrt{T}$ regret against an arbitrary sequence of rewards, it is immediate that the row player also achieves $\sqrt{T}$ regret relative to the Nash equilibrium in this game setting. However, partly motivated by the fact that the EXP3 strategy is myopic to the structure of the game, O'Donoghue et al. (2021) proposed a UCB-style algorithm that leverages the game structure and demonstrated that this algorithm greatly outperforms EXP3 empirically. While they showed that this UCB-style algorithm achieved $\sqrt{T}$ regret, in this paper we ask if there exists an algorithm that provably achieves $\text{polylog}(T)$ regret against any adversary, analogous to results from stochastic bandits. We propose a novel algorithm that answers this question in the affirmative for the simple $2 \times 2$ setting, providing the first instance-dependent guarantees for games in the regret setting. Our algorithm overcomes two major hurdles: 1) obtaining logarithmic regret even though the Nash equilibrium is estimable only at a $1/\sqrt{T}$ rate, and 2) designing row-player strategies that guarantee that either the adversary provides information about the Nash equilibrium, or the row player incurs negative regret. Moreover, in the full information case we address the general $n \times m$ case where the first hurdle is still relevant. Finally, we show that EXP3 and the UCB-based algorithm necessarily cannot perform better than $\sqrt{T}$. △ Less

Submitted 24 October, 2023; v1 submitted 22 June, 2023; originally announced June 2023.

Comments: 68 pages, 2 figures

LabelBench: A Comprehensive Framework for Benchmarking Adaptive Label-Efficient Learning

Authors: Jifan Zhang, Yifang Chen, Gregory Canal, Stephen Mussmann, Arnav M. Das, Gantavya Bhatt, Yinglun Zhu, Jeffrey Bilmes, Simon Shaolei Du, Kevin Jamieson, Robert D Nowak

Abstract: Labeled data are critical to modern machine learning applications, but obtaining labels can be expensive. To mitigate this cost, machine learning methods, such as transfer learning, semi-supervised learning and active learning, aim to be label-efficient: achieving high predictive performance from relatively few labeled examples. While obtaining the best label-efficiency in practice often requires… ▽ More Labeled data are critical to modern machine learning applications, but obtaining labels can be expensive. To mitigate this cost, machine learning methods, such as transfer learning, semi-supervised learning and active learning, aim to be label-efficient: achieving high predictive performance from relatively few labeled examples. While obtaining the best label-efficiency in practice often requires combinations of these techniques, existing benchmark and evaluation frameworks do not capture a concerted combination of all such techniques. This paper addresses this deficiency by introducing LabelBench, a new computationally-efficient framework for joint evaluation of multiple label-efficient learning techniques. As an application of LabelBench, we introduce a novel benchmark of state-of-the-art active learning methods in combination with semi-supervised learning for fine-tuning pretrained vision transformers. Our benchmark demonstrates better label-efficiencies than previously reported in active learning. LabelBench's modular codebase is open-sourced for the broader community to contribute label-efficient learning methods and benchmarks. The repository can be found at: https://github.com/EfficientTraining/LabelBench. △ Less

Submitted 1 March, 2024; v1 submitted 16 June, 2023; originally announced June 2023.

Optimal Exploration for Model-Based RL in Nonlinear Systems

Authors: Andrew Wagenmaker, Guanya Shi, Kevin Jamieson

Abstract: Learning to control unknown nonlinear dynamical systems is a fundamental problem in reinforcement learning and control theory. A commonly applied approach is to first explore the environment (exploration), learn an accurate model of it (system identification), and then compute an optimal controller with the minimum cost on this estimated system (policy optimization). While existing work has shown… ▽ More Learning to control unknown nonlinear dynamical systems is a fundamental problem in reinforcement learning and control theory. A commonly applied approach is to first explore the environment (exploration), learn an accurate model of it (system identification), and then compute an optimal controller with the minimum cost on this estimated system (policy optimization). While existing work has shown that it is possible to learn a uniformly good model of the system~\citep{mania2020active}, in practice, if we aim to learn a good controller with a low cost on the actual system, certain system parameters may be significantly more critical than others, and we therefore ought to focus our exploration on learning such parameters. In this work, we consider the setting of nonlinear dynamical systems and seek to formally quantify, in such settings, (a) which parameters are most relevant to learning a good controller, and (b) how we can best explore so as to minimize uncertainty in such parameters. Inspired by recent work in linear systems~\citep{wagenmaker2021task}, we show that minimizing the controller loss in nonlinear systems translates to estimating the system parameters in a particular, task-dependent metric. Motivated by this, we develop an algorithm able to efficiently explore the system to reduce uncertainty in this metric, and prove a lower bound showing that our approach learns a controller at a near-instance-optimal rate. Our algorithm relies on a general reduction from policy optimization to optimal experiment design in arbitrary systems, and may be of independent interest. We conclude with experiments demonstrating the effectiveness of our method in realistic nonlinear robotic systems. △ Less

Submitted 15 June, 2023; originally announced June 2023.

Active Representation Learning for General Task Space with Applications in Robotics

Authors: Yifang Chen, Yingbing Huang, Simon S. Du, Kevin Jamieson, Guanya Shi

Abstract: Representation learning based on multi-task pretraining has become a powerful approach in many domains. In particular, task-aware representation learning aims to learn an optimal representation for a specific target task by sampling data from a set of source tasks, while task-agnostic representation learning seeks to learn a universal representation for a class of tasks. In this paper, we propose… ▽ More Representation learning based on multi-task pretraining has become a powerful approach in many domains. In particular, task-aware representation learning aims to learn an optimal representation for a specific target task by sampling data from a set of source tasks, while task-agnostic representation learning seeks to learn a universal representation for a class of tasks. In this paper, we propose a general and versatile algorithmic and theoretic framework for \textit{active representation learning}, where the learner optimally chooses which source tasks to sample from. This framework, along with a tractable meta algorithm, allows most arbitrary target and source task spaces (from discrete to continuous), covers both task-aware and task-agnostic settings, and is compatible with deep representation learning practices. We provide several instantiations under this framework, from bilinear and feature-based nonlinear to general nonlinear cases. In the bilinear case, by leveraging the non-uniform spectrum of the task representation and the calibrated source-target relevance, we prove that the sample complexity to achieve $\varepsilon$-excess risk on target scales with $ (k^*)^2 \|v^*\|_2^2 \varepsilon^{-2}$ where $k^*$ is the effective dimension of the target and $\|v^*\|_2^2 \in (0,1]$ represents the connection between source and target space. Compared to the passive one, this can save up to $\frac{1}{d_W}$ of sample complexity, where $d_W$ is the task space dimension. Finally, we demonstrate different instantiations of our meta algorithm in synthetic datasets and robotics problems, from pendulum simulations to real-world drone flight datasets. On average, our algorithms outperform baselines by $20\%-70\%$. △ Less

Submitted 15 June, 2023; originally announced June 2023.

Improved Active Multi-Task Representation Learning via Lasso

Authors: Yi** Wang, Yifang Chen, Kevin Jamieson, Simon S. Du

Abstract: To leverage the copious amount of data from source tasks and overcome the scarcity of the target task samples, representation learning based on multi-task pretraining has become a standard approach in many applications. However, up until now, most existing works design a source task selection strategy from a purely empirical perspective. Recently, \citet{chen2022active} gave the first active multi… ▽ More To leverage the copious amount of data from source tasks and overcome the scarcity of the target task samples, representation learning based on multi-task pretraining has become a standard approach in many applications. However, up until now, most existing works design a source task selection strategy from a purely empirical perspective. Recently, \citet{chen2022active} gave the first active multi-task representation learning (A-MTRL) algorithm which adaptively samples from source tasks and can provably reduce the total sample complexity using the L2-regularized-target-source-relevance parameter $ν^2$. But their work is theoretically suboptimal in terms of total source sample complexity and is less practical in some real-world scenarios where sparse training source task selection is desired. In this paper, we address both issues. Specifically, we show the strict dominance of the L1-regularized-relevance-based ($ν^1$-based) strategy by giving a lower bound for the $ν^2$-based strategy. When $ν^1$ is unknown, we propose a practical algorithm that uses the LASSO program to estimate $ν^1$. Our algorithm successfully recovers the optimal result in the known case. In addition to our sample complexity results, we also characterize the potential of our $ν^1$-based strategy in sample-cost-sensitive settings. Finally, we provide experiments on real-world computer vision datasets to illustrate the effectiveness of our proposed method. △ Less

Submitted 4 June, 2023; originally announced June 2023.

Comments: Accepted by ICML 2023

Demonstrating Large-Scale Package Manipulation via Learned Metrics of Pick Success

Authors: Shuai Li, Azarakhsh Keipour, Kevin Jamieson, Nicolas Hudson, Charles Swan, Kostas Bekris

Abstract: Automating warehouse operations can reduce logistics overhead costs, ultimately driving down the final price for consumers, increasing the speed of delivery, and enhancing the resiliency to workforce fluctuations. The past few years have seen increased interest in automating such repeated tasks but mostly in controlled settings. Tasks such as picking objects from unstructured, cluttered piles have… ▽ More Automating warehouse operations can reduce logistics overhead costs, ultimately driving down the final price for consumers, increasing the speed of delivery, and enhancing the resiliency to workforce fluctuations. The past few years have seen increased interest in automating such repeated tasks but mostly in controlled settings. Tasks such as picking objects from unstructured, cluttered piles have only recently become robust enough for large-scale deployment with minimal human intervention. This paper demonstrates a large-scale package manipulation from unstructured piles in Amazon Robotics' Robot Induction (Robin) fleet, which utilizes a pick success predictor trained on real production data. Specifically, the system was trained on over 394K picks. It is used for singulating up to 5 million packages per day and has manipulated over 200 million packages during this paper's evaluation period. The developed learned pick quality measure ranks various pick alternatives in real-time and prioritizes the most promising ones for execution. The pick success predictor aims to estimate from prior experience the success probability of a desired pick by the deployed industrial robotic arms in cluttered scenes containing deformable and rigid objects with partially known properties. It is a shallow machine learning model, which allows us to evaluate which features are most important for the prediction. An online pick ranker leverages the learned success predictor to prioritize the most promising picks for the robotic arm, which are then assessed for collision avoidance. This learned ranking process is demonstrated to overcome the limitations and outperform the performance of manually engineered and heuristic alternatives. To the best of the authors' knowledge, this paper presents the first large-scale deployment of learned pick quality estimation methods in a real production system. △ Less

Submitted 27 June, 2023; v1 submitted 17 May, 2023; originally announced May 2023.

Comments: Robotics: Science and Systems (RSS 2023) conference, July 10 - 14, 2023, Daegu, Republic of Korea

Uplink MIMO Detection using Ising Machines: A Multi-Stage Ising Approach

Authors: Abhishek Kumar Singh, Ari Kapelyan, Davide Venturelli, Kyle Jamieson

Abstract: Multiple-Input-Multiple-Output~(MIMO) signal detection is central to every state-of-the-art communication system, and enhancements in error performance and computation complexity of MIMO detection would significantly enhance data rate and latency experienced by the users. Theoretically, the optimal MIMO detector is the maximum-likelihood (ML) MIMO detector; however, due to its extremely high compl… ▽ More Multiple-Input-Multiple-Output~(MIMO) signal detection is central to every state-of-the-art communication system, and enhancements in error performance and computation complexity of MIMO detection would significantly enhance data rate and latency experienced by the users. Theoretically, the optimal MIMO detector is the maximum-likelihood (ML) MIMO detector; however, due to its extremely high complexity, it is not feasible for large real-world communication systems. Over the past few years, algorithms based on physics-inspired Ising solvers, like Coherent Ising machines and Quantum Annealers, have shown significant performance improvements for the MIMO detection problem. However, the current state-of-the-art is limited to low-order modulations or systems with few users. In this paper, we propose an adaptive multi-stage Ising machine-based MIMO detector that extends the performance gains of physics-inspired computation to Large and Massive MIMO systems with a large number of users and very high modulation schemes~(up to 256-QAM). We enhance our previously proposed delta Ising formulation and develop a heuristic that adaptively optimizes the performance and complexity of our proposed method. We perform extensive micro-benchmarking to optimize several free parameters of the system and evaluate our methods' BER and spectral efficiency for Large and Massive MIMO systems (up to 32 users and 256 QAM modulation). △ Less

Submitted 25 April, 2023; originally announced April 2023.

Comments: Initial draft. arXiv admin note: text overlap with arXiv:2205.05020

Instance-dependent Sample Complexity Bounds for Zero-sum Matrix Games

Authors: Arnab Maiti, Kevin Jamieson, Lillian J. Ratliff

Abstract: We study the sample complexity of identifying an approximate equilibrium for two-player zero-sum $n\times 2$ matrix games. That is, in a sequence of repeated game plays, how many rounds must the two players play before reaching an approximate equilibrium (e.g., Nash)? We derive instance-dependent bounds that define an ordering over game matrices that captures the intuition that the dynamics of som… ▽ More We study the sample complexity of identifying an approximate equilibrium for two-player zero-sum $n\times 2$ matrix games. That is, in a sequence of repeated game plays, how many rounds must the two players play before reaching an approximate equilibrium (e.g., Nash)? We derive instance-dependent bounds that define an ordering over game matrices that captures the intuition that the dynamics of some games converge faster than others. Specifically, we consider a stochastic observation model such that when the two players choose actions $i$ and $j$, respectively, they both observe each other's played actions and a stochastic observation $X_{ij}$ such that $\mathbb E[ X_{ij}] = A_{ij}$. To our knowledge, our work is the first case of instance-dependent lower bounds on the number of rounds the players must play before reaching an approximate equilibrium in the sense that the number of rounds depends on the specific properties of the game matrix $A$ as well as the desired accuracy. We also prove a converse statement: there exist player strategies that achieve this lower bound. △ Less

Submitted 19 March, 2023; originally announced March 2023.

Comments: 53 pages, Accepted at AISTATS 2023

Cross-Link Channel Prediction for Massive IoT Networks

Authors: Kun Woo Cho, Marco Cominelli, Francesco Gringoli, Joerg Widmer, Kyle Jamieson

Abstract: Tomorrow's massive-scale IoT sensor networks are poised to drive uplink traffic demand, especially in areas of dense deployment. To meet this demand, however, network designers leverage tools that often require accurate estimates of Channel State Information (CSI), which incurs a high overhead and thus reduces network throughput. Furthermore, the overhead generally scales with the number of client… ▽ More Tomorrow's massive-scale IoT sensor networks are poised to drive uplink traffic demand, especially in areas of dense deployment. To meet this demand, however, network designers leverage tools that often require accurate estimates of Channel State Information (CSI), which incurs a high overhead and thus reduces network throughput. Furthermore, the overhead generally scales with the number of clients, and so is of special concern in such massive IoT sensor networks. While prior work has used transmissions over one frequency band to predict the channel of another frequency band on the same link, this paper takes the next step in the effort to reduce CSI overhead: predict the CSI of a nearby but distinct link. We propose Cross-Link Channel Prediction (CLCP), a technique that leverages multi-view representation learning to predict the channel response of a large number of users, thereby reducing channel estimation overhead further than previously possible. CLCP's design is highly practical, exploiting channel estimates obtained from existing transmissions instead of dedicated channel sounding or extra pilot signals. We have implemented CLCP for two different Wi-Fi versions, namely 802.11n and 802.11ax, the latter being the leading candidate for future IoT networks. We evaluate CLCP in two large-scale indoor scenarios involving both line-of-sight and non-line-of-sight transmissions with up to 144 different 802.11ax users. Moreover, we measure its performance with four different channel bandwidths, from 20 MHz up to 160 MHz. Our results show that CLCP provides a 2x throughput gain over baseline 802.11ax and a 30 percent throughput gain over existing cross-band prediction algorithms. △ Less

Submitted 15 December, 2022; originally announced December 2022.

Comments: 13 pages, 12 figures

Decoding Polar Codes via Noisy Quantum Gates: Quantum Circuits and Insights

Authors: Srikar Kasi, John Kaewell, Shahab Hamidi-Rad, Kyle Jamieson

Abstract: The use of quantum computation for wireless network applications is emerging as a promising paradigm to bridge the performance gap between in-practice and optimal wireless algorithms. While today's quantum technology offers limited number of qubits and low fidelity gates, application-based quantum solutions help us to understand and improve the performance of such technology even further. This pap… ▽ More The use of quantum computation for wireless network applications is emerging as a promising paradigm to bridge the performance gap between in-practice and optimal wireless algorithms. While today's quantum technology offers limited number of qubits and low fidelity gates, application-based quantum solutions help us to understand and improve the performance of such technology even further. This paper introduces QGateD-Polar, a novel Quantum Gate-based Maximum-Likelihood Decoder design for Polar error correction codes, which are becoming widespread in today's 5G and tomorrow's NextG wireless networks. QGateD-Polar uses quantum gates to dictate the time evolution of Polar code decoding -- from the received wireless soft data to the final decoded solution -- by leveraging quantum phenomena such as superposition, entanglement, and interference, making it amenable to quantum gate-based computers. Our early results show that QGateD-Polar achieves the Maximum Likelihood performance in ideal quantum simulations, demonstrating how performance varies with noise. △ Less

Submitted 19 October, 2022; originally announced October 2022.

mmWall: A Transflective Metamaterial Surface for mmWave Networks

Authors: Kun Woo Cho, Mohammad H. Mazaheri, Jeremy Gummeson, Omid Abari, Kyle Jamieson

Abstract: Mobile operators are poised to leverage millimeter wave technology as 5G evolves, but despite efforts to bolster their reliability indoors and outdoors, mmWave links remain vulnerable to blockage by walls, people, and obstacles. Further, there is significant interest in bringing outdoor mmWave coverage indoors, which for similar reasons remains challenging today. This paper presents the design, ha… ▽ More Mobile operators are poised to leverage millimeter wave technology as 5G evolves, but despite efforts to bolster their reliability indoors and outdoors, mmWave links remain vulnerable to blockage by walls, people, and obstacles. Further, there is significant interest in bringing outdoor mmWave coverage indoors, which for similar reasons remains challenging today. This paper presents the design, hardware implementation, and experimental evaluation of mmWall, the first electronically almost-360 degree steerable metamaterial surface that operates above 24 GHz and both refracts or reflects incoming mmWave transmissions. Our metamaterial design consists of arrays of varactor-split ring resonator unit cells, miniaturized for mmWave. Custom control circuitry drives each resonator, overcoming coupling challenges that arise at scale. Leveraging beam steering algorithms, we integrate mmWall into the link layer discovery protocols of common mmWave networks. We have fabricated a 10 cm by 20 cm mmWall prototype consisting of a 28 by 76 unit cell array, and evaluate in indoor, outdoor-to-indoor, and multi-beam scenarios. Indoors, mmWall guarantees 91% of locations outage-free under 128-QAM mmWave data rates and boosts SNR by up to 15 dB. Outdoors, mmWall reduces the probability of complete link failure by a ratio of up to 40% under 0-80% path blockage and boosts SNR by up to 30 dB. △ Less

Submitted 25 September, 2022; v1 submitted 23 September, 2022; originally announced September 2022.

Comments: 18 pages, 18 figures

Instance-Dependent Near-Optimal Policy Identification in Linear MDPs via Online Experiment Design

Authors: Andrew Wagenmaker, Kevin Jamieson

Abstract: While much progress has been made in understanding the minimax sample complexity of reinforcement learning (RL) -- the complexity of learning on the "worst-case" instance -- such measures of complexity often do not capture the true difficulty of learning. In practice, on an "easy" instance, we might hope to achieve a complexity far better than that achievable on the worst-case instance. In this wo… ▽ More While much progress has been made in understanding the minimax sample complexity of reinforcement learning (RL) -- the complexity of learning on the "worst-case" instance -- such measures of complexity often do not capture the true difficulty of learning. In practice, on an "easy" instance, we might hope to achieve a complexity far better than that achievable on the worst-case instance. In this work we seek to understand the "instance-dependent" complexity of learning near-optimal policies (PAC RL) in the setting of RL with linear function approximation. We propose an algorithm, \textsc{Pedel}, which achieves a fine-grained instance-dependent measure of complexity, the first of its kind in the RL with function approximation setting, thereby capturing the difficulty of learning on each particular problem instance. Through an explicit example, we show that \textsc{Pedel} yields provable gains over low-regret, minimax-optimal algorithms and that such algorithms are unable to hit the instance-optimal rate. Our approach relies on a novel online experiment design-based procedure which focuses the exploration budget on the "directions" most relevant to learning a near-optimal policy, and may be of independent interest. △ Less

Submitted 20 July, 2023; v1 submitted 6 July, 2022; originally announced July 2022.

Instance-optimal PAC Algorithms for Contextual Bandits

Authors: Zhaoqi Li, Lillian Ratliff, Houssam Nassif, Kevin Jamieson, Lalit Jain

Abstract: In the stochastic contextual bandit setting, regret-minimizing algorithms have been extensively researched, but their instance-minimizing best-arm identification counterparts remain seldom studied. In this work, we focus on the stochastic bandit problem in the $(ε,δ)$-$\textit{PAC}$ setting: given a policy class $Π$ the goal of the learner is to return a policy $π\in Π$ whose expected reward is wi… ▽ More In the stochastic contextual bandit setting, regret-minimizing algorithms have been extensively researched, but their instance-minimizing best-arm identification counterparts remain seldom studied. In this work, we focus on the stochastic bandit problem in the $(ε,δ)$-$\textit{PAC}$ setting: given a policy class $Π$ the goal of the learner is to return a policy $π\in Π$ whose expected reward is within $ε$ of the optimal policy with probability greater than $1-δ$. We characterize the first $\textit{instance-dependent}$ PAC sample complexity of contextual bandits through a quantity $ρ_Π$, and provide matching upper and lower bounds in terms of $ρ_Π$ for the agnostic and linear contextual best-arm identification settings. We show that no algorithm can be simultaneously minimax-optimal for regret minimization and instance-dependent PAC for best-arm identification. Our main result is a new instance-optimal and computationally efficient algorithm that relies on a polynomial number of calls to an argmax oracle. △ Less

Submitted 5 July, 2022; originally announced July 2022.

Journal ref: Conference on Neural Information Processing Systems (NeurIPS'22), New Orleans, pp. 37590-37603, 2022

Towards Dual-band Reconfigurable Metamaterial Surfaces for Satellite Networking

Authors: Kun Woo Cho, Yasaman Ghasempour, Kyle Jamieson

Abstract: The first low earth orbit satellite networks for internet service have recently been deployed and are growing in size, yet will face deployment challenges in many practical circumstances of interest. This paper explores how a dual-band, electronically tunable smart surface can enable dynamic beam alignment between the satellite and mobile users, make service possible in urban canyons, and improve… ▽ More The first low earth orbit satellite networks for internet service have recently been deployed and are growing in size, yet will face deployment challenges in many practical circumstances of interest. This paper explores how a dual-band, electronically tunable smart surface can enable dynamic beam alignment between the satellite and mobile users, make service possible in urban canyons, and improve service in rural areas. Our design is the first of its kind to target dual channels in the Ku radio frequency band with a novel dual Huygens resonator design that leverages radio reciprocity to allow our surface to simultaneously steer and modulate energy in the satellite uplink and downlink directions, and in both reflective and transmissive modes of operation. Our surface, Wall-E, is designed and evaluated in an electromagnetic simulator and demonstrates 94% transmission efficiency and an 85% reflection efficiency, with at most 6 dB power loss at steering angles over a 150-degree field of view for both transmission and reflection. With a 75 sq cm surface, our link budget calculations predict a 4 dB and 24 dB improvement in the SNR of a link entering the window of a rural home in comparison to the free-space path and brick wall penetration, respectively. △ Less

Submitted 29 June, 2022; originally announced June 2022.

Comments: 9 pages including references, 9 figures

ACM Class: C.2.5; C.3

Active Learning with Safety Constraints

Authors: Romain Camilleri, Andrew Wagenmaker, Jamie Morgenstern, Lalit Jain, Kevin Jamieson

Abstract: Active learning methods have shown great promise in reducing the number of samples necessary for learning. As automated learning systems are adopted into real-time, real-world decision-making pipelines, it is increasingly important that such algorithms are designed with safety in mind. In this work we investigate the complexity of learning the best safe decision in interactive environments. We red… ▽ More Active learning methods have shown great promise in reducing the number of samples necessary for learning. As automated learning systems are adopted into real-time, real-world decision-making pipelines, it is increasingly important that such algorithms are designed with safety in mind. In this work we investigate the complexity of learning the best safe decision in interactive environments. We reduce this problem to a constrained linear bandits problem, where our goal is to find the best arm satisfying certain (unknown) safety constraints. We propose an adaptive experimental design-based algorithm, which we show efficiently trades off between the difficulty of showing an arm is unsafe vs suboptimal. To our knowledge, our results are the first on best-arm identification in linear bandits with safety constraints. In practice, we demonstrate that this approach performs well on synthetic and real world datasets. △ Less

Submitted 22 June, 2022; originally announced June 2022.

A Finite-Range Search Formulation of Maximum Likelihood MIMO Detection for Coherent Ising Machines

Authors: Abhishek Kumar Singh, Davide Venturelli, Kyle Jamieson

Abstract: The last couple of years have seen an emergence of physics-inspired computing for maximum likelihood MIMO detection. These methods involve transforming the MIMO detection problem into an Ising minimization problem, which can then be solved on an Ising Machine. Recent works have shown promising projections for MIMO wireless detection using Quantum Annealing optimizers and Coherent Ising Machines. W… ▽ More The last couple of years have seen an emergence of physics-inspired computing for maximum likelihood MIMO detection. These methods involve transforming the MIMO detection problem into an Ising minimization problem, which can then be solved on an Ising Machine. Recent works have shown promising projections for MIMO wireless detection using Quantum Annealing optimizers and Coherent Ising Machines. While these methods perform very well for BPSK and 4-QAM, they struggle to provide good BER for 16-QAM and higher modulations. In this paper, we explore an enhanced CIM model, and propose a novel Ising formulation, which together are shown to be the first Ising solver that provides significant gains in the BER performance of large and massive MIMO systems, like $16\times16$ and $16\times32$, and sustain its performance gain even at 256-QAM modulation. We further perform a spectral efficiency analysis and show that, for a $16\times16$ MIMO with Adaptive Modulation and Coding, our method can provide substantial throughput gains over MMSE, achieving $2\times$ throughput for SNR $\leq25$ dB, and up to $1.5\times$ throughput for SNR $\geq 30$ dB. △ Less

Submitted 10 May, 2022; originally announced May 2022.

Comments: Article under review for IEEE Globecom 2022

Dashlet: Taming Swipe Uncertainty for Robust Short Video Streaming

Authors: Zhuqi Li, Yaxiong Xie, Ravi Netravali, Kyle Jamieson

Abstract: Short video streaming applications have recently gained substantial traction, but the non-linear video presentation they afford swi** users fundamentally changes the problem of maximizing user quality of experience in the face of the vagaries of network throughput and user swipe timing. This paper describes the design and implementation of Dashlet, a system tailored for high quality of experienc… ▽ More Short video streaming applications have recently gained substantial traction, but the non-linear video presentation they afford swi** users fundamentally changes the problem of maximizing user quality of experience in the face of the vagaries of network throughput and user swipe timing. This paper describes the design and implementation of Dashlet, a system tailored for high quality of experience in short video streaming applications. With the insights we glean from an in-the-wild TikTok performance study and a user study focused on swipe patterns, Dashlet proposes a novel out-of-order video chunk pre-buffering mechanism that leverages a simple, non machine learning-based model of users' swipe statistics to determine the pre-buffering order and bitrate. The net result is a system that achieves 77-99% of an oracle system's QoE and outperforms TikTok by 43.9-45.1x, while also reducing by 30% the number of bytes wasted on downloaded video that is never watched. △ Less

Submitted 27 April, 2022; originally announced April 2022.

TreeStep: Tree Search for Vector Perturbation Precoding under per-Antenna Power Constraint

Authors: Abhishek Kumar Singh, Kyle Jamieson

Abstract: Vector Perturbation Precoding (VPP) can speed up downlink data transmissions in Large and Massive Multi-User MIMO systems but is known to be NP-hard. While there are several algorithms in the literature for VPP under total power constraint, they are not applicable for VPP under per-antenna power constraint. This paper proposes a novel, parallel tree search algorithm for VPP under per-antenna power… ▽ More Vector Perturbation Precoding (VPP) can speed up downlink data transmissions in Large and Massive Multi-User MIMO systems but is known to be NP-hard. While there are several algorithms in the literature for VPP under total power constraint, they are not applicable for VPP under per-antenna power constraint. This paper proposes a novel, parallel tree search algorithm for VPP under per-antenna power constraint, called \emph{\textbf{TreeStep}}, to find good quality solutions to the VPP problem with practical computational complexity. We show that our method can provide huge performance gain over simple linear precoding like Regularised Zero Forcing. We evaluate TreeStep for several large MIMO~($16\times16$ and $24\times24$) and massive MIMO~($16\times32$ and $24\times 48$) and demonstrate that TreeStep outperforms the popular polynomial-time VPP algorithm, the Fixed Complexity Sphere Encoder, by achieving the extremely low BER of $10^{-6}$ at a much lower SNR. △ Less

Submitted 26 April, 2022; v1 submitted 15 April, 2022; originally announced April 2022.

Comments: Article under review for IEEE Globecom 22

Active Multi-Task Representation Learning

Authors: Yifang Chen, Simon S. Du, Kevin Jamieson

Abstract: To leverage the power of big data from source tasks and overcome the scarcity of the target task samples, representation learning based on multi-task pretraining has become a standard approach in many applications. However, up until now, choosing which source tasks to include in the multi-task learning has been more art than science. In this paper, we give the first formal study on resource task s… ▽ More To leverage the power of big data from source tasks and overcome the scarcity of the target task samples, representation learning based on multi-task pretraining has become a standard approach in many applications. However, up until now, choosing which source tasks to include in the multi-task learning has been more art than science. In this paper, we give the first formal study on resource task sampling by leveraging the techniques from active learning. We propose an algorithm that iteratively estimates the relevance of each source task to the target task and samples from each source task based on the estimated relevance. Theoretically, we show that for the linear representation class, to achieve the same error rate, our algorithm can save up to a \textit{number of source tasks} factor in the source task sample complexity, compared with the naive uniform sampling from all source tasks. We also provide experiments on real-world computer vision datasets to illustrate the effectiveness of our proposed method on both linear and convolutional neural network representation classes. We believe our paper serves as an important initial step to bring techniques from active learning to representation learning. △ Less

Submitted 2 February, 2022; originally announced February 2022.

Reward-Free RL is No Harder Than Reward-Aware RL in Linear Markov Decision Processes

Authors: Andrew Wagenmaker, Yifang Chen, Max Simchowitz, Simon S. Du, Kevin Jamieson

Abstract: Reward-free reinforcement learning (RL) considers the setting where the agent does not have access to a reward function during exploration, but must propose a near-optimal policy for an arbitrary reward function revealed only after exploring. In the the tabular setting, it is well known that this is a more difficult problem than reward-aware (PAC) RL -- where the agent has access to the reward fun… ▽ More Reward-free reinforcement learning (RL) considers the setting where the agent does not have access to a reward function during exploration, but must propose a near-optimal policy for an arbitrary reward function revealed only after exploring. In the the tabular setting, it is well known that this is a more difficult problem than reward-aware (PAC) RL -- where the agent has access to the reward function during exploration -- with optimal sample complexities in the two settings differing by a factor of $|\mathcal{S}|$, the size of the state space. We show that this separation does not exist in the setting of linear MDPs. We first develop a computationally efficient algorithm for reward-free RL in a $d$-dimensional linear MDP with sample complexity scaling as $\widetilde{\mathcal{O}}(d^2 H^5/ε^2)$. We then show a lower bound with matching dimension-dependence of $Ω(d^2 H^2/ε^2)$, which holds for the reward-aware RL setting. To our knowledge, our approach is the first computationally efficient algorithm to achieve optimal $d$ dependence in linear MDPs, even in the single-reward PAC setting. Our algorithm relies on a novel procedure which efficiently traverses a linear MDP, collecting samples in any given ``feature direction'', and enjoys a sample complexity scaling optimally in the (linear MDP equivalent of the) maximal state visitation probability. We show that this exploration procedure can also be applied to solve the problem of obtaining ``well-conditioned'' covariates in linear MDPs. △ Less

Submitted 18 June, 2022; v1 submitted 26 January, 2022; originally announced January 2022.

NG-Scope: Fine-Grained Telemetry for NextG Cellular Networks

Authors: Yaxiong Xie, Kyle Jamieson

Abstract: Accurate and highly-granular channel capacity telemetry of the cellular last hop is crucial for the effective operation of transport layer protocols and cutting-edge applications, such as video on demand and videotelephony. This paper presents the design, implementation, and experimental performance evaluation of NG-Scope, the first such telemetry tool able to fuse physical-layer channel occupancy… ▽ More Accurate and highly-granular channel capacity telemetry of the cellular last hop is crucial for the effective operation of transport layer protocols and cutting-edge applications, such as video on demand and videotelephony. This paper presents the design, implementation, and experimental performance evaluation of NG-Scope, the first such telemetry tool able to fuse physical-layer channel occupancy readings from the cellular control channel with higher-layer packet arrival statistics and make accurate capacity estimates. NG-Scope handles the latest cellular innovations, such as when multiple base stations aggregate their signals together to serve mobile users. End-to-end experiments in a commercial cellular network demonstrate that wireless capacity varies significantly with channel quality, mobility, competing traffic within each cell, and the number of aggregated cells. Our experiments demonstrate significantly improved cell load estimation accuracy, missing the detection of less than 1% of data capacity overall, a reduction of 82% compared to OWL, the state-of-the-art in cellular monitoring. Further experiments show that MobileInsight-based CLAW has a root-mean-squared capacity error of 30.5 Mbit/s, which is 3.3x larger than NG-Scope (9.2 Mbit/s) △ Less

Submitted 18 January, 2022; v1 submitted 13 January, 2022; originally announced January 2022.

CurvingLoRa to Boost LoRa Network Capacity via Concurrent Transmission

Authors: Chenning Li, Xiuzhen Guo, Longfei Shangguan, Zhichao Cao, Kyle Jamieson

Abstract: LoRaWAN has emerged as an appealing technology to connect IoT devices but it functions without explicit coordination among transmitters, which can lead to many packet collisions as the network scales. State-of-the-art work proposes various approaches to deal with these collisions, but most functions only in high signal-to-interference ratio (SIR) conditions and thus does not scale to many scenario… ▽ More LoRaWAN has emerged as an appealing technology to connect IoT devices but it functions without explicit coordination among transmitters, which can lead to many packet collisions as the network scales. State-of-the-art work proposes various approaches to deal with these collisions, but most functions only in high signal-to-interference ratio (SIR) conditions and thus does not scale to many scenarios where weak receptions are easily buried by stronger receptions from nearby transmitters. In this paper, we take a fresh look at LoRa's physical layer, revealing that its underlying linear chirp modulation fundamentally limits the capacity and scalability of concurrentLoRa transmissions. We show that by replacing linear chirps with their non-linear counterparts, we can boost the capacity of concurrent LoRa transmissions and empower the LoRa receiver to successfully receive weak transmissions in the presence of strong colliding signals. Such a non-linear chirp design further enables the receiver to demodulate fully aligned collision symbols - a case where none of the existing approaches can deal with. We implement these ideas in a holistic LoRaWAN stack based on the USRP N210 software-defined radio platform. Our head-to-head comparison with two state-of-the-art research systems and a standard LoRaWAN baseline demonstrates that CurvingLoRa improves the network throughput by 1.6-7.6x while simultaneously sacrificing neither power efficiency nor noise resilience. An open-source dataset and code will be made available before publication. △ Less

Submitted 13 January, 2022; originally announced January 2022.

First-Order Regret in Reinforcement Learning with Linear Function Approximation: A Robust Estimation Approach

Authors: Andrew Wagenmaker, Yifang Chen, Max Simchowitz, Simon S. Du, Kevin Jamieson

Abstract: Obtaining first-order regret bounds -- regret bounds scaling not as the worst-case but with some measure of the performance of the optimal policy on a given instance -- is a core question in sequential decision-making. While such bounds exist in many settings, they have proven elusive in reinforcement learning with large state spaces. In this work we address this gap, and show that it is possible… ▽ More Obtaining first-order regret bounds -- regret bounds scaling not as the worst-case but with some measure of the performance of the optimal policy on a given instance -- is a core question in sequential decision-making. While such bounds exist in many settings, they have proven elusive in reinforcement learning with large state spaces. In this work we address this gap, and show that it is possible to obtain regret scaling as $\widetilde{\mathcal{O}}(\sqrt{d^3 H^3 \cdot V_1^\star \cdot K} + d^{3.5}H^3\log K )$ in reinforcement learning with large state spaces, namely the linear MDP setting. Here $V_1^\star$ is the value of the optimal policy and $K$ is the number of episodes. We demonstrate that existing techniques based on least squares estimation are insufficient to obtain this result, and instead develop a novel robust self-normalized concentration bound based on the robust Catoni mean estimator, which may be of independent interest. △ Less

Submitted 20 October, 2022; v1 submitted 6 December, 2021; originally announced December 2021.

Best Arm Identification with Safety Constraints

Authors: Zhenlin Wang, Andrew Wagenmaker, Kevin Jamieson

Abstract: The best arm identification problem in the multi-armed bandit setting is an excellent model of many real-world decision-making problems, yet it fails to capture the fact that in the real-world, safety constraints often must be met while learning. In this work we study the question of best-arm identification in safety-critical settings, where the goal of the agent is to find the best safe option ou… ▽ More The best arm identification problem in the multi-armed bandit setting is an excellent model of many real-world decision-making problems, yet it fails to capture the fact that in the real-world, safety constraints often must be met while learning. In this work we study the question of best-arm identification in safety-critical settings, where the goal of the agent is to find the best safe option out of many, while exploring in a way that guarantees certain, initially unknown safety constraints are met. We first analyze this problem in the setting where the reward and safety constraint takes a linear structure, and show nearly matching upper and lower bounds. We then analyze a much more general version of the problem where we only assume the reward and safety constraint can be modeled by monotonic functions, and propose an algorithm in this setting which is guaranteed to learn safely. We conclude with experimental results demonstrating the effectiveness of our approaches in scenarios such as safely identifying the best drug out of many in order to treat an illness. △ Less

Submitted 23 November, 2021; originally announced November 2021.

Practical, Provably-Correct Interactive Learning in the Realizable Setting: The Power of True Believers

Authors: Julian Katz-Samuels, Blake Mason, Kevin Jamieson, Rob Nowak

Abstract: We consider interactive learning in the realizable setting and develop a general framework to handle problems ranging from best arm identification to active classification. We begin our investigation with the observation that agnostic algorithms \emph{cannot} be minimax-optimal in the realizable setting. Hence, we design novel computationally efficient algorithms for the realizable setting that ma… ▽ More We consider interactive learning in the realizable setting and develop a general framework to handle problems ranging from best arm identification to active classification. We begin our investigation with the observation that agnostic algorithms \emph{cannot} be minimax-optimal in the realizable setting. Hence, we design novel computationally efficient algorithms for the realizable setting that match the minimax lower bound up to logarithmic factors and are general-purpose, accommodating a wide variety of function classes including kernel methods, H{ö}lder smooth functions, and convex functions. The sample complexities of our algorithms can be quantified in terms of well-known quantities like the extended teaching dimension and haystack dimension. However, unlike algorithms based directly on those combinatorial quantities, our algorithms are computationally efficient. To achieve computational efficiency, our algorithms sample from the version space using Monte Carlo "hit-and-run" algorithms instead of maintaining the version space explicitly. Our approach has two key strengths. First, it is simple, consisting of two unifying, greedy algorithms. Second, our algorithms have the capability to seamlessly leverage prior knowledge that is often available and useful in practice. In addition to our new theoretical results, we demonstrate empirically that our algorithms are competitive with Gaussian process UCB methods. △ Less

Submitted 8 November, 2021; originally announced November 2021.

Nearly Optimal Algorithms for Level Set Estimation

Authors: Blake Mason, Romain Camilleri, Subhojyoti Mukherjee, Kevin Jamieson, Robert Nowak, Lalit Jain

Abstract: The level set estimation problem seeks to find all points in a domain ${\cal X}$ where the value of an unknown function $f:{\cal X}\rightarrow \mathbb{R}$ exceeds a threshold $α$. The estimation is based on noisy function evaluations that may be acquired at sequentially and adaptively chosen locations in ${\cal X}$. The threshold value $α$ can either be \emph{explicit} and provided a priori, or \e… ▽ More The level set estimation problem seeks to find all points in a domain ${\cal X}$ where the value of an unknown function $f:{\cal X}\rightarrow \mathbb{R}$ exceeds a threshold $α$. The estimation is based on noisy function evaluations that may be acquired at sequentially and adaptively chosen locations in ${\cal X}$. The threshold value $α$ can either be \emph{explicit} and provided a priori, or \emph{implicit} and defined relative to the optimal function value, i.e. $α= (1-ε)f(x_\ast)$ for a given $ε> 0$ where $f(x_\ast)$ is the maximal function value and is unknown. In this work we provide a new approach to the level set estimation problem by relating it to recent adaptive experimental design methods for linear bandits in the Reproducing Kernel Hilbert Space (RKHS) setting. We assume that $f$ can be approximated by a function in the RKHS up to an unknown misspecification and provide novel algorithms for both the implicit and explicit cases in this setting with strong theoretical guarantees. Moreover, in the linear (kernel) setting, we show that our bounds are nearly optimal, namely, our upper bounds match existing lower bounds for threshold linear bandits. To our knowledge this work provides the first instance-dependent, non-asymptotic upper bounds on sample complexity of level-set estimation that match information theoretic lower bounds. △ Less

Submitted 2 November, 2021; originally announced November 2021.

Comments: 9 pages + appendices. 6 Figures

Selective Sampling for Online Best-arm Identification

Authors: Romain Camilleri, Zhihan Xiong, Maryam Fazel, Lalit Jain, Kevin Jamieson

Abstract: This work considers the problem of selective-sampling for best-arm identification. Given a set of potential options $\mathcal{Z}\subset\mathbb{R}^d$, a learner aims to compute with probability greater than $1-δ$, $\arg\max_{z\in \mathcal{Z}} z^{\top}θ_{\ast}$ where $θ_{\ast}$ is unknown. At each time step, a potential measurement $x_t\in \mathcal{X}\subset\mathbb{R}^d$ is drawn IID and the learner… ▽ More This work considers the problem of selective-sampling for best-arm identification. Given a set of potential options $\mathcal{Z}\subset\mathbb{R}^d$, a learner aims to compute with probability greater than $1-δ$, $\arg\max_{z\in \mathcal{Z}} z^{\top}θ_{\ast}$ where $θ_{\ast}$ is unknown. At each time step, a potential measurement $x_t\in \mathcal{X}\subset\mathbb{R}^d$ is drawn IID and the learner can either choose to take the measurement, in which case they observe a noisy measurement of $x^{\top}θ_{\ast}$, or to abstain from taking the measurement and wait for a potentially more informative point to arrive in the stream. Hence the learner faces a fundamental trade-off between the number of labeled samples they take and when they have collected enough evidence to declare the best arm and stop sampling. The main results of this work precisely characterize this trade-off between labeled samples and stop** time and provide an algorithm that nearly-optimally achieves the minimal label complexity given a desired stop** time. In addition, we show that the optimal decision rule has a simple geometric form based on deciding whether a point is in an ellipse or not. Finally, our framework is general enough to capture binary classification improving upon previous works. △ Less

Submitted 1 November, 2021; v1 submitted 27 October, 2021; originally announced October 2021.

Comments: 42 pages, 2 figures, Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS 2021)

A Cost and Power Feasibility Analysis of Quantum Annealing for NextG Cellular Wireless Networks

Authors: Srikar Kasi, P. A. Warburton, John Kaewell, Kyle Jamieson

Abstract: In order to meet mobile cellular users' ever-increasing data demands, today's 4G and 5G networks are designed mainly with the goal of maximizing spectral efficiency. While they have made progress in this regard, controlling the carbon footprint and operational costs of such networks remains a long-standing problem among network designers. This paper takes a long view on this problem, envisioning a… ▽ More In order to meet mobile cellular users' ever-increasing data demands, today's 4G and 5G networks are designed mainly with the goal of maximizing spectral efficiency. While they have made progress in this regard, controlling the carbon footprint and operational costs of such networks remains a long-standing problem among network designers. This paper takes a long view on this problem, envisioning a NextG scenario where the network leverages quantum annealing for cellular baseband processing. We gather and synthesize insights on power consumption, computational throughput and latency, spectral efficiency, operational cost, and feasibility timelines surrounding quantum technology. Armed with these data, we analyze and project the quantitative performance targets future quantum annealing hardware must meet in order to provide a computational and power advantage over CMOS hardware, while matching its whole-network spectral efficiency. Our quantitative analysis predicts that with quantum annealing hardware operating at a 102 $μ$s problem latency and 3.1M qubits, quantum annealing will achieve a spectral efficiency equal to CMOS computation while reducing power consumption by 41 kW (45% lower) in a representative 5G base station scenario with 400 MHz bandwidth and 64 antennas, and an 8 kW power reduction (16% lower) using 1.5M qubits in a 200 MHz-bandwidth 5G scenario. △ Less

Submitted 14 January, 2022; v1 submitted 3 September, 2021; originally announced September 2021.

Beyond No Regret: Instance-Dependent PAC Reinforcement Learning

Authors: Andrew Wagenmaker, Max Simchowitz, Kevin Jamieson

Abstract: The theory of reinforcement learning has focused on two fundamental problems: achieving low regret, and identifying $ε$-optimal policies. While a simple reduction allows one to apply a low-regret algorithm to obtain an $ε$-optimal policy and achieve the worst-case optimal rate, it is unknown whether low-regret algorithms can obtain the instance-optimal rate for policy identification. We show this… ▽ More The theory of reinforcement learning has focused on two fundamental problems: achieving low regret, and identifying $ε$-optimal policies. While a simple reduction allows one to apply a low-regret algorithm to obtain an $ε$-optimal policy and achieve the worst-case optimal rate, it is unknown whether low-regret algorithms can obtain the instance-optimal rate for policy identification. We show this is not possible -- there exists a fundamental tradeoff between achieving low regret and identifying an $ε$-optimal policy at the instance-optimal rate. Motivated by our negative finding, we propose a new measure of instance-dependent sample complexity for PAC tabular reinforcement learning which explicitly accounts for the attainable state visitation distributions in the underlying MDP. We then propose and analyze a novel, planning-based algorithm which attains this sample complexity -- yielding a complexity which scales with the suboptimality gaps and the "reachability" of a state. We show our algorithm is nearly minimax optimal, and on several examples that our instance-dependent sample complexity offers significant improvements over worst-case bounds. △ Less

Submitted 21 June, 2022; v1 submitted 5 August, 2021; originally announced August 2021.