Showing 1–50 of 86 results for author: Bates, S

Data-Adaptive Tradeoffs among Multiple Risks in Distribution-Free Prediction

Authors: Drew T. Nguyen, Reese Pathak, Anastasios N. Angelopoulos, Stephen Bates, Michael I. Jordan

Abstract: Decision-making pipelines are generally characterized by tradeoffs among various risk functions. It is often desirable to manage such tradeoffs in a data-adaptive manner. As we demonstrate, if this is done naively, state-of-the art uncertainty quantification methods can lead to significant violations of putative risk guarantees. To address this issue, we develop methods that permit valid control… ▽ More Decision-making pipelines are generally characterized by tradeoffs among various risk functions. It is often desirable to manage such tradeoffs in a data-adaptive manner. As we demonstrate, if this is done naively, state-of-the art uncertainty quantification methods can lead to significant violations of putative risk guarantees. To address this issue, we develop methods that permit valid control of risk when threshold and tradeoff parameters are chosen adaptively. Our methodology supports monotone and nearly-monotone risks, but otherwise makes no distributional assumptions. To illustrate the benefits of our approach, we carry out numerical experiments on synthetic data and the large-scale vision dataset MS-COCO. △ Less

Submitted 28 March, 2024; originally announced March 2024.

Comments: 27 pages, 10 figures

Online conformal prediction with decaying step sizes

Authors: Anastasios N. Angelopoulos, Rina Foygel Barber, Stephen Bates

Abstract: We introduce a method for online conformal prediction with decaying step sizes. Like previous methods, ours possesses a retrospective guarantee of coverage for arbitrary sequences. However, unlike previous methods, we can simultaneously estimate a population quantile when it exists. Our theory and experiments indicate substantially improved practical properties: in particular, when the distributio… ▽ More We introduce a method for online conformal prediction with decaying step sizes. Like previous methods, ours possesses a retrospective guarantee of coverage for arbitrary sequences. However, unlike previous methods, we can simultaneously estimate a population quantile when it exists. Our theory and experiments indicate substantially improved practical properties: in particular, when the distribution is stable, the coverage is close to the desired level for every time point, not just on average over the observed sequence. △ Less

Submitted 28 May, 2024; v1 submitted 1 February, 2024; originally announced February 2024.

Uncertainty Intervals for Prediction Errors in Time Series Forecasting

Authors: Hui Xu, Song Mei, Stephen Bates, Jonathan Taylor, Robert Tibshirani

Abstract: Inference for prediction errors is critical in time series forecasting pipelines. However, providing statistically meaningful uncertainty intervals for prediction errors remains relatively under-explored. Practitioners often resort to forward cross-validation (FCV) for obtaining point estimators and constructing confidence intervals based on the Central Limit Theorem (CLT). The naive version assum… ▽ More Inference for prediction errors is critical in time series forecasting pipelines. However, providing statistically meaningful uncertainty intervals for prediction errors remains relatively under-explored. Practitioners often resort to forward cross-validation (FCV) for obtaining point estimators and constructing confidence intervals based on the Central Limit Theorem (CLT). The naive version assumes independence, a condition that is usually invalid due to time correlation. These approaches lack statistical interpretations and theoretical justifications even under stationarity. This paper systematically investigates uncertainty intervals for prediction errors in time series forecasting. We first distinguish two key inferential targets: the stochastic test error over near future data points, and the expected test error as the expectation of the former. The stochastic test error is often more relevant in applications needing to quantify uncertainty over individual time series instances. To construct prediction intervals for the stochastic test error, we propose the quantile-based forward cross-validation (QFCV) method. Under an ergodicity assumption, QFCV intervals have asymptotically valid coverage and are shorter than marginal empirical quantiles. In addition, we also illustrate why naive CLT-based FCV intervals fail to provide valid uncertainty intervals, even with certain corrections. For non-stationary time series, we further provide rolling intervals by combining QFCV with adaptive conformal prediction to give time-average coverage guarantees. Overall, we advocate the use of QFCV procedures and demonstrate their coverage and efficiency through simulations and real data examples. △ Less

Submitted 14 September, 2023; originally announced September 2023.

Comments: 35 pages, 17 figures

Delegating Data Collection in Decentralized Machine Learning

Authors: Nivasini Ananthakrishnan, Stephen Bates, Michael I. Jordan, Nika Haghtalab

Abstract: Motivated by the emergence of decentralized machine learning (ML) ecosystems, we study the delegation of data collection. Taking the field of contract theory as our starting point, we design optimal and near-optimal contracts that deal with two fundamental information asymmetries that arise in decentralized ML: uncertainty in the assessment of model quality and uncertainty regarding the optimal pe… ▽ More Motivated by the emergence of decentralized machine learning (ML) ecosystems, we study the delegation of data collection. Taking the field of contract theory as our starting point, we design optimal and near-optimal contracts that deal with two fundamental information asymmetries that arise in decentralized ML: uncertainty in the assessment of model quality and uncertainty regarding the optimal performance of any model. We show that a principal can cope with such asymmetry via simple linear contracts that achieve 1-1/e fraction of the optimal utility. To address the lack of a priori knowledge regarding the optimal performance, we give a convex program that can adaptively and efficiently compute the optimal contract. We also study linear contracts and derive the optimal utility in the more complex setting of multiple interactions. △ Less

Submitted 2 May, 2024; v1 submitted 4 September, 2023; originally announced September 2023.

Incentive-Theoretic Bayesian Inference for Collaborative Science

Authors: Stephen Bates, Michael I. Jordan, Michael Sklar, Jake A. Soloff

Abstract: Contemporary scientific research is a distributed, collaborative endeavor, carried out by teams of researchers, regulatory institutions, funding agencies, commercial partners, and scientific bodies, all interacting with each other and facing different incentives. To maintain scientific rigor, statistical methods should acknowledge this state of affairs. To this end, we study hypothesis testing whe… ▽ More Contemporary scientific research is a distributed, collaborative endeavor, carried out by teams of researchers, regulatory institutions, funding agencies, commercial partners, and scientific bodies, all interacting with each other and facing different incentives. To maintain scientific rigor, statistical methods should acknowledge this state of affairs. To this end, we study hypothesis testing when there is an agent (e.g., a researcher or a pharmaceutical company) with a private prior about an unknown parameter and a principal (e.g., a policymaker or regulator) who wishes to make decisions based on the parameter value. The agent chooses whether to run a statistical trial based on their private prior and then the result of the trial is used by the principal to reach a decision. We show how the principal can conduct statistical inference that leverages the information that is revealed by an agent's strategic behavior -- their choice to run a trial or not. In particular, we show how the principal can design a policy to elucidate partial information about the agent's private prior beliefs and use this to control the posterior probability of the null. One implication is a simple guideline for the choice of significance threshold in clinical trials: the type-I error level should be set to be strictly less than the cost of the trial divided by the firm's profit if the trial is successful. △ Less

Submitted 8 February, 2024; v1 submitted 7 July, 2023; originally announced July 2023.

Class-Conditional Conformal Prediction with Many Classes

Authors: Tiffany Ding, Anastasios N. Angelopoulos, Stephen Bates, Michael I. Jordan, Ryan J. Tibshirani

Abstract: Standard conformal prediction methods provide a marginal coverage guarantee, which means that for a random test point, the conformal prediction set contains the true label with a user-specified probability. In many classification problems, we would like to obtain a stronger guarantee--that for test points of a specific class, the prediction set contains the true label with the same user-chosen pro… ▽ More Standard conformal prediction methods provide a marginal coverage guarantee, which means that for a random test point, the conformal prediction set contains the true label with a user-specified probability. In many classification problems, we would like to obtain a stronger guarantee--that for test points of a specific class, the prediction set contains the true label with the same user-chosen probability. For the latter goal, existing conformal prediction methods do not work well when there is a limited amount of labeled data per class, as is often the case in real applications where the number of classes is large. We propose a method called clustered conformal prediction that clusters together classes having "similar" conformal scores and performs conformal prediction at the cluster level. Based on empirical evaluation across four image data sets with many (up to 1000) classes, we find that clustered conformal typically outperforms existing methods in terms of class-conditional coverage and set size metrics. △ Less

Submitted 27 October, 2023; v1 submitted 15 June, 2023; originally announced June 2023.

Operationalizing Counterfactual Metrics: Incentives, Ranking, and Information Asymmetry

Authors: Serena Wang, Stephen Bates, P. M. Aronow, Michael I. Jordan

Abstract: From the social sciences to machine learning, it has been well documented that metrics to be optimized are not always aligned with social welfare. In healthcare, Dranove et al. (2003) showed that publishing surgery mortality metrics actually harmed the welfare of sicker patients by increasing provider selection behavior. We analyze the incentive misalignments that arise from such average treated o… ▽ More From the social sciences to machine learning, it has been well documented that metrics to be optimized are not always aligned with social welfare. In healthcare, Dranove et al. (2003) showed that publishing surgery mortality metrics actually harmed the welfare of sicker patients by increasing provider selection behavior. We analyze the incentive misalignments that arise from such average treated outcome metrics, and show that the incentives driving treatment decisions would align with maximizing total patient welfare if the metrics (i) accounted for counterfactual untreated outcomes and (ii) considered total welfare instead of averaging over treated patients. Operationalizing this, we show how counterfactual metrics can be modified to behave reasonably in patient-facing ranking systems. Extending to realistic settings when providers observe more about patients than the regulatory agencies do, we bound the decay in performance by the degree of information asymmetry between principal and agent. In doing so, our model connects principal-agent information asymmetry with unobserved heterogeneity in causal inference. △ Less

Submitted 29 November, 2023; v1 submitted 23 May, 2023; originally announced May 2023.

Web and Mobile Platforms for Managing Elections based on IoT And Machine Learning Algorithms

Authors: G. M. I. K. Galagoda, W. M. C. A. Karunarathne, R. S. Bates, K. M. H. V. P. Gangathilaka, Kanishka Yapa, Erandika Gamage

Abstract: The global pandemic situation has severely affected all countries. As a result, almost all countries had to adjust to online technologies to continue their processes. In addition, Sri Lanka is yearly spending ten billion on elections. We have examined a proper way of minimizing the cost of hosting these events online. To solve the existing problems and increase the time potency and cost reduction… ▽ More The global pandemic situation has severely affected all countries. As a result, almost all countries had to adjust to online technologies to continue their processes. In addition, Sri Lanka is yearly spending ten billion on elections. We have examined a proper way of minimizing the cost of hosting these events online. To solve the existing problems and increase the time potency and cost reduction we have used IoT and ML-based technologies. IoT-based data will identify, register, and be used to secure from fraud, while ML algorithms manipulate the election data and produce winning predictions, weather-based voters attendance, and election violence. All the data will be saved in cloud computing and a standard database to store and access the data. This study mainly focuses on four aspects of an E-voting system. The most frequent problems across the world in E-voting are the security, accuracy, and reliability of the systems. E-government systems must be secured against various cyber-attacks and ensure that only authorized users can access valuable, and sometimes sensitive information. Being able to access a system without passwords but using biometric details has been there for a while now, however, our proposed system has a different approach to taking the credentials, processing, and combining the images, reformatting and producing the output, and tracking. In addition, we ensure to enhance e-voting safety. While ML-based algorithms use different data sets and provide predictions in advance. △ Less

Submitted 15 March, 2023; originally announced March 2023.

Journal ref: International Journal of Engineering Applied Sciences and Technology, 2022, Vol 7, No 7, 29-35

Prediction-Powered Inference

Authors: Anastasios N. Angelopoulos, Stephen Bates, Clara Fannjiang, Michael I. Jordan, Tijana Zrnic

Abstract: Prediction-powered inference is a framework for performing valid statistical inference when an experimental dataset is supplemented with predictions from a machine-learning system. The framework yields simple algorithms for computing provably valid confidence intervals for quantities such as means, quantiles, and linear and logistic regression coefficients, without making any assumptions on the ma… ▽ More Prediction-powered inference is a framework for performing valid statistical inference when an experimental dataset is supplemented with predictions from a machine-learning system. The framework yields simple algorithms for computing provably valid confidence intervals for quantities such as means, quantiles, and linear and logistic regression coefficients, without making any assumptions on the machine-learning algorithm that supplies the predictions. Furthermore, more accurate predictions translate to smaller confidence intervals. Prediction-powered inference could enable researchers to draw valid and more data-efficient conclusions using machine learning. The benefits of prediction-powered inference are demonstrated with datasets from proteomics, astronomy, genomics, remote sensing, census analysis, and ecology. △ Less

Submitted 9 November, 2023; v1 submitted 23 January, 2023; originally announced January 2023.

Comments: Code is available at https://github.com/aangelopoulos/ppi_py

The Sample Complexity of Online Contract Design

Authors: Banghua Zhu, Stephen Bates, Zhuoran Yang, Yixin Wang, Jiantao Jiao, Michael I. Jordan

Abstract: We study the hidden-action principal-agent problem in an online setting. In each round, the principal posts a contract that specifies the payment to the agent based on each outcome. The agent then makes a strategic choice of action that maximizes her own utility, but the action is not directly observable by the principal. The principal observes the outcome and receives utility from the agent's cho… ▽ More We study the hidden-action principal-agent problem in an online setting. In each round, the principal posts a contract that specifies the payment to the agent based on each outcome. The agent then makes a strategic choice of action that maximizes her own utility, but the action is not directly observable by the principal. The principal observes the outcome and receives utility from the agent's choice of action. Based on past observations, the principal dynamically adjusts the contracts with the goal of maximizing her utility. We introduce an online learning algorithm and provide an upper bound on its Stackelberg regret. We show that when the contract space is $[0,1]^m$, the Stackelberg regret is upper bounded by $\widetilde O(\sqrt{m} \cdot T^{1-1/(2m+1)})$, and lower bounded by $Ω(T^{1-1/(m+2)})$, where $\widetilde O$ omits logarithmic factors. This result shows that exponential-in-$m$ samples are sufficient and necessary to learn a near-optimal contract, resolving an open problem on the hardness of online contract design. Moreover, when contracts are restricted to some subset $\mathcal{F} \subset [0,1]^m$, we define an intrinsic dimension of $\mathcal{F}$ that depends on the covering number of the spherical code in the space and bound the regret in terms of this intrinsic dimension. When $\mathcal{F}$ is the family of linear contracts, we show that the Stackelberg regret grows exactly as $Θ(T^{2/3})$. The contract design problem is challenging because the utility function is discontinuous. Bounding the discretization error in this setting has been an open problem. In this paper, we identify a limited set of directions in which the utility function is continuous, allowing us to design a new discretization method and bound its error. This approach enables the first upper bound with no restrictions on the contract and action space. △ Less

Submitted 19 May, 2023; v1 submitted 10 November, 2022; originally announced November 2022.

Conformal Prediction is Robust to Dispersive Label Noise

Authors: Shai Feldman, Bat-Sheva Einbinder, Stephen Bates, Anastasios N. Angelopoulos, Asaf Gendler, Yaniv Romano

Abstract: We study the robustness of conformal prediction, a powerful tool for uncertainty quantification, to label noise. Our analysis tackles both regression and classification problems, characterizing when and how it is possible to construct uncertainty sets that correctly cover the unobserved noiseless ground truth labels. We further extend our theory and formulate the requirements for correctly control… ▽ More We study the robustness of conformal prediction, a powerful tool for uncertainty quantification, to label noise. Our analysis tackles both regression and classification problems, characterizing when and how it is possible to construct uncertainty sets that correctly cover the unobserved noiseless ground truth labels. We further extend our theory and formulate the requirements for correctly controlling a general loss function, such as the false negative proportion, with noisy labels. Our theory and experiments suggest that conformal prediction and risk-controlling techniques with noisy labels attain conservative risk over the clean ground truth labels except in adversarial cases. In such cases, we can also correct for noise of bounded size in the conformal prediction algorithm in order to ensure achieving the correct risk of the ground truth labels without score or data regularity. △ Less

Submitted 19 September, 2023; v1 submitted 28 September, 2022; originally announced September 2022.

Conformal Risk Control

Authors: Anastasios N. Angelopoulos, Stephen Bates, Adam Fisch, Lihua Lei, Tal Schuster

Abstract: We extend conformal prediction to control the expected value of any monotone loss function. The algorithm generalizes split conformal prediction together with its coverage guarantee. Like conformal prediction, the conformal risk control procedure is tight up to an $\mathcal{O}(1/n)$ factor. We also introduce extensions of the idea to distribution shift, quantile risk control, multiple and adversar… ▽ More We extend conformal prediction to control the expected value of any monotone loss function. The algorithm generalizes split conformal prediction together with its coverage guarantee. Like conformal prediction, the conformal risk control procedure is tight up to an $\mathcal{O}(1/n)$ factor. We also introduce extensions of the idea to distribution shift, quantile risk control, multiple and adversarial risk control, and expectations of U-statistics. Worked examples from computer vision and natural language processing demonstrate the usage of our algorithm to bound the false negative rate, graph distance, and token-level F1-score. △ Less

Submitted 29 April, 2023; v1 submitted 4 August, 2022; originally announced August 2022.

Comments: Code available at https://github.com/aangelopoulos/conformal-risk

Semantic uncertainty intervals for disentangled latent spaces

Authors: Swami Sankaranarayanan, Anastasios N. Angelopoulos, Stephen Bates, Yaniv Romano, Phillip Isola

Abstract: Meaningful uncertainty quantification in computer vision requires reasoning about semantic information -- say, the hair color of the person in a photo or the location of a car on the street. To this end, recent breakthroughs in generative modeling allow us to represent semantic information in disentangled latent spaces, but providing uncertainties on the semantic latent variables has remained chal… ▽ More Meaningful uncertainty quantification in computer vision requires reasoning about semantic information -- say, the hair color of the person in a photo or the location of a car on the street. To this end, recent breakthroughs in generative modeling allow us to represent semantic information in disentangled latent spaces, but providing uncertainties on the semantic latent variables has remained challenging. In this work, we provide principled uncertainty intervals that are guaranteed to contain the true semantic factors for any underlying generative model. The method does the following: (1) it uses quantile regression to output a heuristic uncertainty interval for each element in the latent space (2) calibrates these uncertainties such that they contain the true value of the latent for a new, unseen input. The endpoints of these calibrated intervals can then be propagated through the generator to produce interpretable uncertainty visualizations for each semantic factor. This technique reliably communicates semantically meaningful, principled, and instance-adaptive uncertainty in inverse problems like image super-resolution and image completion. △ Less

Submitted 30 November, 2022; v1 submitted 20 July, 2022; originally announced July 2022.

Comments: Accepted to NeurIPS 2022. Project page: https://swamiviv.github.io/semantic_uncertainty_intervals/

Recommendation Systems with Distribution-Free Reliability Guarantees

Authors: Anastasios N. Angelopoulos, Karl Krauth, Stephen Bates, Yixin Wang, Michael I. Jordan

Abstract: When building recommendation systems, we seek to output a helpful set of items to the user. Under the hood, a ranking model predicts which of two candidate items is better, and we must distill these pairwise comparisons into the user-facing output. However, a learned ranking model is never perfect, so taking its predictions at face value gives no guarantee that the user-facing output is reliable.… ▽ More When building recommendation systems, we seek to output a helpful set of items to the user. Under the hood, a ranking model predicts which of two candidate items is better, and we must distill these pairwise comparisons into the user-facing output. However, a learned ranking model is never perfect, so taking its predictions at face value gives no guarantee that the user-facing output is reliable. Building from a pre-trained ranking model, we show how to return a set of items that is rigorously guaranteed to contain mostly good items. Our procedure endows any ranking model with rigorous finite-sample control of the false discovery rate (FDR), regardless of the (unknown) data distribution. Moreover, our calibration algorithm enables the easy and principled integration of multiple objectives in recommender systems. As an example, we show how to optimize for recommendation diversity subject to a user-specified level of FDR control, circumventing the need to specify ad hoc weights of a diversity loss against an accuracy loss. Throughout, we focus on the problem of learning to rank a set of possible recommendations, evaluating our methods on the Yahoo! Learning to Rank and MSMarco datasets. △ Less

Submitted 4 July, 2022; originally announced July 2022.

Robust Calibration with Multi-domain Temperature Scaling

Authors: Yaodong Yu, Stephen Bates, Yi Ma, Michael I. Jordan

Abstract: Uncertainty quantification is essential for the reliable deployment of machine learning models to high-stakes application domains. Uncertainty quantification is all the more challenging when training distribution and test distribution are different, even the distribution shifts are mild. Despite the ubiquity of distribution shifts in real-world applications, existing uncertainty quantification app… ▽ More Uncertainty quantification is essential for the reliable deployment of machine learning models to high-stakes application domains. Uncertainty quantification is all the more challenging when training distribution and test distribution are different, even the distribution shifts are mild. Despite the ubiquity of distribution shifts in real-world applications, existing uncertainty quantification approaches mainly study the in-distribution setting where the train and test distributions are the same. In this paper, we develop a systematic calibration model to handle distribution shifts by leveraging data from multiple domains. Our proposed method -- multi-domain temperature scaling -- uses the heterogeneity in the domains to improve calibration robustness under distribution shift. Through experiments on three benchmark data sets, we find our proposed method outperforms existing methods as measured on both in-distribution and out-of-distribution test sets. △ Less

Submitted 6 June, 2022; originally announced June 2022.

Achieving Risk Control in Online Learning Settings

Authors: Shai Feldman, Liran Ringel, Stephen Bates, Yaniv Romano

Abstract: To provide rigorous uncertainty quantification for online learning models, we develop a framework for constructing uncertainty sets that provably control risk -- such as coverage of confidence intervals, false negative rate, or F1 score -- in the online setting. This extends conformal prediction to apply to a larger class of online learning problems. Our method guarantees risk control at any user-… ▽ More To provide rigorous uncertainty quantification for online learning models, we develop a framework for constructing uncertainty sets that provably control risk -- such as coverage of confidence intervals, false negative rate, or F1 score -- in the online setting. This extends conformal prediction to apply to a larger class of online learning problems. Our method guarantees risk control at any user-specified level even when the underlying data distribution shifts drastically, even adversarially, over time in an unknown fashion. The technique we propose is highly flexible as it can be applied with any base online learning algorithm (e.g., a deep neural network trained online), requiring minimal implementation effort and essentially zero additional computational cost. We further extend our approach to control multiple risks simultaneously, so the prediction sets we generate are valid for all given risks. To demonstrate the utility of our method, we conduct experiments on real-world tabular time-series data sets showing that the proposed method rigorously controls various natural risks. Furthermore, we show how to construct valid intervals for an online image-depth estimation problem that previous sequential calibration schemes cannot handle. △ Less

Submitted 27 January, 2023; v1 submitted 18 May, 2022; originally announced May 2022.

Principal-Agent Hypothesis Testing

Authors: Stephen Bates, Michael I. Jordan, Michael Sklar, Jake A. Soloff

Abstract: Consider the relationship between a regulator (the principal) and an experimenter (the agent) such as a pharmaceutical company. The pharmaceutical company wishes to sell a drug for profit, whereas the regulator wishes to allow only efficacious drugs to be marketed. The efficacy of the drug is not known to the regulator, so the pharmaceutical company must run a costly trial to prove efficacy to the… ▽ More Consider the relationship between a regulator (the principal) and an experimenter (the agent) such as a pharmaceutical company. The pharmaceutical company wishes to sell a drug for profit, whereas the regulator wishes to allow only efficacious drugs to be marketed. The efficacy of the drug is not known to the regulator, so the pharmaceutical company must run a costly trial to prove efficacy to the regulator. Critically, the statistical protocol used to establish efficacy affects the behavior of a strategic, self-interested agent; a lower standard of statistical evidence incentivizes the agent to run more trials that are less likely to be effective. The interaction between the statistical protocol and the incentives of the pharmaceutical company is crucial for understanding this system and designing protocols with high social utility. In this work, we discuss how the regulator can set up a protocol with payoffs based on statistical evidence. We show how to design protocols that are robust to an agent's strategic actions, and derive the optimal protocol in the presence of strategic entrants. △ Less

Submitted 15 April, 2024; v1 submitted 13 May, 2022; originally announced May 2022.

Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging

Authors: Anastasios N Angelopoulos, Amit P Kohli, Stephen Bates, Michael I Jordan, Jitendra Malik, Thayer Alshaabi, Srigokul Upadhyayula, Yaniv Romano

Abstract: Image-to-image regression is an important learning task, used frequently in biological imaging. Current algorithms, however, do not generally offer statistical guarantees that protect against a model's mistakes and hallucinations. To address this, we develop uncertainty quantification techniques with rigorous statistical guarantees for image-to-image regression problems. In particular, we show how… ▽ More Image-to-image regression is an important learning task, used frequently in biological imaging. Current algorithms, however, do not generally offer statistical guarantees that protect against a model's mistakes and hallucinations. To address this, we develop uncertainty quantification techniques with rigorous statistical guarantees for image-to-image regression problems. In particular, we show how to derive uncertainty intervals around each pixel that are guaranteed to contain the true value with a user-specified confidence probability. Our methods work in conjunction with any base machine learning model, such as a neural network, and endow it with formal mathematical guarantees -- regardless of the true unknown data distribution or choice of model. Furthermore, they are simple to implement and computationally inexpensive. We evaluate our procedure on three image-to-image regression tasks: quantitative phase microscopy, accelerated magnetic resonance imaging, and super-resolution transmission electron microscopy of a Drosophila melanogaster brain. △ Less

Submitted 10 February, 2022; originally announced February 2022.

Comments: Code available at https://github.com/aangelopoulos/im2im-uq

Conformal prediction for the design problem

Authors: Clara Fannjiang, Stephen Bates, Anastasios N. Angelopoulos, Jennifer Listgarten, Michael I. Jordan

Abstract: Many applications of machine learning methods involve an iterative protocol in which data are collected, a model is trained, and then outputs of that model are used to choose what data to consider next. For example, one data-driven approach for designing proteins is to train a regression model to predict the fitness of protein sequences, then use it to propose new sequences believed to exhibit gre… ▽ More Many applications of machine learning methods involve an iterative protocol in which data are collected, a model is trained, and then outputs of that model are used to choose what data to consider next. For example, one data-driven approach for designing proteins is to train a regression model to predict the fitness of protein sequences, then use it to propose new sequences believed to exhibit greater fitness than observed in the training data. Since validating designed sequences in the wet lab is typically costly, it is important to quantify the uncertainty in the model's predictions. This is challenging because of a characteristic type of distribution shift between the training and test data in the design setting -- one in which the training and test data are statistically dependent, as the latter is chosen based on the former. Consequently, the model's error on the test data -- that is, the designed sequences -- has an unknown and possibly complex relationship with its error on the training data. We introduce a method to quantify predictive uncertainty in such settings. We do so by constructing confidence sets for predictions that account for the dependence between the training and test data. The confidence sets we construct have finite-sample guarantees that hold for any prediction algorithm, even when a trained model chooses the test-time input distribution. As a motivating use case, we demonstrate with several real data sets how our method quantifies uncertainty for the predicted fitness of designed proteins, and can therefore be used to select design algorithms that achieve acceptable trade-offs between high predicted fitness and low predictive uncertainty. △ Less

Submitted 31 May, 2022; v1 submitted 7 February, 2022; originally announced February 2022.

Comments: for associated code, see https://github.com/clarafy/conformal-for-design

Journal ref: Proc. Natl. Acad. Sci. 119 (43) e2204569119 (2022)

Nonlinear Regression with Residuals: Causal Estimation with Time-varying Treatments and Covariates

Authors: Stephen Bates, Edward Kennedy, Robert Tibshirani, Valerie Ventura, Larry Wasserman

Abstract: Standard regression adjustment gives inconsistent estimates of causal effects when there are time-varying treatment effects and time-varying covariates. Loosely speaking, the issue is that some covariates are post-treatment variables because they may be affected by prior treatment status, and regressing out post-treatment variables causes bias. More precisely, the bias is due to certain non-confou… ▽ More Standard regression adjustment gives inconsistent estimates of causal effects when there are time-varying treatment effects and time-varying covariates. Loosely speaking, the issue is that some covariates are post-treatment variables because they may be affected by prior treatment status, and regressing out post-treatment variables causes bias. More precisely, the bias is due to certain non-confounding latent variables that create colliders in the causal graph. These latent variables, which we call phantoms, do not harm the identifiability of the causal effect, but they render naive regression estimates inconsistent. Motivated by this, we ask: how can we modify regression methods so that they hold up even in the presence of phantoms? We develop an estimator for this setting based on regression modeling (linear, log-linear, probit and Cox regression), proving that it is consistent for a reasonable causal estimand. In particular, the estimator is a regression model fit with a simple adjustment for collinearity, making it easy to understand and implement with standard regression software. The proposed estimators are instances of the parametric g-formula, extending the regression-with-residuals approach to several canonical nonlinear models. △ Less

Submitted 10 March, 2024; v1 submitted 31 January, 2022; originally announced January 2022.

Confidence Intervals for the Generalisation Error of Random Forests

Authors: Samyak Rajanala, Stephen Bates, Trevor Hastie, Robert Tibshirani

Abstract: Out-of-bag error is commonly used as an estimate of generalisation error in ensemble-based learning models such as random forests. We present confidence intervals for this quantity using the delta-method-after-bootstrap and the jackknife-after-bootstrap techniques. These methods do not require growing any additional trees. We show that these new confidence intervals have improved coverage properti… ▽ More Out-of-bag error is commonly used as an estimate of generalisation error in ensemble-based learning models such as random forests. We present confidence intervals for this quantity using the delta-method-after-bootstrap and the jackknife-after-bootstrap techniques. These methods do not require growing any additional trees. We show that these new confidence intervals have improved coverage properties over the naive confidence interval, in real and simulated examples. △ Less

Submitted 26 January, 2022; originally announced January 2022.

Comments: 25 pages, 8 tables, 8 figures

Optimal Data Selection: An Online Distributed View

Authors: Mariel Werner, Anastasios Angelopoulos, Stephen Bates, Michael I. Jordan

Abstract: The blessing of ubiquitous data also comes with a curse: the communication, storage, and labeling of massive, mostly redundant datasets. We seek to solve this problem at its core, collecting only valuable data and throwing out the rest via submodular maximization. Specifically, we develop algorithms for the online and distributed version of the problem, where data selection occurs in an uncoordina… ▽ More The blessing of ubiquitous data also comes with a curse: the communication, storage, and labeling of massive, mostly redundant datasets. We seek to solve this problem at its core, collecting only valuable data and throwing out the rest via submodular maximization. Specifically, we develop algorithms for the online and distributed version of the problem, where data selection occurs in an uncoordinated fashion across multiple data streams. We design a general and flexible core selection routine for our algorithms which, given any stream of data, any assessment of its value, and any formulation of its selection cost, extracts the most valuable subset of the stream up to a constant factor while using minimal memory. Notably, our methods have the same theoretical guarantees as their offline counterparts, and, as far as we know, provide the first guarantees for online distributed submodular optimization in the literature. Finally, in learning tasks on ImageNet and MNIST, we show that our selection methods outperform random selection by $5-20\%$. △ Less

Submitted 14 December, 2023; v1 submitted 25 January, 2022; originally announced January 2022.

Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control

Authors: Anastasios N. Angelopoulos, Stephen Bates, Emmanuel J. Candès, Michael I. Jordan, Lihua Lei

Abstract: We introduce a framework for calibrating machine learning models so that their predictions satisfy explicit, finite-sample statistical guarantees. Our calibration algorithms work with any underlying model and (unknown) data-generating distribution and do not require model refitting. The framework addresses, among other examples, false discovery rate control in multi-label classification, intersect… ▽ More We introduce a framework for calibrating machine learning models so that their predictions satisfy explicit, finite-sample statistical guarantees. Our calibration algorithms work with any underlying model and (unknown) data-generating distribution and do not require model refitting. The framework addresses, among other examples, false discovery rate control in multi-label classification, intersection-over-union control in instance segmentation, and the simultaneous control of the type-1 error of outlier detection and confidence set coverage in classification or regression. Our main insight is to reframe the risk-control problem as multiple hypothesis testing, enabling techniques and mathematical arguments different from those in the previous literature. We use the framework to provide new calibration methods for several core machine learning tasks, with detailed worked examples in computer vision and tabular medical data. △ Less

Submitted 29 September, 2022; v1 submitted 3 October, 2021; originally announced October 2021.

Comments: Code available at https://github.com/aangelopoulos/ltt

Calibrated Multiple-Output Quantile Regression with Representation Learning

Authors: Shai Feldman, Stephen Bates, Yaniv Romano

Abstract: We develop a method to generate predictive regions that cover a multivariate response variable with a user-specified probability. Our work is composed of two components. First, we use a deep generative model to learn a representation of the response that has a unimodal distribution. Existing multiple-output quantile regression approaches are effective in such cases, so we apply them on the learned… ▽ More We develop a method to generate predictive regions that cover a multivariate response variable with a user-specified probability. Our work is composed of two components. First, we use a deep generative model to learn a representation of the response that has a unimodal distribution. Existing multiple-output quantile regression approaches are effective in such cases, so we apply them on the learned representation, and then transform the solution to the original space of the response. This process results in a flexible and informative region that can have an arbitrary shape, a property that existing methods lack. Second, we propose an extension of conformal prediction to the multivariate response setting that modifies any method to return sets with a pre-specified coverage level. The desired coverage is theoretically guaranteed in the finite-sample case for any distribution. Experiments conducted on both real and synthetic data show that our method constructs regions that are significantly smaller compared to existing techniques. △ Less

Submitted 23 December, 2022; v1 submitted 2 October, 2021; originally announced October 2021.

Discriminative Attribution from Counterfactuals

Authors: Nils Eckstein, Alexander S. Bates, Gregory S. X. E. Jefferis, Jan Funke

Abstract: We present a method for neural network interpretability by combining feature attribution with counterfactual explanations to generate attribution maps that highlight the most discriminative features between pairs of classes. We show that this method can be used to quantitatively evaluate the performance of feature attribution methods in an objective manner, thus preventing potential observer bias.… ▽ More We present a method for neural network interpretability by combining feature attribution with counterfactual explanations to generate attribution maps that highlight the most discriminative features between pairs of classes. We show that this method can be used to quantitatively evaluate the performance of feature attribution methods in an objective manner, thus preventing potential observer bias. We evaluate the proposed method on three diverse datasets, including a challenging artificial dataset and real-world biological data. We show quantitatively and qualitatively that the highlighted features are substantially more discriminative than those extracted using conventional attribution methods and argue that this type of explanation is better suited for understanding fine grained class differences as learned by a deep neural network. △ Less

Submitted 27 September, 2021; originally announced September 2021.

A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification

Authors: Anastasios N. Angelopoulos, Stephen Bates

Abstract: Black-box machine learning models are now routinely used in high-risk settings, like medical diagnostics, which demand uncertainty quantification to avoid consequential model failures. Conformal prediction is a user-friendly paradigm for creating statistically rigorous uncertainty sets/intervals for the predictions of such models. Critically, the sets are valid in a distribution-free sense: they p… ▽ More Black-box machine learning models are now routinely used in high-risk settings, like medical diagnostics, which demand uncertainty quantification to avoid consequential model failures. Conformal prediction is a user-friendly paradigm for creating statistically rigorous uncertainty sets/intervals for the predictions of such models. Critically, the sets are valid in a distribution-free sense: they possess explicit, non-asymptotic guarantees even without distributional assumptions or model assumptions. One can use conformal prediction with any pre-trained model, such as a neural network, to produce sets that are guaranteed to contain the ground truth with a user-specified probability, such as 90%. It is easy-to-understand, easy-to-use, and general, applying naturally to problems arising in the fields of computer vision, natural language processing, deep reinforcement learning, and so on. This hands-on introduction is aimed to provide the reader a working understanding of conformal prediction and related distribution-free uncertainty quantification techniques with one self-contained document. We lead the reader through practical theory for and examples of conformal prediction and describe its extensions to complex machine learning tasks involving structured outputs, distribution shift, time-series, outliers, models that abstain, and more. Throughout, there are many explanatory illustrations, examples, and code samples in Python. With each code sample comes a Jupyter notebook implementing the method on a real-data example; the notebooks can be accessed and easily run using our codebase. △ Less

Submitted 7 December, 2022; v1 submitted 15 July, 2021; originally announced July 2021.

Comments: Blog and tutorial video at http://angelopoulos.ai/blog/posts/gentle-intro/ ; Code is available at https://github.com/aangelopoulos/conformal-prediction

Test-time Collective Prediction

Authors: Celestine Mendler-Dünner, Wenshuo Guo, Stephen Bates, Michael I. Jordan

Abstract: An increasingly common setting in machine learning involves multiple parties, each with their own data, who want to jointly make predictions on future test points. Agents wish to benefit from the collective expertise of the full set of agents to make better predictions than they would individually, but may not be willing to release their data or model parameters. In this work, we explore a decentr… ▽ More An increasingly common setting in machine learning involves multiple parties, each with their own data, who want to jointly make predictions on future test points. Agents wish to benefit from the collective expertise of the full set of agents to make better predictions than they would individually, but may not be willing to release their data or model parameters. In this work, we explore a decentralized mechanism to make collective predictions at test time, leveraging each agent's pre-trained model without relying on external validation, model retraining, or data pooling. Our approach takes inspiration from the literature in social science on human consensus-making. We analyze our mechanism theoretically, showing that it converges to inverse meansquared-error (MSE) weighting in the large-sample limit. To compute error bars on the collective predictions we propose a decentralized Jackknife procedure that evaluates the sensitivity of our mechanism to a single agent's prediction. Empirically, we demonstrate that our scheme effectively combines models with differing quality across the input space. The proposed consensus prediction achieves significant gains over classical model averaging, and even outperforms weighted averaging schemes that have access to additional validation data. △ Less

Submitted 22 June, 2021; originally announced June 2021.

Improving Conditional Coverage via Orthogonal Quantile Regression

Authors: Shai Feldman, Stephen Bates, Yaniv Romano

Abstract: We develop a method to generate prediction intervals that have a user-specified coverage level across all regions of feature-space, a property called conditional coverage. A typical approach to this task is to estimate the conditional quantiles with quantile regression -- it is well-known that this leads to correct coverage in the large-sample limit, although it may not be accurate in finite sampl… ▽ More We develop a method to generate prediction intervals that have a user-specified coverage level across all regions of feature-space, a property called conditional coverage. A typical approach to this task is to estimate the conditional quantiles with quantile regression -- it is well-known that this leads to correct coverage in the large-sample limit, although it may not be accurate in finite samples. We find in experiments that traditional quantile regression can have poor conditional coverage. To remedy this, we modify the loss function to promote independence between the size of the intervals and the indicator of a miscoverage event. For the true conditional quantiles, these two quantities are independent (orthogonal), so the modified loss function continues to be valid. Moreover, we empirically show that the modified loss function leads to improved conditional coverage, as evaluated by several metrics. We also introduce two new metrics that check conditional coverage by looking at the strength of the dependence between the interval size and the indicator of miscoverage. △ Less

Submitted 2 October, 2021; v1 submitted 1 June, 2021; originally announced June 2021.

Comments: 20 pages, 5 figures

Testing for Outliers with Conformal p-values

Authors: Stephen Bates, Emmanuel Candès, Lihua Lei, Yaniv Romano, Matteo Sesia

Abstract: This paper studies the construction of p-values for nonparametric outlier detection, taking a multiple-testing perspective. The goal is to test whether new independent samples belong to the same distribution as a reference data set or are outliers. We propose a solution based on conformal inference, a broadly applicable framework which yields p-values that are marginally valid but mutually depende… ▽ More This paper studies the construction of p-values for nonparametric outlier detection, taking a multiple-testing perspective. The goal is to test whether new independent samples belong to the same distribution as a reference data set or are outliers. We propose a solution based on conformal inference, a broadly applicable framework which yields p-values that are marginally valid but mutually dependent for different test points. We prove these p-values are positively dependent and enable exact false discovery rate control, although in a relatively weak marginal sense. We then introduce a new method to compute p-values that are both valid conditionally on the training data and independent of each other for different test points; this paves the way to stronger type-I error guarantees. Our results depart from classical conformal inference as we leverage concentration inequalities rather than combinatorial arguments to establish our finite-sample guarantees. Furthermore, our techniques also yield a uniform confidence bound for the false positive rate of any outlier detection algorithm, as a function of the threshold applied to its raw statistics. Finally, the relevance of our results is demonstrated by numerical experiments on real and simulated data. △ Less

Submitted 24 May, 2022; v1 submitted 16 April, 2021; originally announced April 2021.

Comments: Revision May 24, 2022: added "asymptotic" and "Monte Carlo" conditional calibration methods; added power analyses; updated numerical experiments to include new methods

Journal ref: Ann. Statist. 51(1): 149-178 (February 2023)

Cross-validation: what does it estimate and how well does it do it?

Authors: Stephen Bates, Trevor Hastie, Robert Tibshirani

Abstract: Cross-validation is a widely-used technique to estimate prediction error, but its behavior is complex and not fully understood. Ideally, one would like to think that cross-validation estimates the prediction error for the model at hand, fit to the training data. We prove that this is not the case for the linear model fit by ordinary least squares; rather it estimates the average prediction error o… ▽ More Cross-validation is a widely-used technique to estimate prediction error, but its behavior is complex and not fully understood. Ideally, one would like to think that cross-validation estimates the prediction error for the model at hand, fit to the training data. We prove that this is not the case for the linear model fit by ordinary least squares; rather it estimates the average prediction error of models fit on other unseen training sets drawn from the same population. We further show that this phenomenon occurs for most popular estimates of prediction error, including data splitting, bootstrap**, and Mallow's Cp. Next, the standard confidence intervals for prediction error derived from cross-validation may have coverage far below the desired level. Because each data point is used for both training and testing, there are correlations among the measured accuracies for each fold, and so the usual estimate of variance is too small. We introduce a nested cross-validation scheme to estimate this variance more accurately, and we show empirically that this modification leads to intervals with approximately correct coverage in many examples where traditional cross-validation intervals fail. △ Less

Submitted 18 July, 2022; v1 submitted 1 April, 2021; originally announced April 2021.

Private Prediction Sets

Authors: Anastasios N. Angelopoulos, Stephen Bates, Tijana Zrnic, Michael I. Jordan

Abstract: In real-world settings involving consequential decision-making, the deployment of machine learning systems generally requires both reliable uncertainty quantification and protection of individuals' privacy. We present a framework that treats these two desiderata jointly. Our framework is based on conformal prediction, a methodology that augments predictive models to return prediction sets that pro… ▽ More In real-world settings involving consequential decision-making, the deployment of machine learning systems generally requires both reliable uncertainty quantification and protection of individuals' privacy. We present a framework that treats these two desiderata jointly. Our framework is based on conformal prediction, a methodology that augments predictive models to return prediction sets that provide uncertainty quantification -- they provably cover the true response with a user-specified probability, such as 90%. One might hope that when used with privately-trained models, conformal prediction would yield privacy guarantees for the resulting prediction sets; unfortunately, this is not the case. To remedy this key problem, we develop a method that takes any pre-trained predictive model and outputs differentially private prediction sets. Our method follows the general approach of split conformal prediction; we use holdout data to calibrate the size of the prediction sets but preserve privacy by using a privatized quantile subroutine. This subroutine compensates for the noise introduced to preserve privacy in order to guarantee correct coverage. We evaluate the method on large-scale computer vision datasets. △ Less

Submitted 3 March, 2024; v1 submitted 11 February, 2021; originally announced February 2021.

Comments: Code available at https://github.com/aangelopoulos/private_prediction_sets

Journal ref: Harvard Data Science Review, 4(2). 2022

Distribution-Free, Risk-Controlling Prediction Sets

Authors: Stephen Bates, Anastasios Angelopoulos, Lihua Lei, Jitendra Malik, Michael I. Jordan

Abstract: While improving prediction accuracy has been the focus of machine learning in recent years, this alone does not suffice for reliable decision-making. Deploying learning systems in consequential settings also requires calibrating and communicating the uncertainty of predictions. To convey instance-wise uncertainty for prediction tasks, we show how to generate set-valued predictions from a black-box… ▽ More While improving prediction accuracy has been the focus of machine learning in recent years, this alone does not suffice for reliable decision-making. Deploying learning systems in consequential settings also requires calibrating and communicating the uncertainty of predictions. To convey instance-wise uncertainty for prediction tasks, we show how to generate set-valued predictions from a black-box predictor that control the expected loss on future test points at a user-specified level. Our approach provides explicit finite-sample guarantees for any dataset by using a holdout set to calibrate the size of the prediction sets. This framework enables simple, distribution-free, rigorous error control for many tasks, and we demonstrate it in five large-scale machine learning problems: (1) classification problems where some mistakes are more costly than others; (2) multi-label classification, where each observation has multiple associated labels; (3) classification problems where the labels have a hierarchical structure; (4) image segmentation, where we wish to predict a set of pixels containing an object of interest; and (5) protein structure prediction. Lastly, we discuss extensions to uncertainty quantification for ranking, metric learning and distributionally robust learning. △ Less

Submitted 4 August, 2021; v1 submitted 7 January, 2021; originally announced January 2021.

Comments: Project website available at http://www.angelopoulos.ai/blog/posts/rcps/ and codebase available at https://github.com/aangelopoulos/rcps

Uncertainty Sets for Image Classifiers using Conformal Prediction

Authors: Anastasios Angelopoulos, Stephen Bates, Jitendra Malik, Michael I. Jordan

Abstract: Convolutional image classifiers can achieve high predictive accuracy, but quantifying their uncertainty remains an unresolved challenge, hindering their deployment in consequential settings. Existing uncertainty quantification techniques, such as Platt scaling, attempt to calibrate the network's probability estimates, but they do not have formal guarantees. We present an algorithm that modifies an… ▽ More Convolutional image classifiers can achieve high predictive accuracy, but quantifying their uncertainty remains an unresolved challenge, hindering their deployment in consequential settings. Existing uncertainty quantification techniques, such as Platt scaling, attempt to calibrate the network's probability estimates, but they do not have formal guarantees. We present an algorithm that modifies any classifier to output a predictive set containing the true label with a user-specified probability, such as 90%. The algorithm is simple and fast like Platt scaling, but provides a formal finite-sample coverage guarantee for every model and dataset. Our method modifies an existing conformal prediction algorithm to give more stable predictive sets by regularizing the small scores of unlikely classes after Platt scaling. In experiments on both Imagenet and Imagenet-V2 with ResNet-152 and other classifiers, our scheme outperforms existing approaches, achieving coverage with sets that are often factors of 5 to 10 smaller than a stand-alone Platt scaling baseline. △ Less

Submitted 3 September, 2022; v1 submitted 29 September, 2020; originally announced September 2020.

Comments: ICLR 2021 Spotlight, https://openreview.net/forum?id=eNdiU_DbM9 . Project website at https://people.eecs.berkeley.edu/~angelopoulos/blog/posts/conformal-classification/ . Codebase at https://github.com/aangelopoulos/conformal_classification

Achieving Equalized Odds by Resampling Sensitive Attributes

Authors: Yaniv Romano, Stephen Bates, Emmanuel J. Candès

Abstract: We present a flexible framework for learning predictive models that approximately satisfy the equalized odds notion of fairness. This is achieved by introducing a general discrepancy functional that rigorously quantifies violations of this criterion. This differentiable functional is used as a penalty driving the model parameters towards equalized odds. To rigorously evaluate fitted models, we dev… ▽ More We present a flexible framework for learning predictive models that approximately satisfy the equalized odds notion of fairness. This is achieved by introducing a general discrepancy functional that rigorously quantifies violations of this criterion. This differentiable functional is used as a penalty driving the model parameters towards equalized odds. To rigorously evaluate fitted models, we develop a formal hypothesis test to detect whether a prediction rule violates this property, the first such test in the literature. Both the model fitting and hypothesis testing leverage a resampled version of the sensitive attribute obeying equalized odds, by construction. We demonstrate the applicability and validity of the proposed framework both in regression and multi-class classification problems, reporting improved performance over state-of-the-art methods. Lastly, we show how to incorporate techniques for equitable uncertainty quantification---unbiased for each group under study---to communicate the results of the data analysis in exact terms. △ Less

Submitted 7 June, 2020; originally announced June 2020.

Comments: 14 pages, 4 figures

Causal Inference in Genetic Trio Studies

Authors: Stephen Bates, Matteo Sesia, Chiara Sabatti, Emmanuel Candes

Abstract: We introduce a method to rigorously draw causal inferences---inferences immune to all possible confounding---from genetic data that include parents and offspring. Causal conclusions are possible with these data because the natural randomness in meiosis can be viewed as a high-dimensional randomized experiment. We make this observation actionable by develo** a novel conditional independence test… ▽ More We introduce a method to rigorously draw causal inferences---inferences immune to all possible confounding---from genetic data that include parents and offspring. Causal conclusions are possible with these data because the natural randomness in meiosis can be viewed as a high-dimensional randomized experiment. We make this observation actionable by develo** a novel conditional independence test that identifies regions of the genome containing distinct causal variants. The proposed Digital Twin Test compares an observed offspring to carefully constructed synthetic offspring from the same parents in order to determine statistical significance, and it can leverage any black-box multivariate model and additional non-trio genetic data in order to increase power. Crucially, our inferences are based only on a well-established mathematical description of the rearrangement of genetic material during meiosis and make no assumptions about the relationship between the genotypes and phenotypes. △ Less

Submitted 22 February, 2020; originally announced February 2020.

Journal ref: Proc. Natl. Acad. Sci. U.S.A. 177 (2020) 24117-24126

The High Time Resolution Universe Pulsar Survey -- XVI. Discovery and timing of 40 pulsars from the southern Galactic plane

Authors: A. D. Cameron, D. J. Champion, M. Bailes, V. Balakrishnan, E. D. Barr, C. G. Bassa, S. Bates, S. Bhandari, N. D. R. Bhat, M. Burgay, S. Burke-Spolaor, C. M. L. Flynn, A. Jameson, S. Johnston, M. J. Keith, M. Kramer, L. Levin, A. G. Lyne, C. Ng, E. Petroff, A. Possenti, D. A. Smith, B. W. Stappers, W. van Straten, C. Tiburzi , et al. (1 additional authors not shown)

Abstract: We present the results of processing an additional 44% of the High Time Resolution Universe South Low Latitude (HTRU-S LowLat) pulsar survey, the most sensitive blind pulsar survey of the southern Galactic plane to date. Our partially-coherent segmented acceleration search pipeline is designed to enable the discovery of pulsars in short, highly-accelerated orbits, while our 72-min integration leng… ▽ More We present the results of processing an additional 44% of the High Time Resolution Universe South Low Latitude (HTRU-S LowLat) pulsar survey, the most sensitive blind pulsar survey of the southern Galactic plane to date. Our partially-coherent segmented acceleration search pipeline is designed to enable the discovery of pulsars in short, highly-accelerated orbits, while our 72-min integration lengths will allow us to discover pulsars at the lower end of the pulsar luminosity distribution. We report the discovery of 40 pulsars, including three millisecond pulsar-white dwarf binary systems (PSRs J1537-5312, J1547-5709 and J1618-4624), a black-widow binary system (PSR J1745-23) and a candidate black-widow binary system (PSR J1727-2951), a glitching pulsar (PSR J1706-4434), an eclipsing binary pulsar with a 1.5-yr orbital period (PSR J1653-45), and a pair of long spin-period binary pulsars which display either nulling or intermittent behaviour (PSRs J1812-15 and J1831-04). We show that the total population of 100 pulsars discovered in the HTRU-S LowLat survey to date represents both an older and lower-luminosity population, and indicates that we have yet to reach the bottom of the luminosity distribution function. We present evaluations of the performance of our search technique and of the overall yield of the survey, considering the 94% of the survey which we have processed to date. We show that our pulsar yield falls below earlier predictions by approximately 25% (especially in the case of millisecond pulsars), and discuss explanations for this discrepancy as well as future adaptations in RFI mitigation and searching techniques which may address these shortfalls. △ Less

Submitted 6 January, 2020; originally announced January 2020.

Comments: 28 pages, 9 figures, 13 tables

Uncooled Microbolometer Arrays for Ground Based Astronomy

Authors: Maisie F. Rashman, Iain A. Steele, Stuart D. Bates, Dave Copley, Steven N. Longmore

Abstract: We describe the design and commissioning of a simple prototype, low-cost 10$μ$m imaging instrument. The system is built using commercially available components including an uncooled microbolometer array as a detector. The incorporation of adjustable germanium reimaging optics rescale the image to the appropriate plate scale for the 2-m diameter Liverpool Telescope. From observations of bright sola… ▽ More We describe the design and commissioning of a simple prototype, low-cost 10$μ$m imaging instrument. The system is built using commercially available components including an uncooled microbolometer array as a detector. The incorporation of adjustable germanium reimaging optics rescale the image to the appropriate plate scale for the 2-m diameter Liverpool Telescope. From observations of bright solar system and stellar sources, we demonstrate a plate scale of 0.75$^{\prime\prime}$ per pixel and confirm the optical design allows diffraction limited imaging. We record a $\sim$ 10$\%$ photometric stability due to sky variability. We measure a $3 σ$ sensitivity of $7 \times 10^{3}$ Jy for a single, $\sim$ 0.11 second exposure. This corresponds to a sensitivity limit of $3 \times 10^{2}$ Jy for a 60 second total integration. We present an example science case from observations of the 2019 Jan total lunar eclipse and show that the system can detect and measure the anomalous cooling rate associated with the features Bellot and Langrenus during eclipse. △ Less

Submitted 17 December, 2019; originally announced December 2019.

Comments: Accepted for publication by MNRAS

Metropolized Knockoff Sampling

Authors: Stephen Bates, Emmanuel Candès, Lucas Janson, Wenshuo Wang

Abstract: Model-X knockoffs is a wrapper that transforms essentially any feature importance measure into a variable selection algorithm, which discovers true effects while rigorously controlling the expected fraction of false positives. A frequently discussed challenge to apply this method is to construct knockoff variables, which are synthetic variables obeying a crucial exchangeability property with the e… ▽ More Model-X knockoffs is a wrapper that transforms essentially any feature importance measure into a variable selection algorithm, which discovers true effects while rigorously controlling the expected fraction of false positives. A frequently discussed challenge to apply this method is to construct knockoff variables, which are synthetic variables obeying a crucial exchangeability property with the explanatory variables under study. This paper introduces techniques for knockoff generation in great generality: we provide a sequential characterization of all possible knockoff distributions, which leads to a Metropolis-Hastingsformulation of an exact knockoff sampler. We further show how to use conditional independence structure to speed up computations. Combining these two threads, we introduce an explicit set of sequential algorithms and empirically demonstrate their effectiveness. Our theoretical analysis proves that our algorithms achieve near-optimal computational complexity in certain cases. The techniques we develop are sufficiently rich to enable knockoff sampling in challenging models including cases where the covariates are continuous and heavy-tailed, and follow a graphical model such as the Ising model. △ Less

Submitted 1 March, 2019; originally announced March 2019.

Journal ref: Journal of the American Statistical Association, 116:535, 1413-1427, 2021

The High Time Resolution Universe Pulsar Survey -- XV: completion of the intermediate latitude survey with the discovery and timing of 25 further pulsars

Authors: M. Burgay, B. Stappers, M. Bailes, E. D. Barr, S. Bates, N. D. R. Bhat, S. Burke-Spolaor, A. D. Cameron, D. J. Champion, R. P. Eatough, C. M. L. Flynn, A. Jameson, S. Johnston, M. J. Keith, E. F. Keane, M. Kramer, L. Levin, C. Ng, E. Petroff, A. Possenti, W. van Straten, C. Tiburzi, L. Bondonneau, A. G. Lyne

Abstract: We report on the latest six pulsars discovered through our standard pipeline in the intermediate-latitude region (|b| < 15 deg) of the Parkes High Time Resolution Universe Survey (HTRU). We also present timing solutions for the new discoveries and for 19 further pulsars for which only discovery parameters were previously published. Highlights of the presented sample include the isolated millisecon… ▽ More We report on the latest six pulsars discovered through our standard pipeline in the intermediate-latitude region (|b| < 15 deg) of the Parkes High Time Resolution Universe Survey (HTRU). We also present timing solutions for the new discoveries and for 19 further pulsars for which only discovery parameters were previously published. Highlights of the presented sample include the isolated millisecond pulsar J1826-2415, the long-period binary pulsar J1837-0822 in a mildly eccentric 98-day orbit with a > 0.27 M_sun companion, and the nulling pulsar J1638-4233, detected only 10% of the time. Other interesting objects are PSR J1757-1500, exhibiting sporadic mode changes, and PSR J1635-2616 showing one glitch over 6 years. The new discoveries bring the total count of HTRU intermediate-latitude pulsars to 113, 25% of which are recycled pulsars. This is the higest ratio of recycled over ordinary pulsars discoveries of all recent pulsar surveys in this region of the sky. Among HTRU recycled pulsars, four are isolated objects. Comparing the characteristics of Galactic fully-recycled isolated MSPs with those of eclipsing binaries ('spiders'), from which the former are believed to have formed, we highlight a discrepancy in their spatial distribution. This may reflect a difference in the natal kick, hence, possibly, a different formation path. On the other hand, however, isolated fully-recycled MSPs spin periods are, on average, longer than those of spiders, in line with what one would expect, from simple magnetic-dipole spin-down, if the former were indeed evolved from the latter. △ Less

Submitted 14 February, 2019; originally announced February 2019.

Comments: Accepted for publication in MNRAS; 12 pages, 9 figures, 7 tables

The High Time Resolution Universe survey XIV: Discovery of 23 pulsars through GPU-accelerated reprocessing

Authors: V. Morello, E. D. Barr, S. Cooper, M. Bailes, S. Bates, N. D. R. Bhat, M. Burgay, S. Burke-Spolaor, A. D. Cameron, D. J. Champion, R. P. Eatough, C. M. L. Flynn, A. Jameson, S. Johnston, M. J. Keith, E. F. Keane, M. Kramer, L. Levin, C. Ng, E. Petroff, A. Possenti, B. W. Stappers, W. van Straten, C. Tiburzi

Abstract: We have performed a new search for radio pulsars in archival data of the intermediate and high Galactic latitude parts of the Southern High Time Resolution Universe pulsar survey. This is the first time the entire dataset has been searched for binary pulsars, an achievement enabled by GPU-accelerated dedispersion and periodicity search codes nearly 50 times faster than the previously used pipeline… ▽ More We have performed a new search for radio pulsars in archival data of the intermediate and high Galactic latitude parts of the Southern High Time Resolution Universe pulsar survey. This is the first time the entire dataset has been searched for binary pulsars, an achievement enabled by GPU-accelerated dedispersion and periodicity search codes nearly 50 times faster than the previously used pipeline. Candidate selection was handled entirely by a Machine Learning algorithm, allowing for the assessment of 17.6 million candidates in a few person-days. We have also introduced an outlier detection algorithm for efficient radio-frequency interference (RFI) mitigation on folded data, a new approach that enabled the discovery of pulsars previously masked by RFI. We discuss implications for future searches, particularly the importance of expanding work on RFI mitigation to improve survey completeness. In total we discovered 23 previously unknown sources, including 6 millisecond pulsars and at least 4 pulsars in binary systems. We also found an elusive but credible redback candidate that we have yet to confirm. △ Less

Submitted 12 November, 2018; originally announced November 2018.

Comments: Accepted for publication in MNRAS, 14 pages, 5 figures, 10 tables

A fast radio burst with a low dispersion measure

Authors: E. Petroff, L. C. Oostrum, B. W. Stappers, M. Bailes, E. D. Barr, S. Bates, S. Bhandari, N. D. R. Bhat, M. Burgay, S. Burke-Spolaor, A. D. Cameron, D. J. Champion, R. P. Eatough, C. M. L. Flynn, A. Jameson, S. Johnston, E. F. Keane, M. J. Keith, L. Levin, V. Morello, C. Ng, A. Possenti, V. Ravi, W. van Straten, D. Thornton , et al. (1 additional authors not shown)

Abstract: Fast radio bursts (FRBs) are millisecond pulses of radio emission of seemingly extragalactic origin. More than 50 FRBs have now been detected, with only one seen to repeat. Here we present a new FRB discovery, FRB 110214, which was detected in the high latitude portion of the High Time Resolution Universe South survey at the Parkes telescope. FRB 110214 has one of the lowest dispersion measures of… ▽ More Fast radio bursts (FRBs) are millisecond pulses of radio emission of seemingly extragalactic origin. More than 50 FRBs have now been detected, with only one seen to repeat. Here we present a new FRB discovery, FRB 110214, which was detected in the high latitude portion of the High Time Resolution Universe South survey at the Parkes telescope. FRB 110214 has one of the lowest dispersion measures of any known FRB (DM = 168.9$\pm$0.5 pc cm$^{-3}$), and was detected in two beams of the Parkes multi-beam receiver. A triangulation of the burst origin on the sky identified three possible regions in the beam pattern where it may have originated, all in sidelobes of the primary detection beam. Depending on the true location of the burst the intrinsic fluence is estimated to fall in the range of 50 -- 2000 Jy ms, making FRB 110214 one of the highest-fluence FRBs detected with the Parkes telescope. No repeating pulses were seen in almost 100 hours of follow-up observations with the Parkes telescope down to a limiting fluence of 0.3 Jy ms for a 2-ms pulse. Similar low-DM, ultra-bright FRBs may be detected in telescope sidelobes in the future, making careful modeling of multi-beam instrument beam patterns of utmost importance for upcoming FRB surveys. △ Less

Submitted 25 October, 2018; originally announced October 2018.

Comments: 8 pages, 3 figures, accepted for publication in MNRAS

Stable Frank-Kasper phases of self-assembled, soft matter spheres

Authors: Abhiram Reddy, Michael B. Buckley, Akash Arora, Frank S. Bates, Kevin D. Dorfman, Gregory M. Grason

Abstract: Single molecular species can self-assemble into Frank Kasper (FK) phases, finite approximants of dodecagonal quasicrystals, defying intuitive notions that thermodynamic ground states are maximally symmetric. FK phases are speculated to emerge as the minimal-distortional packings of space-filling spherical domains, but a precise quantitation of this distortion and how it affects assembly thermodyna… ▽ More Single molecular species can self-assemble into Frank Kasper (FK) phases, finite approximants of dodecagonal quasicrystals, defying intuitive notions that thermodynamic ground states are maximally symmetric. FK phases are speculated to emerge as the minimal-distortional packings of space-filling spherical domains, but a precise quantitation of this distortion and how it affects assembly thermodynamics remains ambiguous. We use two complementary approaches to demonstrate that the principles driving FK lattice formation in diblock copolymers emerge directly from the strong-stretching theory of spherical domains, in which minimal inter-block area competes with minimal stretching of space-filling chains. The relative stability of FK lattices is studied first using a diblock foam model with unconstrained particle volumes and shapes, which correctly predicts not only the equilibrium σ lattice, but also the unequal volumes of the equilibrium domains. We then provide a molecular interpretation for these results via self-consistent field theory, illuminating how molecular stiffness regulates the coupling between intra-domain chain configurations and the asymmetry of local packing. These findings shed new light on the role of volume exchange on the formation of distinct FK phases in copolymers, and suggest a paradigm for formation of FK phases in soft matter systems in which unequal domain volumes are selected by the thermodynamic competition between distinct measures of shape asymmetry. △ Less

Submitted 5 June, 2018; originally announced June 2018.

Comments: 40 pages, 22 figures

Log-ratio Lasso: Scalable, Sparse Estimation for Log-ratio Models

Authors: Stephen Bates, Robert Tibshirani

Abstract: Positive-valued signal data is common in many biological and medical applications, where the data are often generated from imaging techniques such as mass spectrometry. In such a setting, the relative intensities of the raw features are often the scientifically meaningful quantities, so it is of interest to identify relevant features that take the form of log-ratios of the raw inputs. When includi… ▽ More Positive-valued signal data is common in many biological and medical applications, where the data are often generated from imaging techniques such as mass spectrometry. In such a setting, the relative intensities of the raw features are often the scientifically meaningful quantities, so it is of interest to identify relevant features that take the form of log-ratios of the raw inputs. When including the log-ratios of all pairs of predictors, the dimensionality of this predictor space becomes large, so computationally efficient statistical procedures are required. We introduce an embedding of the log-ratio parameter space into a space of much lower dimension and develop efficient penalized fitting procedure using this more tractable representation. This procedure serves as the foundation for a two-step fitting procedure that combines a convex filtering step with a second non-convex pruning step to yield highly sparse solutions. On a cancer proteomics data set we find that these methods fit highly sparse models with log-ratio features of known biological relevance while greatly improving upon the predictive accuracy of less interpretable methods. △ Less

Submitted 4 September, 2017; originally announced September 2017.

Journal ref: Biometrics 109 (2019) 613-624

In-Datacenter Performance Analysis of a Tensor Processing Unit

Authors: Norman P. Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, Suresh Bhatia, Nan Boden, Al Borchers, Rick Boyle, Pierre-luc Cantin, Clifford Chao, Chris Clark, Jeremy Coriell, Mike Daley, Matt Dau, Jeffrey Dean, Ben Gelb, Tara Vazir Ghaemmaghami, Rajendra Gottipati, William Gulland, Robert Hagmann, C. Richard Ho, Doug Hogberg , et al. (50 additional authors not shown)

Abstract: Many architects believe that major improvements in cost-energy-performance must now come from domain-specific hardware. This paper evaluates a custom ASIC---called a Tensor Processing Unit (TPU)---deployed in datacenters since 2015 that accelerates the inference phase of neural networks (NN). The heart of the TPU is a 65,536 8-bit MAC matrix multiply unit that offers a peak throughput of 92 TeraOp… ▽ More Many architects believe that major improvements in cost-energy-performance must now come from domain-specific hardware. This paper evaluates a custom ASIC---called a Tensor Processing Unit (TPU)---deployed in datacenters since 2015 that accelerates the inference phase of neural networks (NN). The heart of the TPU is a 65,536 8-bit MAC matrix multiply unit that offers a peak throughput of 92 TeraOps/second (TOPS) and a large (28 MiB) software-managed on-chip memory. The TPU's deterministic execution model is a better match to the 99th-percentile response-time requirement of our NN applications than are the time-varying optimizations of CPUs and GPUs (caches, out-of-order execution, multithreading, multiprocessing, prefetching, ...) that help average throughput more than guaranteed latency. The lack of such features helps explain why, despite having myriad MACs and a big memory, the TPU is relatively small and low power. We compare the TPU to a server-class Intel Haswell CPU and an Nvidia K80 GPU, which are contemporaries deployed in the same datacenters. Our workload, written in the high-level TensorFlow framework, uses production NN applications (MLPs, CNNs, and LSTMs) that represent 95% of our datacenters' NN inference demand. Despite low utilization for some applications, the TPU is on average about 15X - 30X faster than its contemporary GPU or CPU, with TOPS/Watt about 30X - 80X higher. Moreover, using the GPU's GDDR5 memory in the TPU would triple achieved TOPS and raise TOPS/Watt to nearly 70X the GPU and 200X the CPU. △ Less

Submitted 16 April, 2017; originally announced April 2017.

Comments: 17 pages, 11 figures, 8 tables. To appear at the 44th International Symposium on Computer Architecture (ISCA), Toronto, Canada, June 24-28, 2017

LOTUS: A low cost, ultraviolet spectrograph

Authors: I. A. Steele, J. M. Marchant, H. E. Jermak, R. M. Barnsley, S. D. Bates, N. R. Clay, A. Fitzsimmons, E. Jehin, G. Jones, C. J. Mottram, R. J. Smith, C. Snodgrass, M. de Val-Borro

Abstract: We describe the design, construction and commissioning of LOTUS; a simple, low-cost long-slit spectrograph for the Liverpool Telescope. The design is optimized for near-UV and visible wavelengths and uses all transmitting optics. It exploits the instrument focal plane field curvature to partially correct axial chromatic aberration. A stepped slit provides narrow (2.5x95 arcsec) and wide (5x25 arcs… ▽ More We describe the design, construction and commissioning of LOTUS; a simple, low-cost long-slit spectrograph for the Liverpool Telescope. The design is optimized for near-UV and visible wavelengths and uses all transmitting optics. It exploits the instrument focal plane field curvature to partially correct axial chromatic aberration. A stepped slit provides narrow (2.5x95 arcsec) and wide (5x25 arcsec) options that are optimized for spectral resolution and flux calibration respectively. On sky testing shows a wavelength range of 3200-6300 Angstroms with a peak system throughput (including detector quantum efficiency) of 15 per cent and wavelength dependant spectral resolution of R=225-430. By repeated observations of the symbiotic emission line star AG Peg we demonstrate the wavelength stability of the system is less than 2 Angstroms rms and is limited by the positioning of the object in the slit. The spectrograph is now in routine operation monitoring the activity of comet 67P/Churyumov-Gerasimenko during its current post-perihelion apparition. △ Less

Submitted 26 May, 2016; originally announced May 2016.

Comments: Accepted for publication in MNRAS. 10 pages. 14 figures

New Discoveries from the Arecibo 327 MHz Drift Pulsar Survey Radio Transient Search

Authors: J. S. Deneva, K. Stovall, M. A. McLaughlin, M. Bagchi, S. D. Bates, P. C. C. Freire, J. G. Martinez, F. Jenet, N. Garver-Daniels

Abstract: We present Clusterrank, a new algorithm for identifying dispersed astrophysical pulses. Such pulses are commonly detected from Galactic pulsars and rotating radio transients (RRATs), which are neutron stars with sporadic radio emission. More recently, isolated, highly dispersed pulses dubbed fast radio bursts (FRBs) have been identified as the potential signature of an extragalactic cataclysmic ra… ▽ More We present Clusterrank, a new algorithm for identifying dispersed astrophysical pulses. Such pulses are commonly detected from Galactic pulsars and rotating radio transients (RRATs), which are neutron stars with sporadic radio emission. More recently, isolated, highly dispersed pulses dubbed fast radio bursts (FRBs) have been identified as the potential signature of an extragalactic cataclysmic radio source distinct from pulsars and RRATs. Clusterrank helped us discover 14 pulsars and 8 RRATs in data from the Arecibo 327 MHz Drift Pulsar Survey (AO327). The new RRATs have DMs in the range $23.5 - 86.6$ pc cm$^{-3}$ and periods in the range $0.172 - 3.901$ s. The new pulsars have DMs in the range $23.6 - 133.3$ pc cm$^{-3}$ and periods in the range $1.249 - 5.012$ s, and include two nullers and a mode-switching object. We estimate an upper limit on the all-sky FRB rate of $10^5$ day$^{-1}$ for bursts with a width of 10 ms and flux density $\gtrsim 83$ mJy. The DMs of all new discoveries are consistent with a Galactic origin. In comparing statistics of the new RRATs with sources from the RRATalog, we find that both sets are drawn from the same period distribution. In contrast, we find that the period distribution of the new pulsars is different from the period distributions of canonical pulsars in the ATNF catalog or pulsars found in AO327 data by a periodicity search. This indicates that Clusterrank is a powerful complement to periodicity searches and uncovers a subset of the pulsar population that has so far been underrepresented in survey results and therefore in Galactic pulsar population models. △ Less

Submitted 30 March, 2016; v1 submitted 2 March, 2016; originally announced March 2016.

Comments: 41 pages, 16 figures, 4 tables, accepted by ApJ; added minor corrections to final ApJ proof

Five new Fast Radio Bursts from the HTRU high latitude survey: first evidence for two-component bursts

Authors: D. J. Champion, E. Petroff, M. Kramer, M. J. Keith, M. Bailes, E. D. Barr, S. D. Bates, N. D. R. Bhat, M. Burgay, S. Burke-Spolaor, C. M. L. Flynn, A. Jameson, S. Johnston, C. Ng, L. Levin, A. Possenti, B. W. Stappers, W. van Straten, C. Tiburzi, A. G. Lyne

Abstract: The detection of five new fast radio bursts (FRBs) found in the High Time Resolution Universe high latitude survey is presented. The rate implied is 6$^{+4}_{-3}\times~10^3$ (95%) FRBs sky$^{-1}$ day$^{-1}$ above a fluence of between 0.13 and 5.9 Jy ms for FRBs between 0.128 and 262 ms in duration. One of these FRBs has a clear two-component profile, each component is similar to the known populati… ▽ More The detection of five new fast radio bursts (FRBs) found in the High Time Resolution Universe high latitude survey is presented. The rate implied is 6$^{+4}_{-3}\times~10^3$ (95%) FRBs sky$^{-1}$ day$^{-1}$ above a fluence of between 0.13 and 5.9 Jy ms for FRBs between 0.128 and 262 ms in duration. One of these FRBs has a clear two-component profile, each component is similar to the known population of single component FRBs and are separated by 2.4(4) ms. All the FRB components appear to be unresolved following deconvolution with a scattering tail and accounting for intra-channel smearing. The two-component FRB also has the highest dispersion measure (1629 pc cm$^{-3}$) of any FRB to-date. Many of the proposed models to explain FRBs use a single high energy event involving compact objects (such as neutron star mergers) and therefore cannot easily explain a two-component FRB. Models that are based on extreme versions of flaring, pulsing or orbital events however could produce multiple component profiles. The compatibility of these models and the FRB rate implied by these detections is discussed. △ Less

Submitted 24 November, 2015; originally announced November 2015.

Comments: 5 pages, 1 figure, 1 table, submitted to MNRAS

Pulsar J0453+1559: A Double Neutron Star System with a Large Mass Asymmetry

Authors: J. G. Martinez, K. Stovall, P. C. C. Freire, J. S. Deneva, F. A. Jenet, M. A. McLaughlin, M. Bagchi, S. D. Bates, A. Ridolfi

Abstract: To understand the nature of supernovae and neutron star (NS) formation, as well as binary stellar evolution and their interactions, it is important to probe the distribution of NS masses. Until now, all double NS (DNS) systems have been measured to have a mass ratio close to unity (q $\geq$ 0.91). Here we report the measurement of the individual masses of the 4.07-day binary pulsar J0453+1559 from… ▽ More To understand the nature of supernovae and neutron star (NS) formation, as well as binary stellar evolution and their interactions, it is important to probe the distribution of NS masses. Until now, all double NS (DNS) systems have been measured to have a mass ratio close to unity (q $\geq$ 0.91). Here we report the measurement of the individual masses of the 4.07-day binary pulsar J0453+1559 from measurements of the rate of advance of periastron and Shapiro delay: The mass of the pulsar is 1.559(5) $M_{\odot}$ and that of its companion is 1.174(4) $M_{\odot}$; q = 0.75. If this companion is also a neutron star (NS), as indicated by the orbital eccentricity of the system (e=0.11), then its mass is the smallest precisely measured for any such object. The pulsar has a spin period of 45.7 ms and a spin derivative of 1.8616(7) x$10^-19$; from these we derive a characteristic age of ~ 4.1 x $10^9$ years and a magnetic field of ~ 2.9 x $10^9$ G,i.e, this pulsar was mildly recycled by accretion of matter from the progenitor of the companion star. This suggests that it was formed with (very approximately) its current mass. Thus NSs form with a wide range of masses, which is important for understanding their formation in supernovae. It is also important for the search for gravitational waves released during a NS-NS merger: it is now evident that we should not assume all DNS systems are symmetric. △ Less

Submitted 29 September, 2015; originally announced September 2015.

IO:I: A Near-Infrared Camera for the Liverpool Telescope

Authors: Robert Barnsley, Helen Jermak, Iain Steele, Robert Smith, Stuart Bates, Chris Mottram

Abstract: IO:I is a new instrument that has recently been commissioned for the Liverpool Telescope, extending current imaging capabilities beyond the optical and into the near infrared. Cost has been minimised by use of a previously decommissioned instrument's cryostat as the base for a prototype and retrofitting it with Teledyne's 1.7$μm$ cutoff Hawaii-2RG HgCdTe detector, SIDECAR ASIC controller and JADE2… ▽ More IO:I is a new instrument that has recently been commissioned for the Liverpool Telescope, extending current imaging capabilities beyond the optical and into the near infrared. Cost has been minimised by use of a previously decommissioned instrument's cryostat as the base for a prototype and retrofitting it with Teledyne's 1.7$μm$ cutoff Hawaii-2RG HgCdTe detector, SIDECAR ASIC controller and JADE2 interface card. In this paper, the mechanical, electronic and cryogenic aspects of the cryostat retrofitting process will be reviewed together with a description of the software/hardware setup. This is followed by a discussion of the results derived from characterisation tests, including measurements of read noise, conversion gain, full well depth and linearity. The paper closes with a brief overview of the autonomous data reduction process and the presentation of results from photometric testing conducted on on-sky, pipeline processed data. △ Less

Submitted 21 December, 2015; v1 submitted 3 July, 2015; originally announced July 2015.

Comments: v1: 35 pages, 18 figures. Submitted to the Journal of Astronomical Telescopes, Instruments, and Systems (JATIS). v2: post peer review

A search for rotating radio transients and fast radio bursts in the Parkes high-latitude pulsar survey

Authors: A. Rane, D. R. Lorimer, S. D. Bates, N. McMann, M. A. McLaughlin, K. Rajwade

Abstract: Discoveries of rotating radio transients and fast radio bursts (FRBs) in pulsar surveys suggest that more of such transient sources await discovery in archival data sets. Here we report on a single-pulse search for dispersed radio bursts over a wide range of Galactic latitudes (|b| < $60^{\circ}$) in data previously searched for periodic sources by Burgay et al. We re-detected 20 of the 42 pulsars… ▽ More Discoveries of rotating radio transients and fast radio bursts (FRBs) in pulsar surveys suggest that more of such transient sources await discovery in archival data sets. Here we report on a single-pulse search for dispersed radio bursts over a wide range of Galactic latitudes (|b| < $60^{\circ}$) in data previously searched for periodic sources by Burgay et al. We re-detected 20 of the 42 pulsars reported by Burgay et al. and one rotating radio transient reported by Burke-Spolaor. No FRBs were discovered in this survey. Taking into account this result, and other recent surveys at Parkes, we corrected for detection sensitivities based on the search software used in the analyses and the different backends used in these surveys and find that the all-sky FRB event rate for sources with a fluence above 4.0 Jy ms at 1.4 GHz to be ${\cal R} = 4.4^{+5.2}_{-3.1} \times 10^3$ FRBs day$^{-1}$ sky$^{-1}$, where the uncertainties represent a $99\%$ confidence interval. While this rate is lower than inferred from previous studies, as we demonstrate, this combined event rate is consistent with the results of all systematic FRB searches at Parkes to date and does not require the need to postulate a dearth of FRBs at intermediate latitudes. △ Less

Submitted 15 October, 2015; v1 submitted 4 May, 2015; originally announced May 2015.

Comments: Accepted, 10 pages, 6 figures