Showing 1–50 of 132 results for author: Roberts, S

Optimized Model Selection for Estimating Treatment Effects from Costly Simulations of the US Opioid Epidemic

Authors: Abdulrahman A. Ahmed, M. Amin Rahimian, Mark S. Roberts

Abstract: Agent-based simulation with a synthetic population can help us compare different treatment conditions while kee** everything else constant within the same population (i.e., as digital twins). Such population-scale simulations require large computational power (i.e., CPU resources) to get accurate estimates for treatment effects. We can use meta models of the simulation results to circumvent the… ▽ More Agent-based simulation with a synthetic population can help us compare different treatment conditions while kee** everything else constant within the same population (i.e., as digital twins). Such population-scale simulations require large computational power (i.e., CPU resources) to get accurate estimates for treatment effects. We can use meta models of the simulation results to circumvent the need to simulate every treatment condition. Selecting the best estimating model at a given sample size (number of simulation runs) is a crucial problem. Depending on the sample size, the ability of the method to estimate accurately can change significantly. In this paper, we discuss different methods to explore what model works best at a specific sample size. In addition to the empirical results, we provide a mathematical analysis of the MSE equation and how its components decide which model to select and why a specific method behaves that way in a range of sample sizes. The analysis showed why the direction estimation method is better than model-based methods in larger sample sizes and how the between-group variation and the within-group variation affect the MSE equation. △ Less

Submitted 23 March, 2024; originally announced March 2024.

Comments: To be presented in 2024 Annual Simulation Conference (ANNSIM'24)

Few-Shot Learning Patterns in Financial Time-Series for Trend-Following Strategies

Authors: Kieran Wood, Samuel Kessler, Stephen J. Roberts, Stefan Zohren

Abstract: Forecasting models for systematic trading strategies do not adapt quickly when financial market conditions rapidly change, as was seen in the advent of the COVID-19 pandemic in 2020, causing many forecasting models to take loss-making positions. To deal with such situations, we propose a novel time-series trend-following forecaster that can quickly adapt to new market conditions, referred to as re… ▽ More Forecasting models for systematic trading strategies do not adapt quickly when financial market conditions rapidly change, as was seen in the advent of the COVID-19 pandemic in 2020, causing many forecasting models to take loss-making positions. To deal with such situations, we propose a novel time-series trend-following forecaster that can quickly adapt to new market conditions, referred to as regimes. We leverage recent developments from the deep learning community and use few-shot learning. We propose the Cross Attentive Time-Series Trend Network -- X-Trend -- which takes positions attending over a context set of financial time-series regimes. X-Trend transfers trends from similar patterns in the context set to make forecasts, then subsequently takes positions for a new distinct target regime. By quickly adapting to new financial regimes, X-Trend increases Sharpe ratio by 18.9% over a neural forecaster and 10-fold over a conventional Time-series Momentum strategy during the turbulent market period from 2018 to 2023. Our strategy recovers twice as quickly from the COVID-19 drawdown compared to the neural-forecaster. X-Trend can also take zero-shot positions on novel unseen financial assets obtaining a 5-fold Sharpe ratio increase versus a neural time-series trend forecaster over the same period. Furthermore, the cross-attention mechanism allows us to interpret the relationship between forecasts and patterns in the context set. △ Less

Submitted 28 March, 2024; v1 submitted 16 October, 2023; originally announced October 2023.

Comments: minor edits

In Consideration of Indigenous Data Sovereignty: Data Mining as a Colonial Practice

Authors: Jennafer Shae Roberts, Laura N Montoya

Abstract: Data mining reproduces colonialism, and Indigenous voices are being left out of the development of technology that relies on data, such as artificial intelligence. This research stresses the need for the inclusion of Indigenous Data Sovereignty and centers on the importance of Indigenous rights over their own data. Inclusion is necessary in order to integrate Indigenous knowledge into the design,… ▽ More Data mining reproduces colonialism, and Indigenous voices are being left out of the development of technology that relies on data, such as artificial intelligence. This research stresses the need for the inclusion of Indigenous Data Sovereignty and centers on the importance of Indigenous rights over their own data. Inclusion is necessary in order to integrate Indigenous knowledge into the design, development, and implementation of data-reliant technology. To support this hypothesis and address the problem, the CARE Principles for Indigenous Data Governance (Collective Benefit, Authority to Control, Responsibility, and Ethics) are applied. We cover how the colonial practices of data mining do not align with Indigenous convictions. The included case studies highlight connections to Indigenous rights in relation to the protection of data and environmental ecosystems, thus establishing how data governance can serve both the people and the Earth. By applying the CARE Principles to the issues that arise from data mining and neocolonialism, our goal is to provide a framework that can be used in technological development. The theory is that this could reflect outwards to promote data sovereignty generally and create new relationships between people and data that are ethical as opposed to driven by speed and profit. △ Less

Submitted 18 September, 2023; originally announced September 2023.

Comments: 12 pages, 1 Figure, Future Technologies Conference (FTC) 2023. arXiv admin note: substantial text overlap with arXiv:2208.04700

Projected Task-Specific Layers for Multi-Task Reinforcement Learning

Authors: Josselin Somerville Roberts, Julia Di

Abstract: Multi-task reinforcement learning could enable robots to scale across a wide variety of manipulation tasks in homes and workplaces. However, generalizing from one task to another and mitigating negative task interference still remains a challenge. Addressing this challenge by successfully sharing information across tasks will depend on how well the structure underlying the tasks is captured. In th… ▽ More Multi-task reinforcement learning could enable robots to scale across a wide variety of manipulation tasks in homes and workplaces. However, generalizing from one task to another and mitigating negative task interference still remains a challenge. Addressing this challenge by successfully sharing information across tasks will depend on how well the structure underlying the tasks is captured. In this work, we introduce our new architecture, Projected Task-Specific Layers (PTSL), that leverages a common policy with dense task-specific corrections through task-specific layers to better express shared and variable task information. We then show that our model outperforms the state of the art on the MT10 and MT50 benchmarks of Meta-World consisting of 10 and 50 goal-conditioned tasks for a Sawyer arm. △ Less

Submitted 6 March, 2024; v1 submitted 15 September, 2023; originally announced September 2023.

Journal ref: ICRA 2024

A Skeleton-based Approach For Rock Crack Detection Towards A Climbing Robot Application

Authors: Josselin Somerville Roberts, Paul-Emile Giacomelli, Yoni Gozlan, Julia Di

Abstract: Conventional wheeled robots are unable to traverse scientifically interesting, but dangerous, cave environments. Multi-limbed climbing robot designs, such as ReachBot, are able to grasp irregular surface features and execute climbing motions to overcome obstacles, given suitable grasp locations. To support grasp site identification, we present a method for detecting rock cracks and edges, the SKel… ▽ More Conventional wheeled robots are unable to traverse scientifically interesting, but dangerous, cave environments. Multi-limbed climbing robot designs, such as ReachBot, are able to grasp irregular surface features and execute climbing motions to overcome obstacles, given suitable grasp locations. To support grasp site identification, we present a method for detecting rock cracks and edges, the SKeleton Intersection Loss (SKIL). SKIL is a loss designed for thin object segmentation that leverages the skeleton of the label. A dataset of rock face images was collected, manually annotated, and augmented with generated data. A new group of metrics, LineAcc, has been proposed for thin object segmentation such that the impact of the object width on the score is minimized. In addition, the metric is less sensitive to translation which can often lead to a score of zero when computing classical metrics such as Dice on thin objects. Our fine-tuned models outperform previous methods on similar thin object segmentation tasks such as blood vessel segmentation and show promise for integration onto a robotic system. △ Less

Submitted 6 November, 2023; v1 submitted 10 September, 2023; originally announced September 2023.

Journal ref: IEEE IRC 2023

Estimating Treatment Effects Using Costly Simulation Samples from a Population-Scale Model of Opioid Use Disorder

Authors: Abdulrahman A. Ahmed, M. Amin Rahimian, Mark S. Roberts

Abstract: Large-scale models require substantial computational resources for analysis and studying treatment conditions. Specifically, estimating treatment effects using simulations may require a lot of infeasible resources to allocate at every treatment condition. Therefore, it is essential to develop efficient methods to allocate computational resources for estimating treatment effects. Agent-based simula… ▽ More Large-scale models require substantial computational resources for analysis and studying treatment conditions. Specifically, estimating treatment effects using simulations may require a lot of infeasible resources to allocate at every treatment condition. Therefore, it is essential to develop efficient methods to allocate computational resources for estimating treatment effects. Agent-based simulation allows us to generate highly realistic simulation samples. FRED (A Framework for Reconstructing Epidemiological Dynamics) is an agent-based modeling system with a geospatial perspective using a synthetic population constructed based on the U.S. census data. Given its synthetic population, FRED simulations present a baseline for comparable results from different treatment conditions and treatment conditions. In this paper, we show three other methods for estimating treatment effects. In the first method, we resort to brute-force allocation, where all treatment conditions have an equal number of samples with a relatively large number of simulation runs. In the second method, we try to reduce the number of simulation runs by customizing individual samples required for each treatment effect based on the width of confidence intervals around the mean estimates. In the third method, we use a regression model, which allows us to learn across the treatment conditions such that simulation samples allocated for a treatment condition will help better estimate treatment effects in other conditions. We show that the regression-based methods result in a comparable estimate of treatment effects with less computational resources. The reduced variability and faster convergence of model-based estimates come at the cost of increased bias, and the bias-variance trade-off can be controlled by adjusting the number of model parameters (e.g., including higher-order interaction terms in the regression model). △ Less

Submitted 24 August, 2023; originally announced August 2023.

Comments: To be presented in IEEE International Conference on Biomedical and Health Informatics 2023, repository link: https://github.com/abdulrahmanfci/intervention-estimation

Learning to Learn Financial Networks for Optimising Momentum Strategies

Authors: Xingyue Pu, Stefan Zohren, Stephen Roberts, Xiaowen Dong

Abstract: Network momentum provides a novel type of risk premium, which exploits the interconnections among assets in a financial network to predict future returns. However, the current process of constructing financial networks relies heavily on expensive databases and financial expertise, limiting accessibility for small-sized and academic institutions. Furthermore, the traditional approach treats network… ▽ More Network momentum provides a novel type of risk premium, which exploits the interconnections among assets in a financial network to predict future returns. However, the current process of constructing financial networks relies heavily on expensive databases and financial expertise, limiting accessibility for small-sized and academic institutions. Furthermore, the traditional approach treats network construction and portfolio optimisation as separate tasks, potentially hindering optimal portfolio performance. To address these challenges, we propose L2GMOM, an end-to-end machine learning framework that simultaneously learns financial networks and optimises trading signals for network momentum strategies. The model of L2GMOM is a neural network with a highly interpretable forward propagation architecture, which is derived from algorithm unrolling. The L2GMOM is flexible and can be trained with diverse loss functions for portfolio performance, e.g. the negative Sharpe ratio. Backtesting on 64 continuous future contracts demonstrates a significant improvement in portfolio profitability and risk control, with a Sharpe ratio of 1.74 across a 20-year period. △ Less

Submitted 23 August, 2023; originally announced August 2023.

Comments: 9 pages

Network Momentum across Asset Classes

Authors: Xingyue Pu, Stephen Roberts, Xiaowen Dong, Stefan Zohren

Abstract: We investigate the concept of network momentum, a novel trading signal derived from momentum spillover across assets. Initially observed within the confines of pairwise economic and fundamental ties, such as the stock-bond connection of the same company and stocks linked through supply-demand chains, momentum spillover implies a propagation of momentum risk premium from one asset to another. The s… ▽ More We investigate the concept of network momentum, a novel trading signal derived from momentum spillover across assets. Initially observed within the confines of pairwise economic and fundamental ties, such as the stock-bond connection of the same company and stocks linked through supply-demand chains, momentum spillover implies a propagation of momentum risk premium from one asset to another. The similarity of momentum risk premium, exemplified by co-movement patterns, has been spotted across multiple asset classes including commodities, equities, bonds and currencies. However, studying the network effect of momentum spillover across these classes has been challenging due to a lack of readily available common characteristics or economic ties beyond the company level. In this paper, we explore the interconnections of momentum features across a diverse range of 64 continuous future contracts spanning these four classes. We utilise a linear and interpretable graph learning model with minimal assumptions to reveal the intricacies of the momentum spillover network. By leveraging the learned networks, we construct a network momentum strategy that exhibits a Sharpe ratio of 1.5 and an annual return of 22%, after volatility scaling, from 2000 to 2022. This paper pioneers the examination of momentum spillover across multiple asset classes using only pricing data, presents a multi-asset investment strategy based on network momentum, and underscores the effectiveness of this strategy through robust empirical analysis. △ Less

Submitted 22 August, 2023; originally announced August 2023.

Comments: 27 pages

The Glamorisation of Unpaid Labour: AI and its Influencers

Authors: Nana Mgbechikwere Nwachukwu, Jennafer Shae Roberts, Laura N Montoya

Abstract: To harness the true potential of Artificial Intelligence (AI) for societal betterment, we need to move away from prioritising corporate interests which exploit Global South workers in the digital age. The unpaid labour and societal harms which are generated by Digital Value Networks (DVNs) disproportionately affect workers in Africa, Latin America, and India and need to be regulated. In this resea… ▽ More To harness the true potential of Artificial Intelligence (AI) for societal betterment, we need to move away from prioritising corporate interests which exploit Global South workers in the digital age. The unpaid labour and societal harms which are generated by Digital Value Networks (DVNs) disproportionately affect workers in Africa, Latin America, and India and need to be regulated. In this research, we discuss unethical practices to automate Human Intelligence Tasks (HITs) through gig work platforms and the capitalisation of data collection utilising influencers in social media. These are important areas of study in worker and user data practices, where ethical AI could be impactful. We provide suggestions for a path forward focused on responsible AI development. △ Less

Submitted 15 September, 2023; v1 submitted 31 July, 2023; originally announced August 2023.

Comments: 4 pages, 2 pages of references, Deep Learning Indaba 2023 Short Paper

Inferring epidemic dynamics using Gaussian process emulation of agent-based simulations

Authors: Abdulrahman A. Ahmed, M. Amin Rahimian, Mark S. Roberts

Abstract: Computational models help decision makers understand epidemic dynamics to optimize public health interventions. Agent-based simulation of disease spread in synthetic populations allows us to compare and contrast different effects across identical populations or to investigate the effect of interventions kee** every other factor constant between ``digital twins''. FRED (A Framework for Reconstruc… ▽ More Computational models help decision makers understand epidemic dynamics to optimize public health interventions. Agent-based simulation of disease spread in synthetic populations allows us to compare and contrast different effects across identical populations or to investigate the effect of interventions kee** every other factor constant between ``digital twins''. FRED (A Framework for Reconstructing Epidemiological Dynamics) is an agent-based modeling system with a geo-spatial perspective using a synthetic population that is constructed based on the U.S. census data. In this paper, we show how Gaussian process regression can be used on FRED-synthesized data to infer the differing spatial dispersion of the epidemic dynamics for two disease conditions that start from the same initial conditions and spread among identical populations. Our results showcase the utility of agent-based simulation frameworks such as FRED for inferring differences between conditions where controlling for all confounding factors for such comparisons is next to impossible without synthetic data. △ Less

Submitted 11 September, 2023; v1 submitted 22 July, 2023; originally announced July 2023.

Comments: To be presented in Winter Simulation Conference 2023, repository link: https://github.com/abdulrahmanfci/gpr-abm

The instabilities of large learning rate training: a loss landscape view

Authors: Lawrence Wang, Stephen Roberts

Abstract: Modern neural networks are undeniably successful. Numerous works study how the curvature of loss landscapes can affect the quality of solutions. In this work we study the loss landscape by considering the Hessian matrix during network training with large learning rates - an attractive regime that is (in)famously unstable. We characterise the instabilities of gradient descent, and we observe the st… ▽ More Modern neural networks are undeniably successful. Numerous works study how the curvature of loss landscapes can affect the quality of solutions. In this work we study the loss landscape by considering the Hessian matrix during network training with large learning rates - an attractive regime that is (in)famously unstable. We characterise the instabilities of gradient descent, and we observe the striking phenomena of \textit{landscape flattening} and \textit{landscape shift}, both of which are intimately connected to the instabilities of training. △ Less

Submitted 21 July, 2023; originally announced July 2023.

Comments: arXiv admin note: text overlap with arXiv:2305.18490

G-TRACER: Expected Sharpness Optimization

Authors: John Williams, Stephen Roberts

Abstract: We propose a new regularization scheme for the optimization of deep learning architectures, G-TRACER ("Geometric TRACE Ratio"), which promotes generalization by seeking flat minima, and has a sound theoretical basis as an approximation to a natural-gradient descent based optimization of a generalized Bayes objective. By augmenting the loss function with a TRACER, curvature-regularized optimizers (… ▽ More We propose a new regularization scheme for the optimization of deep learning architectures, G-TRACER ("Geometric TRACE Ratio"), which promotes generalization by seeking flat minima, and has a sound theoretical basis as an approximation to a natural-gradient descent based optimization of a generalized Bayes objective. By augmenting the loss function with a TRACER, curvature-regularized optimizers (eg SGD-TRACER and Adam-TRACER) are simple to implement as modifications to existing optimizers and don't require extensive tuning. We show that the method converges to a neighborhood (depending on the regularization strength) of a local minimum of the unregularized objective, and demonstrate competitive performance on a number of benchmark computer vision and NLP datasets, with a particular focus on challenging low signal-to-noise ratio problems. △ Less

Submitted 24 June, 2023; originally announced June 2023.

Comments: 16 pages, 2 figures

MSC Class: 62-08

Sub-Meter Tree Height Map** of California using Aerial Images and LiDAR-Informed U-Net Model

Authors: Fabien H Wagner, Sophia Roberts, Alison L Ritz, Griffin Carter, Ricardo Dalagnol, Samuel Favrichon, Mayumi CM Hirye, Martin Brandt, Philipe Ciais, Sassan Saatchi

Abstract: Tree canopy height is one of the most important indicators of forest biomass, productivity, and species diversity, but it is challenging to measure accurately from the ground and from space. Here, we used a U-Net model adapted for regression to map the canopy height of all trees in the state of California with very high-resolution aerial imagery (60 cm) from the USDA-NAIP program. The U-Net model… ▽ More Tree canopy height is one of the most important indicators of forest biomass, productivity, and species diversity, but it is challenging to measure accurately from the ground and from space. Here, we used a U-Net model adapted for regression to map the canopy height of all trees in the state of California with very high-resolution aerial imagery (60 cm) from the USDA-NAIP program. The U-Net model was trained using canopy height models computed from aerial LiDAR data as a reference, along with corresponding RGB-NIR NAIP images collected in 2020. We evaluated the performance of the deep-learning model using 42 independent 1 km$^2$ sites across various forest types and landscape variations in California. Our predictions of tree heights exhibited a mean error of 2.9 m and showed relatively low systematic bias across the entire range of tree heights present in California. In 2020, trees taller than 5 m covered ~ 19.3% of California. Our model successfully estimated canopy heights up to 50 m without saturation, outperforming existing canopy height products from global models. The approach we used allowed for the reconstruction of the three-dimensional structure of individual trees as observed from nadir-looking optical airborne imagery, suggesting a relatively robust estimation and map** capability, even in the presence of image distortion. These findings demonstrate the potential of large-scale map** and monitoring of tree height, as well as potential biomass estimation, using NAIP imagery. △ Less

Submitted 2 June, 2023; originally announced June 2023.

Comments: 29 pages, 9 figures, submitted to Remote Sensing in Ecology and Conservation (RSEC)

MSC Class: 92-08 ACM Class: I.4.9; I.5.4

SANE: The phases of gradient descent through Sharpness Adjusted Number of Effective parameters

Authors: Lawrence Wang, Stephen J. Roberts

Abstract: Modern neural networks are undeniably successful. Numerous studies have investigated how the curvature of loss landscapes can affect the quality of solutions. In this work we consider the Hessian matrix during network training. We reiterate the connection between the number of "well-determined" or "effective" parameters and the generalisation performance of neural nets, and we demonstrate its use… ▽ More Modern neural networks are undeniably successful. Numerous studies have investigated how the curvature of loss landscapes can affect the quality of solutions. In this work we consider the Hessian matrix during network training. We reiterate the connection between the number of "well-determined" or "effective" parameters and the generalisation performance of neural nets, and we demonstrate its use as a tool for model comparison. By considering the local curvature, we propose Sharpness Adjusted Number of Effective parameters (SANE), a measure of effective dimensionality for the quality of solutions. We show that SANE is robust to large learning rates, which represent learning regimes that are attractive but (in)famously unstable. We provide evidence and characterise the Hessian shifts across "loss basins" at large learning rates. Finally, extending our analysis to deeper neural networks, we provide an approximation to the full-network Hessian, exploiting the natural ordering of neural weights, and use this approximation to provide extensive empirical evidence for our claims. △ Less

Submitted 29 May, 2023; originally announced May 2023.

The Rhythms of Transient Relationships: Allocating time between weekdays and weekends

Authors: Valentín Vergara Hidd, Mailun Zhang, Simone Centellegher, Sam G. B. Roberts, Bruno Lepri, Eduardo López

Abstract: A fundamental question of any new relationship is, will it last? Transient relationships, recently defined by the authors, are an ideal type of social tie to explore this question: these relationships are characterized by distinguishable starting and ending temporal points, linking the question of tie longevity to relationship finite lifetime. In this study, we use mobile phone data sets from the… ▽ More A fundamental question of any new relationship is, will it last? Transient relationships, recently defined by the authors, are an ideal type of social tie to explore this question: these relationships are characterized by distinguishable starting and ending temporal points, linking the question of tie longevity to relationship finite lifetime. In this study, we use mobile phone data sets from the UK and Italy to analyze the weekly allocation of time invested in maintaining transient relationships. We find that more relationships are created during weekdays, with a greater proportion of them receiving more contact during these days of the week in the long term. The smaller group of relationships that receive more phone calls during the weekend tend to remain active for more time. We uncover a sorting process by which some ties are moved from weekdays to weekends and vice versa, mostly in the first half of the relationship. This process also carries more information about the ultimate lifetime of a tie than the part of the week when the relationship started, which suggests an early evaluation period that leads to a decision on how to allocate time to different types of transient ties. △ Less

Submitted 28 August, 2023; v1 submitted 24 May, 2023; originally announced May 2023.

Comments: 15 pages, 4 figures. Submitted for review at Royal Society Open Science R1

Spatio-Temporal Momentum: Jointly Learning Time-Series and Cross-Sectional Strategies

Authors: Wee Ling Tan, Stephen Roberts, Stefan Zohren

Abstract: We introduce Spatio-Temporal Momentum strategies, a class of models that unify both time-series and cross-sectional momentum strategies by trading assets based on their cross-sectional momentum features over time. While both time-series and cross-sectional momentum strategies are designed to systematically capture momentum risk premia, these strategies are regarded as distinct implementations and… ▽ More We introduce Spatio-Temporal Momentum strategies, a class of models that unify both time-series and cross-sectional momentum strategies by trading assets based on their cross-sectional momentum features over time. While both time-series and cross-sectional momentum strategies are designed to systematically capture momentum risk premia, these strategies are regarded as distinct implementations and do not consider the concurrent relationship and predictability between temporal and cross-sectional momentum features of different assets. We model spatio-temporal momentum with neural networks of varying complexities and demonstrate that a simple neural network with only a single fully connected layer learns to simultaneously generate trading signals for all assets in a portfolio by incorporating both their time-series and cross-sectional momentum features. Backtesting on portfolios of 46 actively-traded US equities and 12 equity index futures contracts, we demonstrate that the model is able to retain its performance over benchmarks in the presence of high transaction costs of up to 5-10 basis points. In particular, we find that the model when coupled with least absolute shrinkage and turnover regularization results in the best performance over various transaction cost scenarios. △ Less

Submitted 20 February, 2023; originally announced February 2023.

Journal ref: The Journal of Financial Data Science, Summer 2023

On Sequential Bayesian Inference for Continual Learning

Authors: Samuel Kessler, Adam Cobb, Tim G. J. Rudner, Stefan Zohren, Stephen J. Roberts

Abstract: Sequential Bayesian inference can be used for continual learning to prevent catastrophic forgetting of past tasks and provide an informative prior when learning new tasks. We revisit sequential Bayesian inference and test whether having access to the true posterior is guaranteed to prevent catastrophic forgetting in Bayesian neural networks. To do this we perform sequential Bayesian inference usin… ▽ More Sequential Bayesian inference can be used for continual learning to prevent catastrophic forgetting of past tasks and provide an informative prior when learning new tasks. We revisit sequential Bayesian inference and test whether having access to the true posterior is guaranteed to prevent catastrophic forgetting in Bayesian neural networks. To do this we perform sequential Bayesian inference using Hamiltonian Monte Carlo. We propagate the posterior as a prior for new tasks by fitting a density estimator on Hamiltonian Monte Carlo samples. We find that this approach fails to prevent catastrophic forgetting demonstrating the difficulty in performing sequential Bayesian inference in neural networks. From there we study simple analytical examples of sequential Bayesian inference and CL and highlight the issue of model misspecification which can lead to sub-optimal continual learning performance despite exact inference. Furthermore, we discuss how task data imbalances can cause forgetting. From these limitations, we argue that we need probabilistic models of the continual learning generative process rather than relying on sequential Bayesian inference over Bayesian neural network weights. In this vein, we also propose a simple baseline called Prototypical Bayesian Continual Learning, which is competitive with state-of-the-art Bayesian continual learning methods on class incremental continual learning vision benchmarks. △ Less

Submitted 9 July, 2023; v1 submitted 4 January, 2023; originally announced January 2023.

Comments: Published in Entropy, 24 pages, 14 figures

Statistical Design and Analysis for Robust Machine Learning: A Case Study from COVID-19

Authors: Davide Pigoli, Kieran Baker, Jobie Budd, Lorraine Butler, Harry Coppock, Sabrina Egglestone, Steven G. Gilmour, Chris Holmes, David Hurley, Radka Jersakova, Ivan Kiskin, Vasiliki Koutra, Jonathon Mellor, George Nicholson, Joe Packham, Selina Patel, Richard Payne, Stephen J. Roberts, Björn W. Schuller, Ana Tendero-Cañadas, Tracey Thornley, Alexander Titcomb

Abstract: Since early in the coronavirus disease 2019 (COVID-19) pandemic, there has been interest in using artificial intelligence methods to predict COVID-19 infection status based on vocal audio signals, for example cough recordings. However, existing studies have limitations in terms of data collection and of the assessment of the performances of the proposed predictive models. This paper rigorously ass… ▽ More Since early in the coronavirus disease 2019 (COVID-19) pandemic, there has been interest in using artificial intelligence methods to predict COVID-19 infection status based on vocal audio signals, for example cough recordings. However, existing studies have limitations in terms of data collection and of the assessment of the performances of the proposed predictive models. This paper rigorously assesses state-of-the-art machine learning techniques used to predict COVID-19 infection status based on vocal audio signals, using a dataset collected by the UK Health Security Agency. This dataset includes acoustic recordings and extensive study participant meta-data. We provide guidelines on testing the performance of methods to classify COVID-19 infection status based on acoustic features and we discuss how these can be extended more generally to the development and assessment of predictive methods based on public health datasets. △ Less

Submitted 27 February, 2023; v1 submitted 15 December, 2022; originally announced December 2022.

Audio-based AI classifiers show no evidence of improved COVID-19 screening over simple symptoms checkers

Authors: Harry Coppock, George Nicholson, Ivan Kiskin, Vasiliki Koutra, Kieran Baker, Jobie Budd, Richard Payne, Emma Karoune, David Hurley, Alexander Titcomb, Sabrina Egglestone, Ana Tendero Cañadas, Lorraine Butler, Radka Jersakova, Jonathon Mellor, Selina Patel, Tracey Thornley, Peter Diggle, Sylvia Richardson, Josef Packham, Björn W. Schuller, Davide Pigoli, Steven Gilmour, Stephen Roberts, Chris Holmes

Abstract: Recent work has reported that AI classifiers trained on audio recordings can accurately predict severe acute respiratory syndrome coronavirus 2 (SARSCoV2) infection status. Here, we undertake a large scale study of audio-based deep learning classifiers, as part of the UK governments pandemic response. We collect and analyse a dataset of audio recordings from 67,842 individuals with linked metadata… ▽ More Recent work has reported that AI classifiers trained on audio recordings can accurately predict severe acute respiratory syndrome coronavirus 2 (SARSCoV2) infection status. Here, we undertake a large scale study of audio-based deep learning classifiers, as part of the UK governments pandemic response. We collect and analyse a dataset of audio recordings from 67,842 individuals with linked metadata, including reverse transcription polymerase chain reaction (PCR) test outcomes, of whom 23,514 tested positive for SARS CoV 2. Subjects were recruited via the UK governments National Health Service Test-and-Trace programme and the REal-time Assessment of Community Transmission (REACT) randomised surveillance survey. In an unadjusted analysis of our dataset AI classifiers predict SARS-CoV-2 infection status with high accuracy (Receiver Operating Characteristic Area Under the Curve (ROCAUC) 0.846 [0.838, 0.854]) consistent with the findings of previous studies. However, after matching on measured confounders, such as age, gender, and self reported symptoms, our classifiers performance is much weaker (ROC-AUC 0.619 [0.594, 0.644]). Upon quantifying the utility of audio based classifiers in practical settings, we find them to be outperformed by simple predictive scores based on user reported symptoms. △ Less

Submitted 2 March, 2023; v1 submitted 15 December, 2022; originally announced December 2022.

A large-scale and PCR-referenced vocal audio dataset for COVID-19

Authors: Jobie Budd, Kieran Baker, Emma Karoune, Harry Coppock, Selina Patel, Ana Tendero Cañadas, Alexander Titcomb, Richard Payne, David Hurley, Sabrina Egglestone, Lorraine Butler, Jonathon Mellor, George Nicholson, Ivan Kiskin, Vasiliki Koutra, Radka Jersakova, Rachel A. McKendry, Peter Diggle, Sylvia Richardson, Björn W. Schuller, Steven Gilmour, Davide Pigoli, Stephen Roberts, Josef Packham, Tracey Thornley , et al. (1 additional authors not shown)

Abstract: The UK COVID-19 Vocal Audio Dataset is designed for the training and evaluation of machine learning models that classify SARS-CoV-2 infection status or associated respiratory symptoms using vocal audio. The UK Health Security Agency recruited voluntary participants through the national Test and Trace programme and the REACT-1 survey in England from March 2021 to March 2022, during dominant transmi… ▽ More The UK COVID-19 Vocal Audio Dataset is designed for the training and evaluation of machine learning models that classify SARS-CoV-2 infection status or associated respiratory symptoms using vocal audio. The UK Health Security Agency recruited voluntary participants through the national Test and Trace programme and the REACT-1 survey in England from March 2021 to March 2022, during dominant transmission of the Alpha and Delta SARS-CoV-2 variants and some Omicron variant sublineages. Audio recordings of volitional coughs, exhalations, and speech were collected in the 'Speak up to help beat coronavirus' digital survey alongside demographic, self-reported symptom and respiratory condition data, and linked to SARS-CoV-2 test results. The UK COVID-19 Vocal Audio Dataset represents the largest collection of SARS-CoV-2 PCR-referenced audio recordings to date. PCR results were linked to 70,794 of 72,999 participants and 24,155 of 25,776 positive cases. Respiratory symptoms were reported by 45.62% of participants. This dataset has additional potential uses for bioacoustics research, with 11.30% participants reporting asthma, and 27.20% with linked influenza PCR test results. △ Less

Submitted 3 November, 2023; v1 submitted 15 December, 2022; originally announced December 2022.

Comments: 39 pages, 4 figures

The Effectiveness of World Models for Continual Reinforcement Learning

Authors: Samuel Kessler, Mateusz Ostaszewski, Michał Bortkiewicz, Mateusz Żarski, Maciej Wołczyk, Jack Parker-Holder, Stephen J. Roberts, Piotr Miłoś

Abstract: World models power some of the most efficient reinforcement learning algorithms. In this work, we showcase that they can be harnessed for continual learning - a situation when the agent faces changing environments. World models typically employ a replay buffer for training, which can be naturally extended to continual learning. We systematically study how different selective experience replay meth… ▽ More World models power some of the most efficient reinforcement learning algorithms. In this work, we showcase that they can be harnessed for continual learning - a situation when the agent faces changing environments. World models typically employ a replay buffer for training, which can be naturally extended to continual learning. We systematically study how different selective experience replay methods affect performance, forgetting, and transfer. We also provide recommendations regarding various modeling options for using world models. The best set of choices is called Continual-Dreamer, it is task-agnostic and utilizes the world model for continual exploration. Continual-Dreamer is sample efficient and outperforms state-of-the-art task-agnostic continual reinforcement learning methods on Minigrid and Minihack benchmarks. △ Less

Submitted 12 July, 2023; v1 submitted 29 November, 2022; originally announced November 2022.

Comments: Accepted at CoLLAs 2023, 21 pages, 15 figures

Learning General World Models in a Handful of Reward-Free Deployments

Authors: Yingchen Xu, Jack Parker-Holder, Aldo Pacchiano, Philip J. Ball, Oleh Rybkin, Stephen J. Roberts, Tim Rocktäschel, Edward Grefenstette

Abstract: Building generally capable agents is a grand challenge for deep reinforcement learning (RL). To approach this challenge practically, we outline two key desiderata: 1) to facilitate generalization, exploration should be task agnostic; 2) to facilitate scalability, exploration policies should collect large quantities of data without costly centralized retraining. Combining these two properties, we i… ▽ More Building generally capable agents is a grand challenge for deep reinforcement learning (RL). To approach this challenge practically, we outline two key desiderata: 1) to facilitate generalization, exploration should be task agnostic; 2) to facilitate scalability, exploration policies should collect large quantities of data without costly centralized retraining. Combining these two properties, we introduce the reward-free deployment efficiency setting, a new paradigm for RL research. We then present CASCADE, a novel approach for self-supervised exploration in this new setting. CASCADE seeks to learn a world model by collecting data with a population of agents, using an information theoretic objective inspired by Bayesian Active Learning. CASCADE achieves this by specifically maximizing the diversity of trajectories sampled by the population through a novel cascading objective. We provide theoretical intuition for CASCADE which we show in a tabular setting improves upon naïve approaches that do not account for population diversity. We then demonstrate that CASCADE collects diverse task-agnostic datasets and learns agents that generalize zero-shot to novel, unseen downstream tasks on Atari, MiniGrid, Crafter and the DM Control Suite. Code and videos are available at https://ycxuyingchen.github.io/cascade/ △ Less

Submitted 23 October, 2022; originally announced October 2022.

Comments: To be published at NeurIPS 2022. Code and videos available at https://ycxuyingchen.github.io/cascade/

Transfer Ranking in Finance: Applications to Cross-Sectional Momentum with Data Scarcity

Authors: Daniel Poh, Stephen Roberts, Stefan Zohren

Abstract: Cross-sectional strategies are a classical and popular trading style, with recent high performing variants incorporating sophisticated neural architectures. While these strategies have been applied successfully to data-rich settings involving mature assets with long histories, deploying them on instruments with limited samples generally produce over-fitted models with degraded performance. In this… ▽ More Cross-sectional strategies are a classical and popular trading style, with recent high performing variants incorporating sophisticated neural architectures. While these strategies have been applied successfully to data-rich settings involving mature assets with long histories, deploying them on instruments with limited samples generally produce over-fitted models with degraded performance. In this paper, we introduce Fused Encoder Networks -- a novel and hybrid parameter-sharing transfer ranking model. The model fuses information extracted using an encoder-attention module operated on a source dataset with a similar but separate module focused on a smaller target dataset of interest. This mitigates the issue of models with poor generalisability that are a consequence of training on scarce target data. Additionally, the self-attention mechanism enables interactions among instruments to be accounted for, not just at the loss level during model training, but also at inference time. Focusing on momentum applied to the top ten cryptocurrencies by market capitalisation as a demonstrative use-case, the Fused Encoder Networks outperforms the reference benchmarks on most performance measures, delivering a three-fold boost in the Sharpe ratio over classical momentum as well as an improvement of approximately 50% against the best benchmark model without transaction costs. It continues outperforming baselines even after accounting for the high transaction costs associated with trading cryptocurrencies. △ Less

Submitted 21 February, 2023; v1 submitted 21 August, 2022; originally announced August 2022.

Comments: 18 pages, 12 figures

Decolonisation, Global Data Law, and Indigenous Data Sovereignty

Authors: Jennafer Shae Roberts, Laura N Montoya

Abstract: This research examines the impact of digital neo-colonialism on the Global South and encourages the development of legal and economic incentives to protect Indigenous cultures globally. Data governance is discussed in an evolutionary context while focusing on data sharing and data mining. Case studies that exemplify the need to steer global data law towards protecting the earth, while addressing i… ▽ More This research examines the impact of digital neo-colonialism on the Global South and encourages the development of legal and economic incentives to protect Indigenous cultures globally. Data governance is discussed in an evolutionary context while focusing on data sharing and data mining. Case studies that exemplify the need to steer global data law towards protecting the earth, while addressing issues of data access, privacy, rights, and colonialism in the global South are explored. The case studies highlight connections to indigenous people's rights, in regard to the protection of environmental ecosystems, thus establishing how data law can serve the earth from an autochthonous lens. This framework examines histories shaped by colonialism and suggests how data governance could be used to create healthier balances of power. △ Less

Submitted 28 July, 2022; originally announced August 2022.

Comments: 16 pages, 1 table

Stabilizing Off-Policy Deep Reinforcement Learning from Pixels

Authors: Edoardo Cetin, Philip J. Ball, Steve Roberts, Oya Celiktutan

Abstract: Off-policy reinforcement learning (RL) from pixel observations is notoriously unstable. As a result, many successful algorithms must combine different domain-specific practices and auxiliary losses to learn meaningful behaviors in complex environments. In this work, we provide novel analysis demonstrating that these instabilities arise from performing temporal-difference learning with a convolutio… ▽ More Off-policy reinforcement learning (RL) from pixel observations is notoriously unstable. As a result, many successful algorithms must combine different domain-specific practices and auxiliary losses to learn meaningful behaviors in complex environments. In this work, we provide novel analysis demonstrating that these instabilities arise from performing temporal-difference learning with a convolutional encoder and low-magnitude rewards. We show that this new visual deadly triad causes unstable training and premature convergence to degenerate solutions, a phenomenon we name catastrophic self-overfitting. Based on our analysis, we propose A-LIX, a method providing adaptive regularization to the encoder's gradients that explicitly prevents the occurrence of catastrophic self-overfitting using a dual objective. By applying A-LIX, we significantly outperform the prior state-of-the-art on the DeepMind Control and Atari 100k benchmarks without any data augmentation or auxiliary losses. △ Less

Submitted 3 July, 2022; originally announced July 2022.

Comments: Short presentation at ICML 2022

The ACM Multimedia 2022 Computational Paralinguistics Challenge: Vocalisations, Stuttering, Activity, & Mosquitoes

Authors: Björn W. Schuller, Anton Batliner, Shahin Amiriparian, Christian Bergler, Maurice Gerczuk, Natalie Holz, Pauline Larrouy-Maestri, Sebastian P. Bayerl, Korbinian Riedhammer, Adria Mallol-Ragolta, Maria Pateraki, Harry Coppock, Ivan Kiskin, Marianne Sinka, Stephen Roberts

Abstract: The ACM Multimedia 2022 Computational Paralinguistics Challenge addresses four different problems for the first time in a research competition under well-defined conditions: In the Vocalisations and Stuttering Sub-Challenges, a classification on human non-verbal vocalisations and speech has to be made; the Activity Sub-Challenge aims at beyond-audio human activity recognition from smartwatch senso… ▽ More The ACM Multimedia 2022 Computational Paralinguistics Challenge addresses four different problems for the first time in a research competition under well-defined conditions: In the Vocalisations and Stuttering Sub-Challenges, a classification on human non-verbal vocalisations and speech has to be made; the Activity Sub-Challenge aims at beyond-audio human activity recognition from smartwatch sensor data; and in the Mosquitoes Sub-Challenge, mosquitoes need to be detected. We describe the Sub-Challenges, baseline feature extraction, and classifiers based on the usual ComPaRE and BoAW features, the auDeep toolkit, and deep feature extraction from pre-trained CNNs using the DeepSpectRum toolkit; in addition, we add end-to-end sequential modelling, and a log-mel-128-BNN. △ Less

Submitted 13 May, 2022; originally announced May 2022.

Comments: 5 pages, part of the ACM Multimedia 2022 Grand Challenge "The ACM Multimedia 2022 Computational Paralinguistics Challenge (ComParE 2022)"

MSC Class: 68 ACM Class: I.2.7; I.5.0; J.3

Contextualizing Artificially Intelligent Morality: A Meta-Ethnography of Top-Down, Bottom-Up, and Hybrid Models for Theoretical and Applied Ethics in Artificial Intelligence

Authors: Jennafer S. Roberts, Laura N. Montoya

Abstract: In this meta-ethnography, we explore three different angles of ethical artificial intelligence (AI) design implementation including the philosophical ethical viewpoint, the technical perspective, and framing through a political lens. Our qualitative research includes a literature review that highlights the cross-referencing of these angles by discussing the value and drawbacks of contrastive top-d… ▽ More In this meta-ethnography, we explore three different angles of ethical artificial intelligence (AI) design implementation including the philosophical ethical viewpoint, the technical perspective, and framing through a political lens. Our qualitative research includes a literature review that highlights the cross-referencing of these angles by discussing the value and drawbacks of contrastive top-down, bottom-up, and hybrid approaches previously published. The novel contribution to this framework is the political angle, which constitutes ethics in AI either being determined by corporations and governments and imposed through policies or law (coming from the top), or ethics being called for by the people (coming from the bottom), as well as top-down, bottom-up, and hybrid technicalities of how AI is developed within a moral construct and in consideration of its users, with expected and unexpected consequences and long-term impact in the world. There is a focus on reinforcement learning as an example of a bottom-up applied technical approach and AI ethics principles as a practical top-down approach. This investigation includes real-world case studies to impart a global perspective, as well as philosophical debate on the ethics of AI and theoretical future thought experimentation based on historical facts, current world circumstances, and possible ensuing realities. △ Less

Submitted 8 September, 2022; v1 submitted 15 April, 2022; originally announced April 2022.

Comments: 22 pages, 4 tables, accepted for publication in the Future of Information and Communication Conference (FICC) 2023 proceedings will be published in Springer series "Lecture Notes in Networks and Systems" and submitted for consideration to Web of Science, SCOPUS, INSPEC, WTI Frankfurt eG, zbMATH and SCImago

On-the-fly Strategy Adaptation for ad-hoc Agent Coordination

Authors: Jaleh Zand, Jack Parker-Holder, Stephen J. Roberts

Abstract: Training agents in cooperative settings offers the promise of AI agents able to interact effectively with humans (and other agents) in the real world. Multi-agent reinforcement learning (MARL) has the potential to achieve this goal, demonstrating success in a series of challenging problems. However, whilst these advances are significant, the vast majority of focus has been on the self-play paradig… ▽ More Training agents in cooperative settings offers the promise of AI agents able to interact effectively with humans (and other agents) in the real world. Multi-agent reinforcement learning (MARL) has the potential to achieve this goal, demonstrating success in a series of challenging problems. However, whilst these advances are significant, the vast majority of focus has been on the self-play paradigm. This often results in a coordination problem, caused by agents learning to make use of arbitrary conventions when playing with themselves. This means that even the strongest self-play agents may have very low cross-play with other agents, including other initializations of the same algorithm. In this paper we propose to solve this problem by adapting agent strategies on the fly, using a posterior belief over the other agents' strategy. Concretely, we consider the problem of selecting a strategy from a finite set of previously trained agents, to play with an unknown partner. We propose an extension of the classic statistical technique, Gibbs sampling, to update beliefs about other agents and obtain close to optimal ad-hoc performance. Despite its simplicity, our method is able to achieve strong cross-play with unseen partners in the challenging card game of Hanabi, achieving successful ad-hoc coordination without knowledge of the partner's strategy a priori. △ Less

Submitted 7 March, 2022; originally announced March 2022.

Comments: Extended abstract published in AAMAS 2022

Trading with the Momentum Transformer: An Intelligent and Interpretable Architecture

Authors: Kieran Wood, Sven Giegerich, Stephen Roberts, Stefan Zohren

Abstract: We introduce the Momentum Transformer, an attention-based deep-learning architecture, which outperforms benchmark time-series momentum and mean-reversion trading strategies. Unlike state-of-the-art Long Short-Term Memory (LSTM) architectures, which are sequential in nature and tailored to local processing, an attention mechanism provides our architecture with a direct connection to all previous ti… ▽ More We introduce the Momentum Transformer, an attention-based deep-learning architecture, which outperforms benchmark time-series momentum and mean-reversion trading strategies. Unlike state-of-the-art Long Short-Term Memory (LSTM) architectures, which are sequential in nature and tailored to local processing, an attention mechanism provides our architecture with a direct connection to all previous time-steps. Our architecture, an attention-LSTM hybrid, enables us to learn longer-term dependencies, improves performance when considering returns net of transaction costs and naturally adapts to new market regimes, such as during the SARS-CoV-2 crisis. Via the introduction of multiple attention heads, we can capture concurrent regimes, or temporal dynamics, which are occurring at different timescales. The Momentum Transformer is inherently interpretable, providing us with greater insights into our deep-learning momentum trading strategy, including the importance of different factors over time and the past time-steps which are of the greatest significance to the model. △ Less

Submitted 22 November, 2022; v1 submitted 15 December, 2021; originally announced December 2021.

Comments: included motivation for attention mechanism and additional architecture details

One-Shot Transfer Learning of Physics-Informed Neural Networks

Authors: Shaan Desai, Marios Mattheakis, Hayden Joy, Pavlos Protopapas, Stephen Roberts

Abstract: Solving differential equations efficiently and accurately sits at the heart of progress in many areas of scientific research, from classical dynamical systems to quantum mechanics. There is a surge of interest in using Physics-Informed Neural Networks (PINNs) to tackle such problems as they provide numerous benefits over traditional numerical approaches. Despite their potential benefits for solvin… ▽ More Solving differential equations efficiently and accurately sits at the heart of progress in many areas of scientific research, from classical dynamical systems to quantum mechanics. There is a surge of interest in using Physics-Informed Neural Networks (PINNs) to tackle such problems as they provide numerous benefits over traditional numerical approaches. Despite their potential benefits for solving differential equations, transfer learning has been under explored. In this study, we present a general framework for transfer learning PINNs that results in one-shot inference for linear systems of both ordinary and partial differential equations. This means that highly accurate solutions to many unknown differential equations can be obtained instantaneously without retraining an entire network. We demonstrate the efficacy of the proposed deep learning approach by solving several real-world problems, such as first- and second-order linear ordinary equations, the Poisson equation, and the time-dependent Schrodinger complex-value partial differential equation. △ Less

Submitted 5 July, 2022; v1 submitted 21 October, 2021; originally announced October 2021.

Comments: ICML AI4Science Workshop 2022

HumBugDB: A Large-scale Acoustic Mosquito Dataset

Authors: Ivan Kiskin, Marianne Sinka, Adam D. Cobb, Waqas Rafique, Lawrence Wang, Davide Zilli, Benjamin Gutteridge, Rinita Dam, Theodoros Marinos, Yunpeng Li, Dickson Msaky, Emmanuel Kaindoa, Gerard Killeen, Eva Herreros-Moya, Kathy J. Willis, Stephen J. Roberts

Abstract: This paper presents the first large-scale multi-species dataset of acoustic recordings of mosquitoes tracked continuously in free flight. We present 20 hours of audio recordings that we have expertly labelled and tagged precisely in time. Significantly, 18 hours of recordings contain annotations from 36 different species. Mosquitoes are well-known carriers of diseases such as malaria, dengue and y… ▽ More This paper presents the first large-scale multi-species dataset of acoustic recordings of mosquitoes tracked continuously in free flight. We present 20 hours of audio recordings that we have expertly labelled and tagged precisely in time. Significantly, 18 hours of recordings contain annotations from 36 different species. Mosquitoes are well-known carriers of diseases such as malaria, dengue and yellow fever. Collecting this dataset is motivated by the need to assist applications which utilise mosquito acoustics to conduct surveys to help predict outbreaks and inform intervention policy. The task of detecting mosquitoes from the sound of their wingbeats is challenging due to the difficulty in collecting recordings from realistic scenarios. To address this, as part of the HumBug project, we conducted global experiments to record mosquitoes ranging from those bred in culture cages to mosquitoes captured in the wild. Consequently, the audio recordings vary in signal-to-noise ratio and contain a broad range of indoor and outdoor background environments from Tanzania, Thailand, Kenya, the USA and the UK. In this paper we describe in detail how we collected, labelled and curated the data. The data is provided from a PostgreSQL database, which contains important metadata such as the capture method, age, feeding status and gender of the mosquitoes. Additionally, we provide code to extract features and train Bayesian convolutional neural networks for two key tasks: the identification of mosquitoes from their corresponding background environments, and the classification of detected mosquitoes into species. Our extensive dataset is both challenging to machine learning researchers focusing on acoustic identification, and critical to entomologists, geo-spatial modellers and other domain experts to understand mosquito behaviour, model their distribution, and manage the threat they pose to humans. △ Less

Submitted 14 October, 2021; originally announced October 2021.

Comments: Accepted at the 35th Conference on Neural Information Processing Systems (NeurIPS 2021) Track on Datasets and Benchmarks. 10 pages main, 39 pages including appendix. This paper accompanies the dataset found at https://zenodo.org/record/4904800 with corresponding code at https://github.com/HumBug-Mosquito/HumBugDB

ACM Class: E.0; I.2.1; J.3

Robust and Scalable SDE Learning: A Functional Perspective

Authors: Scott Cameron, Tyron Cameron, Arnu Pretorius, Stephen Roberts

Abstract: Stochastic differential equations provide a rich class of flexible generative models, capable of describing a wide range of spatio-temporal processes. A host of recent work looks to learn data-representing SDEs, using neural networks and other flexible function approximators. Despite these advances, learning remains computationally expensive due to the sequential nature of SDE integrators. In this… ▽ More Stochastic differential equations provide a rich class of flexible generative models, capable of describing a wide range of spatio-temporal processes. A host of recent work looks to learn data-representing SDEs, using neural networks and other flexible function approximators. Despite these advances, learning remains computationally expensive due to the sequential nature of SDE integrators. In this work, we propose an importance-sampling estimator for probabilities of observations of SDEs for the purposes of learning. Crucially, the approach we suggest does not rely on such integrators. The proposed method produces lower-variance gradient estimates compared to algorithms based on SDE integrators and has the added advantage of being embarrassingly parallelizable. This facilitates the effective use of large-scale parallel hardware for massive decreases in computation time. △ Less

Submitted 11 October, 2021; originally announced October 2021.

Revisiting Design Choices in Offline Model-Based Reinforcement Learning

Authors: Cong Lu, Philip J. Ball, Jack Parker-Holder, Michael A. Osborne, Stephen J. Roberts

Abstract: Offline reinforcement learning enables agents to leverage large pre-collected datasets of environment transitions to learn control policies, circumventing the need for potentially expensive or unsafe online data collection. Significant progress has been made recently in offline model-based reinforcement learning, approaches which leverage a learned dynamics model. This typically involves construct… ▽ More Offline reinforcement learning enables agents to leverage large pre-collected datasets of environment transitions to learn control policies, circumventing the need for potentially expensive or unsafe online data collection. Significant progress has been made recently in offline model-based reinforcement learning, approaches which leverage a learned dynamics model. This typically involves constructing a probabilistic model, and using the model uncertainty to penalize rewards where there is insufficient data, solving for a pessimistic MDP that lower bounds the true MDP. Existing methods, however, exhibit a breakdown between theory and practice, whereby pessimistic return ought to be bounded by the total variation distance of the model from the true dynamics, but is instead implemented through a penalty based on estimated model uncertainty. This has spawned a variety of uncertainty heuristics, with little to no comparison between differing approaches. In this paper, we compare these heuristics, and design novel protocols to investigate their interaction with other hyperparameters, such as the number of models, or imaginary rollout horizon. Using these insights, we show that selecting these key hyperparameters using Bayesian Optimization produces superior configurations that are vastly different to those currently used in existing hand-tuned state-of-the-art methods, and result in drastically stronger performance. △ Less

Submitted 16 March, 2022; v1 submitted 8 October, 2021; originally announced October 2021.

Comments: Spotlight @ ICLR 2022; Spotlight @ RL4RealLife Workshop ICML2021

Proximity in face-to-face interaction is associated with mobile phone communication

Authors: Tobias Bornakke, Talayeh Aledavood, Jari Saramäki, Sam G. B. Roberts

Abstract: The frequency of mobile communication is often used as an indicator of the strength of a tie between two individuals, but how mobile communication relates to other forms of behaving close in social relationships is poorly understood. We used a unique multi-channel 10-month dataset from 510 participants to examine how the frequency of mobile communication was related to the frequency of face-to-fac… ▽ More The frequency of mobile communication is often used as an indicator of the strength of a tie between two individuals, but how mobile communication relates to other forms of behaving close in social relationships is poorly understood. We used a unique multi-channel 10-month dataset from 510 participants to examine how the frequency of mobile communication was related to the frequency of face-to-face interaction, as measured by Bluetooth scans between the participants mobile phones. The number of phone calls between a dyad was significantly related to the number of face-to-face interactions. Physical proximity during face-to-face interactions was the single strongest predictor of the number of phone calls. Overall, 36 percent of variance in phone calls could be explained by face-to-face interactions and the control variables. Our results suggest that the amount of mobile communication between a dyad is a useful but noisy measure of tie strength with some significant limitations. △ Less

Submitted 20 July, 2021; v1 submitted 19 July, 2021; originally announced July 2021.

Comments: 31 pages, 4 Tables, 1 Figure

Port-Hamiltonian Neural Networks for Learning Explicit Time-Dependent Dynamical Systems

Authors: Shaan Desai, Marios Mattheakis, David Sondak, Pavlos Protopapas, Stephen Roberts

Abstract: Accurately learning the temporal behavior of dynamical systems requires models with well-chosen learning biases. Recent innovations embed the Hamiltonian and Lagrangian formalisms into neural networks and demonstrate a significant improvement over other approaches in predicting trajectories of physical systems. These methods generally tackle autonomous systems that depend implicitly on time or sys… ▽ More Accurately learning the temporal behavior of dynamical systems requires models with well-chosen learning biases. Recent innovations embed the Hamiltonian and Lagrangian formalisms into neural networks and demonstrate a significant improvement over other approaches in predicting trajectories of physical systems. These methods generally tackle autonomous systems that depend implicitly on time or systems for which a control signal is known apriori. Despite this success, many real world dynamical systems are non-autonomous, driven by time-dependent forces and experience energy dissipation. In this study, we address the challenge of learning from such non-autonomous systems by embedding the port-Hamiltonian formalism into neural networks, a versatile framework that can capture energy dissipation and time-dependent control forces. We show that the proposed \emph{port-Hamiltonian neural network} can efficiently learn the dynamics of nonlinear physical systems of practical interest and accurately recover the underlying stationary Hamiltonian, time-dependent force, and dissipative coefficient. A promising outcome of our network is its ability to learn and predict chaotic systems such as the Duffing equation, for which the trajectories are typically hard to learn. △ Less

Submitted 16 July, 2021; originally announced July 2021.

Comments: [under review]

Journal ref: Phys. Rev. E 104, 034312 (2021)

Tuning Mixed Input Hyperparameters on the Fly for Efficient Population Based AutoRL

Authors: Jack Parker-Holder, Vu Nguyen, Shaan Desai, Stephen Roberts

Abstract: Despite a series of recent successes in reinforcement learning (RL), many RL algorithms remain sensitive to hyperparameters. As such, there has recently been interest in the field of AutoRL, which seeks to automate design decisions to create more general algorithms. Recent work suggests that population based approaches may be effective AutoRL algorithms, by learning hyperparameter schedules on the… ▽ More Despite a series of recent successes in reinforcement learning (RL), many RL algorithms remain sensitive to hyperparameters. As such, there has recently been interest in the field of AutoRL, which seeks to automate design decisions to create more general algorithms. Recent work suggests that population based approaches may be effective AutoRL algorithms, by learning hyperparameter schedules on the fly. In particular, the PB2 algorithm is able to achieve strong performance in RL tasks by formulating online hyperparameter optimization as time varying GP-bandit problem, while also providing theoretical guarantees. However, PB2 is only designed to work for continuous hyperparameters, which severely limits its utility in practice. In this paper we introduce a new (provably) efficient hierarchical approach for optimizing both continuous and categorical variables, using a new time-varying bandit algorithm specifically designed for the population based training regime. We evaluate our approach on the challenging Procgen benchmark, where we show that explicitly modelling dependence between data augmentation and other hyperparameters improves generalization. △ Less

Submitted 30 June, 2021; originally announced June 2021.

Marginalising over Stationary Kernels with Bayesian Quadrature

Authors: Saad Hamid, Sebastian Schulze, Michael A. Osborne, Stephen J. Roberts

Abstract: Marginalising over families of Gaussian Process kernels produces flexible model classes with well-calibrated uncertainty estimates. Existing approaches require likelihood evaluations of many kernels, rendering them prohibitively expensive for larger datasets. We propose a Bayesian Quadrature scheme to make this marginalisation more efficient and thereby more practical. Through use of the maximum m… ▽ More Marginalising over families of Gaussian Process kernels produces flexible model classes with well-calibrated uncertainty estimates. Existing approaches require likelihood evaluations of many kernels, rendering them prohibitively expensive for larger datasets. We propose a Bayesian Quadrature scheme to make this marginalisation more efficient and thereby more practical. Through use of the maximum mean discrepancies between distributions, we define a kernel over kernels that captures invariances between Spectral Mixture (SM) Kernels. Kernel samples are selected by generalising an information-theoretic acquisition function for warped Bayesian Quadrature. We show that our framework achieves more accurate predictions with better calibrated uncertainty than state-of-the-art baselines, especially when given limited (wall-clock) time budgets. △ Less

Submitted 15 March, 2023; v1 submitted 14 June, 2021; originally announced June 2021.

Same State, Different Task: Continual Reinforcement Learning without Interference

Authors: Samuel Kessler, Jack Parker-Holder, Philip Ball, Stefan Zohren, Stephen J. Roberts

Abstract: Continual Learning (CL) considers the problem of training an agent sequentially on a set of tasks while seeking to retain performance on all previous tasks. A key challenge in CL is catastrophic forgetting, which arises when performance on a previously mastered task is reduced when learning a new task. While a variety of methods exist to combat forgetting, in some cases tasks are fundamentally inc… ▽ More Continual Learning (CL) considers the problem of training an agent sequentially on a set of tasks while seeking to retain performance on all previous tasks. A key challenge in CL is catastrophic forgetting, which arises when performance on a previously mastered task is reduced when learning a new task. While a variety of methods exist to combat forgetting, in some cases tasks are fundamentally incompatible with each other and thus cannot be learnt by a single policy. This can occur, in reinforcement learning (RL) when an agent may be rewarded for achieving different goals from the same observation. In this paper we formalize this "interference" as distinct from the problem of forgetting. We show that existing CL methods based on single neural network predictors with shared replay buffers fail in the presence of interference. Instead, we propose a simple method, OWL, to address this challenge. OWL learns a factorized policy, using shared feature extraction layers, but separate heads, each specializing on a new task. The separate heads in OWL are used to prevent interference. At test time, we formulate policy selection as a multi-armed bandit problem, and show it is possible to select the best policy for an unknown task using feedback from the environment. The use of bandit algorithms allows the OWL agent to constructively re-use different continually learnt policies at different times during an episode. We show in multiple RL environments that existing replay based CL methods fail, while OWL is able to achieve close to optimal performance when training sequentially. △ Less

Submitted 15 March, 2022; v1 submitted 5 June, 2021; originally announced June 2021.

Comments: Accepted as an oral at AAAI 2022. 17 pages and 12 figures

Can convolutional ResNets approximately preserve input distances? A frequency analysis perspective

Authors: Lewis Smith, Joost van Amersfoort, Haiwen Huang, Stephen Roberts, Yarin Gal

Abstract: ResNets constrained to be bi-Lipschitz, that is, approximately distance preserving, have been a crucial component of recently proposed techniques for deterministic uncertainty quantification in neural models. We show that theoretical justifications for recent regularisation schemes trying to enforce such a constraint suffer from a crucial flaw -- the theoretical link between the regularisation sch… ▽ More ResNets constrained to be bi-Lipschitz, that is, approximately distance preserving, have been a crucial component of recently proposed techniques for deterministic uncertainty quantification in neural models. We show that theoretical justifications for recent regularisation schemes trying to enforce such a constraint suffer from a crucial flaw -- the theoretical link between the regularisation scheme used and bi-Lipschitzness is only valid under conditions which do not hold in practice, rendering existing theory of limited use, despite the strong empirical performance of these models. We provide a theoretical explanation for the effectiveness of these regularisation schemes using a frequency analysis perspective, showing that under mild conditions these schemes will enforce a lower Lipschitz bound on the low-frequency projection of images. We then provide empirical evidence supporting our theoretical claims, and perform further experiments which demonstrate that our broader conclusions appear to hold when some of the mathematical assumptions of our proof are relaxed, corresponding to the setup used in prior work. In addition, we present a simple constructive algorithm to search for counter examples to the distance preservation condition, and discuss possible implications of our theory for future model design. △ Less

Submitted 17 June, 2021; v1 submitted 4 June, 2021; originally announced June 2021.

Comments: Main paper 10 pages including references, appendix 10 pages. 7 figures and 6 tables including appendix

Slow Momentum with Fast Reversion: A Trading Strategy Using Deep Learning and Changepoint Detection

Authors: Kieran Wood, Stephen Roberts, Stefan Zohren

Abstract: Momentum strategies are an important part of alternative investments and are at the heart of commodity trading advisors (CTAs). These strategies have, however, been found to have difficulties adjusting to rapid changes in market conditions, such as during the 2020 market crash. In particular, immediately after momentum turning points, where a trend reverses from an uptrend (downtrend) to a downtre… ▽ More Momentum strategies are an important part of alternative investments and are at the heart of commodity trading advisors (CTAs). These strategies have, however, been found to have difficulties adjusting to rapid changes in market conditions, such as during the 2020 market crash. In particular, immediately after momentum turning points, where a trend reverses from an uptrend (downtrend) to a downtrend (uptrend), time-series momentum (TSMOM) strategies are prone to making bad bets. To improve the response to regime change, we introduce a novel approach, where we insert an online changepoint detection (CPD) module into a Deep Momentum Network (DMN) [1904.04912] pipeline, which uses an LSTM deep-learning architecture to simultaneously learn both trend estimation and position sizing. Furthermore, our model is able to optimise the way in which it balances 1) a slow momentum strategy which exploits persisting trends, but does not overreact to localised price moves, and 2) a fast mean-reversion strategy regime by quickly flip** its position, then swap** it back again to exploit localised price moves. Our CPD module outputs a changepoint location and severity score, allowing our model to learn to respond to varying degrees of disequilibrium, or smaller and more localised changepoints, in a data driven manner. Back-testing our model over the period 1995-2020, the addition of the CPD module leads to an improvement in Sharpe ratio of one-third. The module is especially beneficial in periods of significant nonstationarity, and in particular, over the most recent years tested (2015-2020) the performance boost is approximately two-thirds. This is interesting as traditional momentum strategies have been underperforming in this period. △ Less

Submitted 20 December, 2021; v1 submitted 28 May, 2021; originally announced May 2021.

Comments: minor changes made to methodology to match implementation

Journal ref: The Journal of Financial Data Science Winter 2022, jfds.2021.1.081

Enhancing Cross-Sectional Currency Strategies by Context-Aware Learning to Rank with Self-Attention

Authors: Daniel Poh, Bryan Lim, Stefan Zohren, Stephen Roberts

Abstract: The performance of a cross-sectional currency strategy depends crucially on accurately ranking instruments prior to portfolio construction. While this ranking step is traditionally performed using heuristics, or by sorting the outputs produced by pointwise regression or classification techniques, strategies using Learning to Rank algorithms have recently presented themselves as competitive and via… ▽ More The performance of a cross-sectional currency strategy depends crucially on accurately ranking instruments prior to portfolio construction. While this ranking step is traditionally performed using heuristics, or by sorting the outputs produced by pointwise regression or classification techniques, strategies using Learning to Rank algorithms have recently presented themselves as competitive and viable alternatives. Although the rankers at the core of these strategies are learned globally and improve ranking accuracy on average, they ignore the differences between the distributions of asset features over the times when the portfolio is rebalanced. This flaw renders them susceptible to producing sub-optimal rankings, possibly at important periods when accuracy is actually needed the most. For example, this might happen during critical risk-off episodes, which consequently exposes the portfolio to substantial, unwanted drawdowns. We tackle this shortcoming with an analogous idea from information retrieval: that a query's top retrieved documents or the local ranking context provide vital information about the query's own characteristics, which can then be used to refine the initial ranked list. In this work, we use a context-aware Learning-to-rank model that is based on the Transformer architecture to encode top/bottom ranked assets, learn the context and exploit this information to re-rank the initial results. Backtesting on a slate of 31 currencies, our proposed methodology increases the Sharpe ratio by around 30% and significantly enhances various performance metrics. Additionally, this approach also improves the Sharpe ratio when separately conditioning on normal and risk-off market states. △ Less

Submitted 27 January, 2022; v1 submitted 20 May, 2021; originally announced May 2021.

Comments: 10 pages, 4 figures

Mesoscale simulation of woven composite design decisions

Authors: Lincoln N. Collins, Scott A. Roberts

Abstract: Characterizing the connection between material design decisions/parameters and their effective properties allows for accelerated materials development and optimization. We present a global sensitivity analysis of woven composite thermophysical properties, including density, volume fraction, thermal conductivity, specific heat, moduli, permeability, and tortuosity, predicted using mesoscale finite… ▽ More Characterizing the connection between material design decisions/parameters and their effective properties allows for accelerated materials development and optimization. We present a global sensitivity analysis of woven composite thermophysical properties, including density, volume fraction, thermal conductivity, specific heat, moduli, permeability, and tortuosity, predicted using mesoscale finite element simulations. The mesoscale simulations use microscale approximations for the tow and matrix phases. We performed Latin hypercube sampling of viable input parameter ranges, and the resulting effective property distributions are analyzed using a surrogate model to determine the correlations between material parameters and responses, interactions between properties, and finally Sobol' indices and sensitivities. We demonstrate that both constituent physical properties and the mesoscale geometry strongly influence the composite material properties. △ Less

Submitted 27 April, 2021; originally announced April 2021.

Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environment

Authors: Philip J. Ball, Cong Lu, Jack Parker-Holder, Stephen Roberts

Abstract: Reinforcement learning from large-scale offline datasets provides us with the ability to learn policies without potentially unsafe or impractical exploration. Significant progress has been made in the past few years in dealing with the challenge of correcting for differing behavior between the data collection and learned policies. However, little attention has been paid to potentially changing dyn… ▽ More Reinforcement learning from large-scale offline datasets provides us with the ability to learn policies without potentially unsafe or impractical exploration. Significant progress has been made in the past few years in dealing with the challenge of correcting for differing behavior between the data collection and learned policies. However, little attention has been paid to potentially changing dynamics when transferring a policy to the online setting, where performance can be up to 90% reduced for existing methods. In this paper we address this problem with Augmented World Models (AugWM). We augment a learned dynamics model with simple transformations that seek to capture potential changes in physical properties of the robot, leading to more robust policies. We not only train our policy in this new setting, but also provide it with the sampled augmentation as a context, allowing it to adapt to changes in the environment. At test time we learn the context in a self-supervised fashion by approximating the augmentation which corresponds to the new environment. We rigorously evaluate our approach on over 100 different changed dynamics settings, and show that this simple approach can significantly improve the zero-shot generalization of a recent state-of-the-art baseline, often achieving successful policies where the baseline fails. △ Less

Submitted 3 August, 2021; v1 submitted 12 April, 2021; originally announced April 2021.

Comments: Accepted @ ICML 2021; Spotlight @ ICLR 2021 "Self-Supervision for Reinforcement Learning Workshop"

Adversarial Robustness Guarantees for Gaussian Processes

Authors: Andrea Patane, Arno Blaas, Luca Laurenti, Luca Cardelli, Stephen Roberts, Marta Kwiatkowska

Abstract: Gaussian processes (GPs) enable principled computation of model uncertainty, making them attractive for safety-critical applications. Such scenarios demand that GP decisions are not only accurate, but also robust to perturbations. In this paper we present a framework to analyse adversarial robustness of GPs, defined as invariance of the model's decision to bounded perturbations. Given a compact su… ▽ More Gaussian processes (GPs) enable principled computation of model uncertainty, making them attractive for safety-critical applications. Such scenarios demand that GP decisions are not only accurate, but also robust to perturbations. In this paper we present a framework to analyse adversarial robustness of GPs, defined as invariance of the model's decision to bounded perturbations. Given a compact subset of the input space $T\subseteq \mathbb{R}^d$, a point $x^*$ and a GP, we provide provable guarantees of adversarial robustness of the GP by computing lower and upper bounds on its prediction range in $T$. We develop a branch-and-bound scheme to refine the bounds and show, for any $ε> 0$, that our algorithm is guaranteed to converge to values $ε$-close to the actual values in finitely many iterations. The algorithm is anytime and can handle both regression and classification tasks, with analytical formulation for most kernels used in practice. We evaluate our methods on a collection of synthetic and standard benchmark datasets, including SPAM, MNIST and FashionMNIST. We study the effect of approximate inference techniques on robustness and demonstrate how our method can be used for interpretability. Our empirical results suggest that the adversarial robustness of GPs increases with accurate posterior estimation. △ Less

Submitted 7 April, 2021; originally announced April 2021.

Comments: Submitted for publication

OffCon$^3$: What is state of the art anyway?

Authors: Philip J. Ball, Stephen J. Roberts

Abstract: Two popular approaches to model-free continuous control tasks are SAC and TD3. At first glance these approaches seem rather different; SAC aims to solve the entropy-augmented MDP by minimising the KL-divergence between a stochastic proposal policy and a hypotheical energy-basd soft Q-function policy, whereas TD3 is derived from DPG, which uses a deterministic policy to perform policy gradient asce… ▽ More Two popular approaches to model-free continuous control tasks are SAC and TD3. At first glance these approaches seem rather different; SAC aims to solve the entropy-augmented MDP by minimising the KL-divergence between a stochastic proposal policy and a hypotheical energy-basd soft Q-function policy, whereas TD3 is derived from DPG, which uses a deterministic policy to perform policy gradient ascent along the value function. In reality, both approaches are remarkably similar, and belong to a family of approaches we call `Off-Policy Continuous Generalized Policy Iteration'. This illuminates their similar performance in most continuous control benchmarks, and indeed when hyperparameters are matched, their performance can be statistically indistinguishable. To further remove any difference due to implementation, we provide OffCon$^3$ (Off-Policy Continuous Control: Consolidated), a code base featuring state-of-the-art versions of both algorithms. △ Less

Submitted 14 March, 2021; v1 submitted 27 January, 2021; originally announced January 2021.

The Effect of Prior Lipschitz Continuity on the Adversarial Robustness of Bayesian Neural Networks

Authors: Arno Blaas, Stephen J. Roberts

Abstract: It is desirable, and often a necessity, for machine learning models to be robust against adversarial attacks. This is particularly true for Bayesian models, as they are well-suited for safety-critical applications, in which adversarial attacks can have catastrophic outcomes. In this work, we take a deeper look at the adversarial robustness of Bayesian Neural Networks (BNNs). In particular, we cons… ▽ More It is desirable, and often a necessity, for machine learning models to be robust against adversarial attacks. This is particularly true for Bayesian models, as they are well-suited for safety-critical applications, in which adversarial attacks can have catastrophic outcomes. In this work, we take a deeper look at the adversarial robustness of Bayesian Neural Networks (BNNs). In particular, we consider whether the adversarial robustness of a BNN can be increased by model choices, particularly the Lipschitz continuity induced by the prior. Conducting in-depth analysis on the case of i.i.d., zero-mean Gaussian priors and posteriors approximated via mean-field variational inference, we find evidence that adversarial robustness is indeed sensitive to the prior variance. △ Less

Submitted 7 January, 2021; originally announced January 2021.

Comments: 4 pages, 2 tables, AAAI 2021 Workshop Towards Robust, Secure and Efficient Machine Learning

Quantifying the unknown impact of segmentation uncertainty on image-based simulations

Authors: Michael C. Krygier, Tyler LaBonte, Carianne Martinez, Chance Norris, Krish Sharma, Lincoln N. Collins, Partha P. Mukherjee, Scott A. Roberts

Abstract: Image-based simulation, the use of 3D images to calculate physical quantities, fundamentally relies on image segmentation to create the computational geometry. However, this process introduces image segmentation uncertainty because there is a variety of different segmentation tools (both manual and machine-learning-based) that will each produce a unique and valid segmentation. First, we demonstrat… ▽ More Image-based simulation, the use of 3D images to calculate physical quantities, fundamentally relies on image segmentation to create the computational geometry. However, this process introduces image segmentation uncertainty because there is a variety of different segmentation tools (both manual and machine-learning-based) that will each produce a unique and valid segmentation. First, we demonstrate that these variations propagate into the physics simulations, compromising the resulting physics quantities. Second, we propose a general framework for rapidly quantifying segmentation uncertainty. Through the creation and sampling of segmentation uncertainty probability maps, we systematically and objectively create uncertainty distributions of the physics quantities. We show that physics quantity uncertainty distributions can follow a Normal distribution, but, in more complicated physics simulations, the resulting uncertainty distribution can be both nonintuitive and surprisingly nontrivial. We also establish that simply bounding the uncertainty can fail in situations that are sensitive to image segmentation. While our work does not eliminate segmentation uncertainty, it makes visible the previously unrecognized range of uncertainty currently plaguing image-based simulation, enabling more credible simulations. △ Less

Submitted 9 September, 2021; v1 submitted 17 December, 2020; originally announced December 2020.

Journal ref: Nature Communications 12, 5414 (2021)

Building Cross-Sectional Systematic Strategies By Learning to Rank

Authors: Daniel Poh, Bryan Lim, Stefan Zohren, Stephen Roberts

Abstract: The success of a cross-sectional systematic strategy depends critically on accurately ranking assets prior to portfolio construction. Contemporary techniques perform this ranking step either with simple heuristics or by sorting outputs from standard regression or classification models, which have been demonstrated to be sub-optimal for ranking in other domains (e.g. information retrieval). To addr… ▽ More The success of a cross-sectional systematic strategy depends critically on accurately ranking assets prior to portfolio construction. Contemporary techniques perform this ranking step either with simple heuristics or by sorting outputs from standard regression or classification models, which have been demonstrated to be sub-optimal for ranking in other domains (e.g. information retrieval). To address this deficiency, we propose a framework to enhance cross-sectional portfolios by incorporating learning-to-rank algorithms, which lead to improvements of ranking accuracy by learning pairwise and listwise structures across instruments. Using cross-sectional momentum as a demonstrative case study, we show that the use of modern machine learning ranking algorithms can substantially improve the trading performance of cross-sectional strategies -- providing approximately threefold boosting of Sharpe Ratios compared to traditional approaches. △ Less

Submitted 13 December, 2020; originally announced December 2020.

Comments: 12 pages, 3 figures

Towards real-time finite-strain anisotropic thermo-visco-elastodynamic analysis of soft tissues for thermal ablative therapy

Authors: **ao Zhang, Remi Jacob Lay, Stuart K. Roberts, Sunita Chauhan

Abstract: Accurate and efficient prediction of soft tissue temperatures is essential to computer-assisted treatment systems for thermal ablation. It can be used to predict tissue temperatures and ablation volumes for personalised treatment planning and image-guided intervention. Numerically, it requires full nonlinear modelling of the coupled computational bioheat transfer and biomechanics, and efficient so… ▽ More Accurate and efficient prediction of soft tissue temperatures is essential to computer-assisted treatment systems for thermal ablation. It can be used to predict tissue temperatures and ablation volumes for personalised treatment planning and image-guided intervention. Numerically, it requires full nonlinear modelling of the coupled computational bioheat transfer and biomechanics, and efficient solution procedures; however, existing studies considered the bioheat analysis alone or the coupled linear analysis, without the fully coupled nonlinear analysis. We present a coupled thermo-visco-hyperelastic finite element algorithm, based on finite-strain thermoelasticity and total Lagrangian explicit dynamics. It considers the coupled nonlinear analysis of (i) bioheat transfer under soft tissue deformations and (ii) soft tissue deformations due to thermal expansion/shrinkage. The presented method accounts for anisotropic, finite-strain, temperature-dependent, thermal, and viscoelastic behaviours of soft tissues, and it is implemented using GPU acceleration for real-time computation. We also demonstrate the translational benefits of the presented method for clinical applications using a simulation of thermal ablation in the liver. The key advantage of the presented method is that it enables full nonlinear modelling of the anisotropic, finite-strain, temperature-dependent, thermal, and viscoelastic behaviours of soft tissues, instead of linear elastic, linear viscoelastic, and thermal-only modelling in the existing methods. It also provides high computational speeds for computer-assisted treatment systems towards enabling the operator to simulate thermal ablation accurately and visualise tissue temperatures and ablation zones immediately. △ Less

Submitted 31 December, 2021; v1 submitted 22 September, 2020; originally announced September 2020.

Comments: Submitted to Computer Methods and Programs in Biomedicine

Journal ref: Computer Methods and Programs in Biomedicine, vol. 198, pp. 105789, 2021

SafePILCO: a software tool for safe and data-efficient policy synthesis

Authors: Kyriakos Polymenakos, Nikitas Rontsis, Alessandro Abate, Stephen Roberts

Abstract: SafePILCO is a software tool for safe and data-efficient policy search with reinforcement learning. It extends the known PILCO algorithm, originally written in MATLAB, to support safe learning. We provide a Python implementation and leverage existing libraries that allow the codebase to remain short and modular, which is appropriate for wider use by the verification, reinforcement learning, and co… ▽ More SafePILCO is a software tool for safe and data-efficient policy search with reinforcement learning. It extends the known PILCO algorithm, originally written in MATLAB, to support safe learning. We provide a Python implementation and leverage existing libraries that allow the codebase to remain short and modular, which is appropriate for wider use by the verification, reinforcement learning, and control communities. △ Less

Submitted 7 August, 2020; originally announced August 2020.

Comments: Shorter Version published as a software tool demonstration at QEST 2020