Showing 1–50 of 81 results for author: Lange, R

Behaviour Distillation

Authors: Andrei Lupu, Chris Lu, Jarek Liesen, Robert Tjarko Lange, Jakob Foerster

Abstract: Dataset distillation aims to condense large datasets into a small number of synthetic examples that can be used as drop-in replacements when training new models. It has applications to interpretability, neural architecture search, privacy, and continual learning. Despite strong successes in supervised domains, such methods have not yet been extended to reinforcement learning, where the lack of a f… ▽ More Dataset distillation aims to condense large datasets into a small number of synthetic examples that can be used as drop-in replacements when training new models. It has applications to interpretability, neural architecture search, privacy, and continual learning. Despite strong successes in supervised domains, such methods have not yet been extended to reinforcement learning, where the lack of a fixed dataset renders most distillation methods unusable. Filling the gap, we formalize behaviour distillation, a setting that aims to discover and then condense the information required for training an expert policy into a synthetic dataset of state-action pairs, without access to expert data. We then introduce Hallucinating Datasets with Evolution Strategies (HaDES), a method for behaviour distillation that can discover datasets of just four state-action pairs which, under supervised learning, train agents to competitive performance levels in continuous control tasks. We show that these datasets generalize out of distribution to training policies with a wide range of architectures and hyperparameters. We also demonstrate application to a downstream task, namely training multi-task agents in a zero-shot fashion. Beyond behaviour distillation, HaDES provides significant improvements in neuroevolution for RL over previous approaches and achieves SoTA results on one standard supervised dataset distillation task. Finally, we show that visualizing the synthetic datasets can provide human-interpretable task insights. △ Less

Submitted 21 June, 2024; originally announced June 2024.

Comments: Published as a conference paper at ICLR 2024

Discovering Minimal Reinforcement Learning Environments

Authors: Jarek Liesen, Chris Lu, Andrei Lupu, Jakob N. Foerster, Henning Sprekeler, Robert T. Lange

Abstract: Reinforcement learning (RL) agents are commonly trained and evaluated in the same environment. In contrast, humans often train in a specialized environment before being evaluated, such as studying a book before taking an exam. The potential of such specialized training environments is still vastly underexplored, despite their capacity to dramatically speed up training. The framework of synthetic… ▽ More Reinforcement learning (RL) agents are commonly trained and evaluated in the same environment. In contrast, humans often train in a specialized environment before being evaluated, such as studying a book before taking an exam. The potential of such specialized training environments is still vastly underexplored, despite their capacity to dramatically speed up training. The framework of synthetic environments takes a first step in this direction by meta-learning neural network-based Markov decision processes (MDPs). The initial approach was limited to toy problems and produced environments that did not transfer to unseen RL algorithms. We extend this approach in three ways: Firstly, we modify the meta-learning algorithm to discover environments invariant towards hyperparameter configurations and learning algorithms. Secondly, by leveraging hardware parallelism and introducing a curriculum on an agent's evaluation episode horizon, we can achieve competitive results on several challenging continuous control problems. Thirdly, we surprisingly find that contextual bandits enable training RL agents that transfer well to their evaluation environment, even if it is a complex MDP. Hence, we set up our experiments to train synthetic contextual bandits, which perform on par with synthetic MDPs, yield additional insights into the evaluation environment, and can speed up downstream applications. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Comments: 10 pages, 7 figures

Discovering Preference Optimization Algorithms with and for Large Language Models

Authors: Chris Lu, Samuel Holt, Claudio Fanconi, Alex J. Chan, Jakob Foerster, Mihaela van der Schaar, Robert Tjarko Lange

Abstract: Offline preference optimization is a key method for enhancing and controlling the quality of Large Language Model (LLM) outputs. Typically, preference optimization is approached as an offline supervised learning task using manually-crafted convex loss functions. While these methods are based on theoretical insights, they are inherently constrained by human creativity, so the large search space of… ▽ More Offline preference optimization is a key method for enhancing and controlling the quality of Large Language Model (LLM) outputs. Typically, preference optimization is approached as an offline supervised learning task using manually-crafted convex loss functions. While these methods are based on theoretical insights, they are inherently constrained by human creativity, so the large search space of possible loss functions remains under explored. We address this by performing LLM-driven objective discovery to automatically discover new state-of-the-art preference optimization algorithms without (expert) human intervention. Specifically, we iteratively prompt an LLM to propose and implement new preference optimization loss functions based on previously-evaluated performance metrics. This process leads to the discovery of previously-unknown and performant preference optimization algorithms. The best performing of these we call Discovered Preference Optimization (DiscoPOP), a novel algorithm that adaptively blends logistic and exponential losses. Experiments demonstrate the state-of-the-art performance of DiscoPOP and its successful transfer to held-out tasks. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Short and simple introduction to Bellman filtering and smoothing

Authors: Rutger-Jan Lange

Abstract: Based on Bellman's dynamic-programming principle, Lange (2024) presents an approximate method for filtering, smoothing and parameter estimation for possibly non-linear and/or non-Gaussian state-space models. While the approach applies more generally, this pedagogical note highlights the main results in the case where (i) the state transition remains linear and Gaussian while (ii) the observation d… ▽ More Based on Bellman's dynamic-programming principle, Lange (2024) presents an approximate method for filtering, smoothing and parameter estimation for possibly non-linear and/or non-Gaussian state-space models. While the approach applies more generally, this pedagogical note highlights the main results in the case where (i) the state transition remains linear and Gaussian while (ii) the observation density is log-concave and sufficiently smooth in the state variable. I demonstrate how Kalman's (1960) filter and Rauch et al.'s (1965) smoother can be obtained as special cases within the proposed framework. The main aim is to present non-experts (and my own students) with an accessible introduction, enabling them to implement the proposed methods. △ Less

Submitted 21 May, 2024; originally announced May 2024.

Comments: 10 pages

Position: Leverage Foundational Models for Black-Box Optimization

Authors: Xingyou Song, Yingtao Tian, Robert Tjarko Lange, Chansoo Lee, Yu** Tang, Yutian Chen

Abstract: Undeniably, Large Language Models (LLMs) have stirred an extraordinary wave of innovation in the machine learning research domain, resulting in substantial impact across diverse fields such as reinforcement learning, robotics, and computer vision. Their incorporation has been rapid and transformative, marking a significant paradigm shift in the field of machine learning research. However, the fiel… ▽ More Undeniably, Large Language Models (LLMs) have stirred an extraordinary wave of innovation in the machine learning research domain, resulting in substantial impact across diverse fields such as reinforcement learning, robotics, and computer vision. Their incorporation has been rapid and transformative, marking a significant paradigm shift in the field of machine learning research. However, the field of experimental design, grounded on black-box optimization, has been much less affected by such a paradigm shift, even though integrating LLMs with optimization presents a unique landscape ripe for exploration. In this position paper, we frame the field of black-box optimization around sequence-based foundation models and organize their relationship with previous literature. We discuss the most promising ways foundational language models can revolutionize optimization, which include harnessing the vast wealth of information encapsulated in free-form text to enrich task comprehension, utilizing highly flexible sequence models such as Transformers to engineer superior optimization strategies, and enhancing performance prediction over previously unseen search spaces. △ Less

Submitted 9 May, 2024; v1 submitted 6 May, 2024; originally announced May 2024.

Comments: International Conference on Machine Learning (ICML) 2024

Evolution Transformer: In-Context Evolutionary Optimization

Authors: Robert Tjarko Lange, Yingtao Tian, Yu** Tang

Abstract: Evolutionary optimization algorithms are often derived from loose biological analogies and struggle to leverage information obtained during the sequential course of optimization. An alternative promising approach is to leverage data and directly discover powerful optimization principles via meta-optimization. In this work, we follow such a paradigm and introduce Evolution Transformer, a causal Tra… ▽ More Evolutionary optimization algorithms are often derived from loose biological analogies and struggle to leverage information obtained during the sequential course of optimization. An alternative promising approach is to leverage data and directly discover powerful optimization principles via meta-optimization. In this work, we follow such a paradigm and introduce Evolution Transformer, a causal Transformer architecture, which can flexibly characterize a family of Evolution Strategies. Given a trajectory of evaluations and search distribution statistics, Evolution Transformer outputs a performance-improving update to the search distribution. The architecture imposes a set of suitable inductive biases, i.e. the invariance of the distribution update to the order of population members within a generation and equivariance to the order of the search dimensions. We train the model weights using Evolutionary Algorithm Distillation, a technique for supervised optimization of sequence models using teacher algorithm trajectories. The resulting model exhibits strong in-context optimization performance and shows strong generalization capabilities to otherwise challenging neuroevolution tasks. We analyze the resulting properties of the Evolution Transformer and propose a technique to fully self-referentially train the Evolution Transformer, starting from a random initialization and bootstrap** its own learning progress. We provide an open source implementation under https://github.com/RobertTLange/evosax. △ Less

Submitted 5 March, 2024; originally announced March 2024.

Large Language Models As Evolution Strategies

Authors: Robert Tjarko Lange, Yingtao Tian, Yu** Tang

Abstract: Large Transformer models are capable of implementing a plethora of so-called in-context learning algorithms. These include gradient descent, classification, sequence completion, transformation, and improvement. In this work, we investigate whether large language models (LLMs), which never explicitly encountered the task of black-box optimization, are in principle capable of implementing evolutiona… ▽ More Large Transformer models are capable of implementing a plethora of so-called in-context learning algorithms. These include gradient descent, classification, sequence completion, transformation, and improvement. In this work, we investigate whether large language models (LLMs), which never explicitly encountered the task of black-box optimization, are in principle capable of implementing evolutionary optimization algorithms. While previous works have solely focused on language-based task specification, we move forward and focus on the zero-shot application of LLMs to black-box optimization. We introduce a novel prompting strategy, consisting of least-to-most sorting of discretized population members and querying the LLM to propose an improvement to the mean statistic, i.e. perform a type of black-box recombination operation. Empirically, we find that our setup allows the user to obtain an LLM-based evolution strategy, which we call `EvoLLM', that robustly outperforms baseline algorithms such as random search and Gaussian Hill Climbing on synthetic BBOB functions as well as small neuroevolution tasks. Hence, LLMs can act as `plug-in' in-context recombination operators. We provide several comparative studies of the LLM's model size, prompt strategy, and context construction. Finally, we show that one can flexibly improve EvoLLM's performance by providing teacher algorithm information via instruction fine-tuning on previously collected teacher optimization trajectories. △ Less

Submitted 28 February, 2024; originally announced February 2024.

Comments: 11 pages, 14 figures

Discovering Temporally-Aware Reinforcement Learning Algorithms

Authors: Matthew Thomas Jackson, Chris Lu, Louis Kirsch, Robert Tjarko Lange, Shimon Whiteson, Jakob Nicolaus Foerster

Abstract: Recent advancements in meta-learning have enabled the automatic discovery of novel reinforcement learning algorithms parameterized by surrogate objective functions. To improve upon manually designed algorithms, the parameterization of this learned objective function must be expressive enough to represent novel principles of learning (instead of merely recovering already established ones) while sti… ▽ More Recent advancements in meta-learning have enabled the automatic discovery of novel reinforcement learning algorithms parameterized by surrogate objective functions. To improve upon manually designed algorithms, the parameterization of this learned objective function must be expressive enough to represent novel principles of learning (instead of merely recovering already established ones) while still generalizing to a wide range of settings outside of its meta-training distribution. However, existing methods focus on discovering objective functions that, like many widely used objective functions in reinforcement learning, do not take into account the total number of steps allowed for training, or "training horizon". In contrast, humans use a plethora of different learning objectives across the course of acquiring a new ability. For instance, students may alter their studying techniques based on the proximity to exam deadlines and their self-assessed capabilities. This paper contends that ignoring the optimization time horizon significantly restricts the expressive potential of discovered learning algorithms. We propose a simple augmentation to two existing objective discovery approaches that allows the discovered algorithm to dynamically update its objective function throughout the agent's training procedure, resulting in expressive schedules and increased generalization across different training horizons. In the process, we find that commonly used meta-gradient approaches fail to discover such adaptive objective functions while evolution strategies discover highly dynamic learning rules. We demonstrate the effectiveness of our approach on a wide range of tasks and analyze the resulting learned algorithms, which we find effectively balance exploration and exploitation by modifying the structure of their learning rules throughout the agent's lifetime. △ Less

Submitted 8 February, 2024; originally announced February 2024.

Comments: Published at ICLR 2024

Multimodality in Group Communication Research

Authors: Robin Lange, Brooke Foucault Welles, Gyanendra Sharma, Richard J. Radke, Javier O. Garcia, Christoph Riedl

Abstract: Team interactions are often multisensory, requiring members to pick up on verbal, visual, spatial and body language cues. Multimodal research, research that captures multiple modes of communication such as audio and visual signals, is therefore integral to understanding these multisensory group communication processes. This type of research has gained traction in biomedical engineering and neurosc… ▽ More Team interactions are often multisensory, requiring members to pick up on verbal, visual, spatial and body language cues. Multimodal research, research that captures multiple modes of communication such as audio and visual signals, is therefore integral to understanding these multisensory group communication processes. This type of research has gained traction in biomedical engineering and neuroscience, but it is unclear the extent to which communication and management researchers conduct multimodal research. Our study finds that despite its' utility, multimodal research is underutilized in the communication and management literature's. This paper then covers introductory guidelines for creating new multimodal research including considerations for sensors, data integration and ethical considerations. △ Less

Submitted 26 January, 2024; originally announced January 2024.

Comments: 27 pages, 3 figures

An Industrial Perspective on Multi-Agent Decision Making for Interoperable Robot Navigation following the VDA5050 Standard

Authors: Niels van Duijkeren, Luigi Palmieri, Ralph Lange, Alexander Kleiner

Abstract: This paper provides a perspective on the literature and current challenges in Multi-Agent Systems for interoperable robot navigation in industry. The focus is on the multi-agent decision stack for Autonomous Mobile Robots operating in mixed environments with humans, manually driven vehicles, and legacy Automated Guided Vehicles. We provide typical characteristics of such Multi-Agent Systems observ… ▽ More This paper provides a perspective on the literature and current challenges in Multi-Agent Systems for interoperable robot navigation in industry. The focus is on the multi-agent decision stack for Autonomous Mobile Robots operating in mixed environments with humans, manually driven vehicles, and legacy Automated Guided Vehicles. We provide typical characteristics of such Multi-Agent Systems observed today and how these are expected to change on the short term due to the new standard VDA5050 and the interoperability framework OpenRMF. We present recent changes in fleet management standards and the role of open middleware frameworks like ROS2 reaching industrial-grade quality. Approaches to increase the robustness and performance of multi-robot navigation systems for transportation are discussed, and research opportunities are derived. △ Less

Submitted 24 November, 2023; originally announced November 2023.

Comments: 6 pages, 2 figures, presented in the Decision Making in Multi-Agent Systems Workshop at IROS2022

JaxMARL: Multi-Agent RL Environments in JAX

Authors: Alexander Rutherford, Benjamin Ellis, Matteo Gallici, Jonathan Cook, Andrei Lupu, Gardar Ingvarsson, Timon Willi, Akbir Khan, Christian Schroeder de Witt, Alexandra Souly, Saptarashmi Bandyopadhyay, Mikayel Samvelyan, Minqi Jiang, Robert Tjarko Lange, Shimon Whiteson, Bruno Lacerda, Nick Hawes, Tim Rocktaschel, Chris Lu, Jakob Nicolaus Foerster

Abstract: Benchmarks play an important role in the development of machine learning algorithms. For example, research in reinforcement learning (RL) has been heavily influenced by available environments and benchmarks. However, RL environments are traditionally run on the CPU, limiting their scalability with typical academic compute. Recent advancements in JAX have enabled the wider use of hardware accelerat… ▽ More Benchmarks play an important role in the development of machine learning algorithms. For example, research in reinforcement learning (RL) has been heavily influenced by available environments and benchmarks. However, RL environments are traditionally run on the CPU, limiting their scalability with typical academic compute. Recent advancements in JAX have enabled the wider use of hardware acceleration to overcome these computational hurdles, enabling massively parallel RL training pipelines and environments. This is particularly useful for multi-agent reinforcement learning (MARL) research. First of all, multiple agents must be considered at each environment step, adding computational burden, and secondly, the sample complexity is increased due to non-stationarity, decentralised partial observability, or other MARL challenges. In this paper, we present JaxMARL, the first open-source code base that combines ease-of-use with GPU enabled efficiency, and supports a large number of commonly used MARL environments as well as popular baseline algorithms. When considering wall clock time, our experiments show that per-run our JAX-based training pipeline is up to 12500x faster than existing approaches. This enables efficient and thorough evaluations, with the potential to alleviate the evaluation crisis of the field. We also introduce and benchmark SMAX, a vectorised, simplified version of the popular StarCraft Multi-Agent Challenge, which removes the need to run the StarCraft II game engine. This not only enables GPU acceleration, but also provides a more flexible MARL environment, unlocking the potential for self-play, meta-learning, and other future applications in MARL. We provide code at https://github.com/flairox/jaxmarl. △ Less

Submitted 19 December, 2023; v1 submitted 16 November, 2023; originally announced November 2023.

NeuroEvoBench: Benchmarking Evolutionary Optimizers for Deep Learning Applications

Authors: Robert Tjarko Lange, Yu** Tang, Yingtao Tian

Abstract: Recently, the Deep Learning community has become interested in evolutionary optimization (EO) as a means to address hard optimization problems, e.g. meta-learning through long inner loop unrolls or optimizing non-differentiable operators. One core reason for this trend has been the recent innovation in hardware acceleration and compatible software - making distributed population evaluations much e… ▽ More Recently, the Deep Learning community has become interested in evolutionary optimization (EO) as a means to address hard optimization problems, e.g. meta-learning through long inner loop unrolls or optimizing non-differentiable operators. One core reason for this trend has been the recent innovation in hardware acceleration and compatible software - making distributed population evaluations much easier than before. Unlike for gradient descent-based methods though, there is a lack of hyperparameter understanding and best practices for EO - arguably due to severely less 'graduate student descent' and benchmarking being performed for EO methods. Additionally, classical benchmarks from the evolutionary community provide few practical insights for Deep Learning applications. This poses challenges for newcomers to hardware-accelerated EO and hinders significant adoption. Hence, we establish a new benchmark of EO methods (NeuroEvoBench) tailored toward Deep Learning applications and exhaustively evaluate traditional and meta-learned EO. We investigate core scientific questions including resource allocation, fitness sha**, normalization, regularization & scalability of EO. The benchmark is open-sourced at https://github.com/neuroevobench/neuroevobench under Apache-2.0 license. △ Less

Submitted 4 November, 2023; originally announced November 2023.

Comments: 22 pages, 20 figures, 37th Conference on Neural Information Processing Systems (NeurIPS 2023) Track on Datasets and Benchmarks

Evaluating the performance of ionic liquid coatings for mitigation of spacecraft surface charges

Authors: M. Wendt, R. Lange, F. Dorn, J. Berdermann, I. Barke, S. Speller

Abstract: To reduce the impact of charging effects on satellites, cheap and lightweight conductive coatings are desirable. We mimic space-like charging environments in ultra-high vacuum (UHV) chambers during deposition of charges via the electron beam of a scanning electron microscope (SEM). We use the charge induced signatures in SEM images of a thin ionic liquid (IL) film on insulating surfaces such as gl… ▽ More To reduce the impact of charging effects on satellites, cheap and lightweight conductive coatings are desirable. We mimic space-like charging environments in ultra-high vacuum (UHV) chambers during deposition of charges via the electron beam of a scanning electron microscope (SEM). We use the charge induced signatures in SEM images of a thin ionic liquid (IL) film on insulating surfaces such as glass, to assess the general performance of such coatings. In order to get a reference structure in SEM, the samples were structured by nanosphere lithography and coated with IL. The IL film (we choose BMP DCA, due to its beneficial physical properties) was applied ex situ and a thickness of 10 to 30 nm was determined by reflectometry. Such an IL film is stable under vacuum conditions. It would also only lead to additional mass of below 20 mg/m$^2$. At about 5 A/m$^2 \approx 3\cdot10^{19}$ e/(s$\cdot$m$^2$), a typical sample charging rate in SEM, imaging is possible with no noticeable contrast changes over many hours; this electron current density is already 6 orders of magnitudes higher than "worst case geosynchronous environments" of $3\cdot10^{-6}$ A/m$^2$. Measurements of the surface potential are used for further insights in the reaction of IL films to the electron beam of a SEM. Participating mechanisms such as polarization or reorientation will are discussed. △ Less

Submitted 4 September, 2023; originally announced September 2023.

Comments: Submitted to Proceedings of the 14th IAA Symposium on Small Satellites for Earth System Observation

A large language model-assisted education tool to provide feedback on open-ended responses

Authors: Jordan K. Matelsky, Felipe Parodi, Tony Liu, Richard D. Lange, Konrad P. Kording

Abstract: Open-ended questions are a favored tool among instructors for assessing student understanding and encouraging critical exploration of course material. Providing feedback for such responses is a time-consuming task that can lead to overwhelmed instructors and decreased feedback quality. Many instructors resort to simpler question formats, like multiple-choice questions, which provide immediate feed… ▽ More Open-ended questions are a favored tool among instructors for assessing student understanding and encouraging critical exploration of course material. Providing feedback for such responses is a time-consuming task that can lead to overwhelmed instructors and decreased feedback quality. Many instructors resort to simpler question formats, like multiple-choice questions, which provide immediate feedback but at the expense of personalized and insightful comments. Here, we present a tool that uses large language models (LLMs), guided by instructor-defined criteria, to automate responses to open-ended questions. Our tool delivers rapid personalized feedback, enabling students to quickly test their knowledge and identify areas for improvement. We provide open-source reference implementations both as a web application and as a Jupyter Notebook widget that can be used with instructional coding or math notebooks. With instructor guidance, LLMs hold promise to enhance student learning outcomes and elevate instructional methodologies. △ Less

Submitted 25 July, 2023; originally announced August 2023.

Lottery Tickets in Evolutionary Optimization: On Sparse Backpropagation-Free Trainability

Authors: Robert Tjarko Lange, Henning Sprekeler

Abstract: Is the lottery ticket phenomenon an idiosyncrasy of gradient-based training or does it generalize to evolutionary optimization? In this paper we establish the existence of highly sparse trainable initializations for evolution strategies (ES) and characterize qualitative differences compared to gradient descent (GD)-based sparse training. We introduce a novel signal-to-noise iterative pruning proce… ▽ More Is the lottery ticket phenomenon an idiosyncrasy of gradient-based training or does it generalize to evolutionary optimization? In this paper we establish the existence of highly sparse trainable initializations for evolution strategies (ES) and characterize qualitative differences compared to gradient descent (GD)-based sparse training. We introduce a novel signal-to-noise iterative pruning procedure, which incorporates loss curvature information into the network pruning step. This can enable the discovery of even sparser trainable network initializations when using black-box evolution as compared to GD-based optimization. Furthermore, we find that these initializations encode an inductive bias, which transfers across different ES, related tasks and even to GD-based training. Finally, we compare the local optima resulting from the different optimization paradigms and sparsity levels. In contrast to GD, ES explore diverse and flat local optima and do not preserve linear mode connectivity across sparsity levels and independent runs. The results highlight qualitative differences between evolution and gradient-based learning dynamics, which can be uncovered by the study of iterative pruning procedures. △ Less

Submitted 31 May, 2023; originally announced June 2023.

Comments: 13 pages, 11 figures, International Conference on Machine Learning (ICML) 2023

Discovering Attention-Based Genetic Algorithms via Meta-Black-Box Optimization

Authors: Robert Tjarko Lange, Tom Schaul, Yutian Chen, Chris Lu, Tom Zahavy, Valentin Dalibard, Sebastian Flennerhag

Abstract: Genetic algorithms constitute a family of black-box optimization algorithms, which take inspiration from the principles of biological evolution. While they provide a general-purpose tool for optimization, their particular instantiations can be heuristic and motivated by loose biological intuition. In this work we explore a fundamentally different approach: Given a sufficiently flexible parametriza… ▽ More Genetic algorithms constitute a family of black-box optimization algorithms, which take inspiration from the principles of biological evolution. While they provide a general-purpose tool for optimization, their particular instantiations can be heuristic and motivated by loose biological intuition. In this work we explore a fundamentally different approach: Given a sufficiently flexible parametrization of the genetic operators, we discover entirely new genetic algorithms in a data-driven fashion. More specifically, we parametrize selection and mutation rate adaptation as cross- and self-attention modules and use Meta-Black-Box-Optimization to evolve their parameters on a set of diverse optimization tasks. The resulting Learned Genetic Algorithm outperforms state-of-the-art adaptive baseline genetic algorithms and generalizes far beyond its meta-training settings. The learned algorithm can be applied to previously unseen optimization problems, search dimensions & evaluation budgets. We conduct extensive analysis of the discovered operators and provide ablation experiments, which highlight the benefits of flexible module parametrization and the ability to transfer (`plug-in') the learned operators to conventional genetic algorithms. △ Less

Submitted 8 April, 2023; originally announced April 2023.

Comments: 14 pages, 31 figures

upstreamFoam: an OpenFOAM-based solver for heterogeneous porous media at different scales

Authors: Roberto Lange, Gabriel M. Magalhães, Franciane F. Rocha, Pedro V. S. Coimbra, Jovani L. Favero, Rodrigo A. C. Dias, Antonio O. S. Moraes, Mateus P. Schwalbert

Abstract: A new OpenFOAM application to simulate multiphase flows in porous media is formulated and tested. The proposed solver combines the Eulerian multi-fluid formulation for a system of phase fractions with Darcy's law for flows through porous media. It is based on the multiphaseEulerFoam and includes models for reservoir simulation of the porousMultiphaseFoam, taking advantage of the most recent techno… ▽ More A new OpenFOAM application to simulate multiphase flows in porous media is formulated and tested. The proposed solver combines the Eulerian multi-fluid formulation for a system of phase fractions with Darcy's law for flows through porous media. It is based on the multiphaseEulerFoam and includes models for reservoir simulation of the porousMultiphaseFoam, taking advantage of the most recent technologies developed for these well-established solvers. With such an innovative combination, we are able to simulate a system of any number of compressible phase fractions in reservoirs that rely on specialized models for relative permeability, capillary pressure, and time step selection. We successfully validate the solver for classical problems with analytical, semi-analytical, and reference solutions. A wide range of flows in porous media has been studied, demonstrating the potential of the solver to approximate complex multiphase problems. △ Less

Submitted 26 June, 2023; v1 submitted 6 April, 2023; originally announced April 2023.

Comments: 30 pages, 14 figures

The Southern-sky MWA Rapid Two-metre (SMART) pulsar survey -- II. Survey status, pulsar census, and first pulsar discoveries

Authors: N. D. R. Bhat, N. A. Swainston, S. J. McSweeney, M. Xue, B. W. Meyers, S. Kudale, S. Dai, S. E. Tremblay, W. van Straten, R. M. Shannon, K. R. Smith, M. Sokolowski, S. M. Ord, G. Sleap, A. Williams, P. J. Hancock, R. Lange, J. Tocknell, M. Johnston-Hollitt, D. L. Kaplan, S. J. Tingay, M. Walker

Abstract: In Paper I, we presented an overview of the Southern-sky MWA Rapid Two-metre (SMART) survey, including the survey design and search pipeline. While the combination of MWA's large field-of-view and the voltage capture system brings a survey speed of ~450 square degrees per hour, the survey progression relies on the availability of compact configuration of the Phase II array. Over the past few years… ▽ More In Paper I, we presented an overview of the Southern-sky MWA Rapid Two-metre (SMART) survey, including the survey design and search pipeline. While the combination of MWA's large field-of-view and the voltage capture system brings a survey speed of ~450 square degrees per hour, the survey progression relies on the availability of compact configuration of the Phase II array. Over the past few years, by taking advantage of multiple windows of opportunity when the compact configuration was available, we have advanced the survey to 75% completion. To date, about 10% of the data collected thus far have been processed for a first-pass search, where 10 minutes of observation is processed for dispersion measures out to 250 ${\rm pc\,cm^{-3}}$, to realise a shallow survey for long-period pulsars. The ongoing analysis has led to two new pulsar discoveries, as well as an independent discovery and a rediscovery of a previously incorrectly characterised pulsar, all from ~3% of the data for which candidate scrutiny is completed. Here we describe the strategies for further detailed follow-up including improved sky localisation and convergence to timing solution, and illustrate them using example pulsar discoveries. The processing has also led to re-detection of 120 pulsars in the SMART observing band, bringing the total number of pulsars detected to date with the MWA to 180, and these are used to assess the search sensitivity of current processing pipelines. The planned second-pass (deep survey) processing is expected to yield a three-fold increase in sensitivity for long-period pulsars, and a substantial improvement to millisecond pulsars by adopting optimal de-dispersion plans. The SMART survey will complement the highly successful Parkes High Time Resolution Universe survey at 1.2-1.5 GHz, and inform future large survey efforts such as those planned with the low-frequency Square Kilometre Array. △ Less

Submitted 23 February, 2023; originally announced February 2023.

Comments: 22 pages, 9 figures, 7 tables, Accepted for publication in PASA

The Southern-sky MWA Rapid Two-metre (SMART) pulsar survey -- I. Survey design and processing pipeline

Authors: N. D. R. Bhat, N. A. Swainston, S. J. McSweeney, M. Xue, B. W. Meyers, S. Kudale, S. Dai, S. E. Tremblay, W. van Straten, R. M. Shannon, K. R. Smith, M. Sokolowski, S. M. Ord, G. Sleap, A. Williams, P. J. Hancock, R. Lange, J. Tocknell, M. Johnston-Hollitt, D. L. Kaplan, S. J. Tingay, M. Walker

Abstract: We present an overview of the Southern-sky MWA Rapid Two-metre (SMART) pulsar survey that exploits the MWA's large field of view and voltage capture system to survey the sky south of 30 degree in declination for pulsars and fast transients in the 140-170 MHz band. The survey is enabled by the advent of the Phase II MWA's compact configuration, which offers an enormous efficiency in beam-forming an… ▽ More We present an overview of the Southern-sky MWA Rapid Two-metre (SMART) pulsar survey that exploits the MWA's large field of view and voltage capture system to survey the sky south of 30 degree in declination for pulsars and fast transients in the 140-170 MHz band. The survey is enabled by the advent of the Phase II MWA's compact configuration, which offers an enormous efficiency in beam-forming and processing costs, thereby making an all-sky survey of this magnitude tractable with the MWA. Even with the long dwell times of the survey (4800 s), data collection can be completed in < 100 hours of telescope time, while still retaining the ability to reach a limiting sensitivity of ~2-3 mJy. Each observation is processed to generate ~5000-8000 tied-array beams that tessellate the full ~610 square degree field of view, which are then processed to search for pulsars. The voltage-capture recording allows a multitude of post hoc processing options including the reprocessing of data for higher time resolution. Due to the substantial computational cost in pulsar searches at low frequencies, processing is undertaken in multiple passes: in the first pass, a shallow survey is performed, where 10 minutes of each observation is processed, reaching about one-third of the full search sensitivity. Here we present the system overview and initial results. Further details including first pulsar discoveries and a census of low-frequency detections are presented in a companion paper. Future plans include deeper searches to reach the full sensitivity and acceleration searches to target binary and millisecond pulsars. Simulation analysis forecasts ~300 new pulsars upon the completion of full processing. The SMART survey will also generate a complete digital record of the low-frequency sky, which will serve as a valuable reference for future pulsar searches planned with the low-frequency Square Kilometre Array. △ Less

Submitted 23 February, 2023; originally announced February 2023.

Comments: 22 pages, 12 figures, 2 tables, Accepted for publication in PASA

Improved limits on the coupling of ultralight bosonic dark matter to photons from optical atomic clock comparisons

Authors: M. Filzinger, S. Dörscher, R. Lange, J. Klose, M. Steinel, E. Benkler, E. Peik, C. Lisdat, N. Huntemann

Abstract: We present improved constraints on the coupling of ultralight bosonic dark matter to photons based on long-term measurements of two optical frequency ratios. In these optical clock comparisons, we relate the frequency of the ${}^2S_{1/2} (F=0)\leftrightarrow {}^2F_{7/2} (F=3)$ electric-octupole (E3) transition in $^{171}$Yb$^{+}$ to that of the… ▽ More We present improved constraints on the coupling of ultralight bosonic dark matter to photons based on long-term measurements of two optical frequency ratios. In these optical clock comparisons, we relate the frequency of the ${}^2S_{1/2} (F=0)\leftrightarrow {}^2F_{7/2} (F=3)$ electric-octupole (E3) transition in $^{171}$Yb$^{+}$ to that of the ${}^2S_{1/2} (F=0)\leftrightarrow \,{}^2D_{3/2} (F=2)$ electric-quadrupole (E2) transition of the same ion, and to that of the ${}^1S_0\leftrightarrow\,{}^3P_0$ transition in $^{87}$Sr. Measurements of the first frequency ratio $ν_\textrm{E3}/ν_\textrm{E2}$ are performed via interleaved interrogation of both transitions in a single ion. The comparison of the single-ion clock based on the E3 transition with a strontium optical lattice clock yields the second frequency ratio $ν_\textrm{E3}/ν_\textrm{Sr}$. By constraining oscillations of the fine-structure constant $α$ with these measurement results, we improve existing bounds on the scalar coupling $d_e$ of ultralight dark matter to photons for dark matter masses in the range of about $ 10^{-24}-10^{-17}\,\textrm{eV}/c^2$. These results constitute an improvement by more than an order of magnitude over previous investigations for most of this range. We also use the repeated measurements of $ν_\textrm{E3}/ν_\textrm{E2}$ to improve existing limits on a linear temporal drift of $α$ and its coupling to gravity. △ Less

Submitted 9 January, 2023; originally announced January 2023.

Comments: 7 pages, 5 figures

Journal ref: Phys. Rev. Lett. 130, 253001 (2023)

evosax: JAX-based Evolution Strategies

Authors: Robert Tjarko Lange

Abstract: The deep learning revolution has greatly been accelerated by the 'hardware lottery': Recent advances in modern hardware accelerators and compilers paved the way for large-scale batch gradient optimization. Evolutionary optimization, on the other hand, has mainly relied on CPU-parallelism, e.g. using Dask scheduling and distributed multi-host infrastructure. Here we argue that also modern evolution… ▽ More The deep learning revolution has greatly been accelerated by the 'hardware lottery': Recent advances in modern hardware accelerators and compilers paved the way for large-scale batch gradient optimization. Evolutionary optimization, on the other hand, has mainly relied on CPU-parallelism, e.g. using Dask scheduling and distributed multi-host infrastructure. Here we argue that also modern evolutionary computation can significantly benefit from the massive computational throughput provided by GPUs and TPUs. In order to better harness these resources and to enable the next generation of black-box optimization algorithms, we release evosax: A JAX-based library of evolution strategies which allows researchers to leverage powerful function transformations such as just-in-time compilation, automatic vectorization and hardware parallelization. evosax implements 30 evolutionary optimization algorithms including finite-difference-based, estimation-of-distribution evolution strategies and various genetic algorithms. Every single algorithm can directly be executed on hardware accelerators and automatically vectorized or parallelized across devices using a single line of code. It is designed in a modular fashion and allows for flexible usage via a simple ask-evaluate-tell API. We thereby hope to facilitate a new wave of scalable evolutionary optimization algorithms. △ Less

Submitted 8 December, 2022; originally announced December 2022.

Comments: 5 pages, 3 figures

Discovering Evolution Strategies via Meta-Black-Box Optimization

Authors: Robert Tjarko Lange, Tom Schaul, Yutian Chen, Tom Zahavy, Valentin Dallibard, Chris Lu, Satinder Singh, Sebastian Flennerhag

Abstract: Optimizing functions without access to gradients is the remit of black-box methods such as evolution strategies. While highly general, their learning dynamics are often times heuristic and inflexible - exactly the limitations that meta-learning can address. Hence, we propose to discover effective update rules for evolution strategies via meta-learning. Concretely, our approach employs a search str… ▽ More Optimizing functions without access to gradients is the remit of black-box methods such as evolution strategies. While highly general, their learning dynamics are often times heuristic and inflexible - exactly the limitations that meta-learning can address. Hence, we propose to discover effective update rules for evolution strategies via meta-learning. Concretely, our approach employs a search strategy parametrized by a self-attention-based architecture, which guarantees the update rule is invariant to the ordering of the candidate solutions. We show that meta-evolving this system on a small set of representative low-dimensional analytic optimization problems is sufficient to discover new evolution strategies capable of generalizing to unseen optimization problems, population sizes and optimization horizons. Furthermore, the same learned evolution strategy can outperform established neuroevolution baselines on supervised and continuous control tasks. As additional contributions, we ablate the individual neural network components of our method; reverse engineer the learned strategy into an explicit heuristic form, which remains highly competitive; and show that it is possible to self-referentially train an evolution strategy from scratch, with the learned update rule used to drive the outer meta-learning loop. △ Less

Submitted 2 March, 2023; v1 submitted 21 November, 2022; originally announced November 2022.

Comments: 25 pages, 21 figures

Journal ref: 11th International Conference on Learning Representations, ICLR 2023

Neural Networks as Paths through the Space of Representations

Authors: Richard D. Lange, Devin Kwok, Jordan Matelsky, Xinyue Wang, David S. Rolnick, Konrad P. Kording

Abstract: Deep neural networks implement a sequence of layer-by-layer operations that are each relatively easy to understand, but the resulting overall computation is generally difficult to understand. We consider a simple hypothesis for interpreting the layer-by-layer construction of useful representations: perhaps the role of each layer is to reformat information to reduce the "distance" to the desired ou… ▽ More Deep neural networks implement a sequence of layer-by-layer operations that are each relatively easy to understand, but the resulting overall computation is generally difficult to understand. We consider a simple hypothesis for interpreting the layer-by-layer construction of useful representations: perhaps the role of each layer is to reformat information to reduce the "distance" to the desired outputs. With this framework, the layer-wise computation implemented by a deep neural network can be viewed as a path through a high-dimensional representation space. We formalize this intuitive idea of a "path" by leveraging recent advances in *metric* representational similarity. We extend existing representational distance methods by computing geodesics, angles, and projections of representations, going beyond mere layer distances. We then demonstrate these tools by visualizing and comparing the paths taken by ResNet and VGG architectures on CIFAR-10. We conclude by sketching additional ways that this kind of representational geometry can be used to understand and interpret network training, and to describe novel kinds of similarities between different models. △ Less

Submitted 27 November, 2022; v1 submitted 22 June, 2022; originally announced June 2022.

Comments: 10 pages, submitted to ICLR 2023

An Optical Atomic Clock Based on a Highly Charged Ion

Authors: Steven A. King, Lukas J. Spieß, Peter Micke, Alexander Wilzewski, Tobias Leopold, Erik Benkler, Richard Lange, Nils Huntemann, Andrey Surzhykov, Vladimir A. Yerokhin, José R. Crespo López-Urrutia, Piet O. Schmidt

Abstract: Optical atomic clocks are the most accurate measurement devices ever constructed and have found many applications in fundamental science and technology. The use of highly charged ions (HCI) as a new class of references for highest accuracy clocks and precision tests of fundamental physics has long been motivated by their extreme atomic properties and reduced sensitivity to perturbations from exter… ▽ More Optical atomic clocks are the most accurate measurement devices ever constructed and have found many applications in fundamental science and technology. The use of highly charged ions (HCI) as a new class of references for highest accuracy clocks and precision tests of fundamental physics has long been motivated by their extreme atomic properties and reduced sensitivity to perturbations from external electric and magnetic fields compared to singly charged ions or neutral atoms. Here we present the first realisation of this new class of clocks, based on an optical magnetic-dipole transition in Ar$^{13+}$. Its comprehensively evaluated systematic frequency uncertainty of $2.2\times10^{-17}$ is comparable to that of many optical clocks in operation. From clock comparisons we improve by eight and nine orders of magnitude upon the uncertainties for the absolute transition frequency and isotope shift ($^{40}$Ar vs. $^{36}$Ar), respectively. These measurements allow us to probe the largely unexplored quantum electrodynamic nuclear recoil, presented as part of improved calculations of the isotope shift which reduce the uncertainty of previous theory by a factor of three. This work establishes forbidden optical transitions in HCI as references for cutting-edge optical clocks and future high-sensitivity searches for physics beyond the standard model. △ Less

Submitted 6 September, 2023; v1 submitted 25 May, 2022; originally announced May 2022.

Comments: Main: 21 pages, 3 figures. Supplement: 20 pages, 2 figures. Accepted version

Journal ref: Nature 611, 43-47 (2022)

Clustering units in neural networks: upstream vs downstream information

Authors: Richard D. Lange, David S. Rolnick, Konrad P. Kording

Abstract: It has been hypothesized that some form of "modular" structure in artificial neural networks should be useful for learning, compositionality, and generalization. However, defining and quantifying modularity remains an open problem. We cast the problem of detecting functional modules into the problem of detecting clusters of similar-functioning units. This begs the question of what makes two units… ▽ More It has been hypothesized that some form of "modular" structure in artificial neural networks should be useful for learning, compositionality, and generalization. However, defining and quantifying modularity remains an open problem. We cast the problem of detecting functional modules into the problem of detecting clusters of similar-functioning units. This begs the question of what makes two units functionally similar. For this, we consider two broad families of methods: those that define similarity based on how units respond to structured variations in inputs ("upstream"), and those based on how variations in hidden unit activations affect outputs ("downstream"). We conduct an empirical study quantifying modularity of hidden layer representations of simple feedforward, fully connected networks, across a range of hyperparameters. For each model, we quantify pairwise associations between hidden units in each layer using a variety of both upstream and downstream measures, then cluster them by maximizing their "modularity score" using established tools from network science. We find two surprising results: first, dropout dramatically increased modularity, while other forms of weight regularization had more modest effects. Second, although we observe that there is usually good agreement about clusters within both upstream methods and downstream methods, there is little agreement about the cluster assignments across these two families of methods. This has important implications for representation-learning, as it suggests that finding modular representations that reflect structure in inputs (e.g. disentanglement) may be a distinct goal from learning modular representations that reflect structure in outputs (e.g. compositionality). △ Less

Submitted 22 March, 2022; originally announced March 2022.

Comments: 12 main text pages, 4 main figures, 5 supplemental figures. Will be submitted to TMLR

Journal ref: TMLR June (2022)

Galaxy And Mass Assembly (GAMA): Data Release 4 and the z < 0.1 total and z < 0.08 morphological galaxy stellar mass functions

Authors: Simon P. Driver, Sabine Bellstedt, Aaron S. G. Robotham, Ivan K. Baldry, Luke J. Davies, Jochen Liske, Danail Obreschkow, Edward N. Taylor, Angus H. Wright, Mehmet Alpaslan, Steven P. Bamford, Amanda E. Bauer, Joss Bland-Hawthorn, Maciej Bilicki, Matias Bravo, Sarah Brough, Sarah Casura, Michelle E. Cluver, Matthew Colless, Christopher J. Conselice, Scott M. Croom, Jelte de Jong, Franceso D'Eugenio, Roberto De Propris, Burak Dogruel , et al. (45 additional authors not shown)

Abstract: In Galaxy And Mass Assembly Data Release 4 (GAMA DR4), we make available our full spectroscopic redshift sample. This includes 248682 galaxy spectra, and, in combination with earlier surveys, results in 330542 redshifts across five sky regions covering ~250deg^2. The redshift density, is the highest available over such a sustained area, has exceptionally high completeness (95 per cent to r_KIDS=19… ▽ More In Galaxy And Mass Assembly Data Release 4 (GAMA DR4), we make available our full spectroscopic redshift sample. This includes 248682 galaxy spectra, and, in combination with earlier surveys, results in 330542 redshifts across five sky regions covering ~250deg^2. The redshift density, is the highest available over such a sustained area, has exceptionally high completeness (95 per cent to r_KIDS=19.65mag), and is well suited for the study of galaxy mergers, galaxy groups, and the low redshift (z<0.25) galaxy population. DR4 includes 32 value-added tables or Data Management Units (DMUs) that provide a number of measured and derived data products including GALEX, ESO KiDS, ESO VIKING, WISE and Herschel Space Observatory imaging. Within this release, we provide visual morphologies for 15330 galaxies to z<0.08, photometric redshift estimates for all 18million objects to r_KIDS~25mag, and stellar velocity dispersions for 111830 galaxies. We conclude by deriving the total galaxy stellar mass function (GSMF) and its sub-division by morphological class (elliptical, compact-bulge and disc, diffuse-bulge and disc, and disc only). This extends our previous measurement of the total GSMF down to 10^6.75 M_sol h^-2_70 and we find a total stellar mass density of rho_*=(2.97+/-0.04)x10^8 M_sol h_70 Mpc^-3 or Omega_*=(2.17+/-0.03)x10^-3 h^-1_70. We conclude that at z<0.1, the Universe has converted 4.9+/-0.1 per cent of the baryonic mass implied by Big Bang Nucleosynthesis into stars that are gravitationally bound within the galaxy population. △ Less

Submitted 16 March, 2022; originally announced March 2022.

Comments: Accepted for publication in MNRAS. GAMA Data Release 4 is available at: http://www.gama-survey.org/dr4/

Interpolating between sampling and variational inference with infinite stochastic mixtures

Authors: Richard D. Lange, Ari Benjamin, Ralf M. Haefner, Xaq Pitkow

Abstract: Sampling and Variational Inference (VI) are two large families of methods for approximate inference that have complementary strengths. Sampling methods excel at approximating arbitrary probability distributions, but can be inefficient. VI methods are efficient, but may misrepresent the true distribution. Here, we develop a general framework where approximations are stochastic mixtures of simple co… ▽ More Sampling and Variational Inference (VI) are two large families of methods for approximate inference that have complementary strengths. Sampling methods excel at approximating arbitrary probability distributions, but can be inefficient. VI methods are efficient, but may misrepresent the true distribution. Here, we develop a general framework where approximations are stochastic mixtures of simple component distributions. Both sampling and VI can be seen as special cases: in sampling, each mixture component is a delta-function and is chosen stochastically, while in standard VI a single component is chosen to minimize divergence. We derive a practical method that interpolates between sampling and VI by solving an optimization problem over a mixing distribution. Intermediate inference methods then arise by varying a single parameter. Our method provably improves on sampling (reducing variance) and on VI (reducing bias+variance despite increasing variance). We demonstrate our method's bias/variance trade-off in practice on reference problems, and we compare outcomes to commonly used sampling and VI methods. This work takes a step towards a highly flexible yet simple family of inference methods that combines the complementary strengths of sampling and VI. △ Less

Submitted 4 March, 2022; v1 submitted 18 October, 2021; originally announced October 2021.

Comments: 9 pages, 4 figures. Submitted to UAI 2022; under double-blind review. Code available at https://github.com/wrongu/sampling-variational-demos

A caster-wheel-aware MPC-based motion planner for mobile robotics

Authors: Jon Arrizabalaga, Niels van Duijkeren, Markus Ryll, Ralph Lange

Abstract: Differential drive mobile robots often use one or more caster wheels for balance. Caster wheels are appreciated for their ability to turn in any direction almost on the spot, allowing the robot to do the same and thereby greatly simplifying the motion planning and control. However, in aligning the caster wheels to the intended direction of motion they produce a so-called bore torque. As a result,… ▽ More Differential drive mobile robots often use one or more caster wheels for balance. Caster wheels are appreciated for their ability to turn in any direction almost on the spot, allowing the robot to do the same and thereby greatly simplifying the motion planning and control. However, in aligning the caster wheels to the intended direction of motion they produce a so-called bore torque. As a result, additional motor torque is required to move the robot, which may in some cases exceed the motor capacity or compromise the motion planner's accuracy. Instead of taking a decoupled approach, where the navigation and disturbance rejection algorithms are separated, we propose to embed the caster wheel awareness into the motion planner. To do so, we present a caster-wheel-aware term that is compatible with MPC-based control methods, leveraging the existence of caster wheels in the motion planning stage. As a proof of concept, this term is combined with a a model-predictive trajectory tracking controller. Since this method requires knowledge of the caster wheel angle and rolling speed, an observer that estimates these states is also presented. The efficacy of the approach is shown in experiments on an intralogistics robot and compared against a decoupled bore-torque reduction approach and a caster-wheel agnostic controller. Moreover, the experiments show that the presented caster wheel estimator performs sufficiently well and therefore avoids the need for additional sensors. △ Less

Submitted 11 October, 2021; originally announced October 2021.

Lifetime of the $^2F_{7/2}$ level in Yb$^+$ for spontaneous emission of electric octupole radiation

Authors: R. Lange, A. A. Peshkov, N. Huntemann, Chr. Tamm, A. Surzhykov, E. Peik

Abstract: We report a measurement of the radiative lifetime of the $^2F_{7/2}$ level of $^{171}$Yb$^+$ that is coupled to the $^2S_{1/2}$ ground state via an electric octupole transition. The radiative lifetime is determined to be $4.98(25)\times 10^7$ s, corresponding to 1.58(8) years. The result reduces the relative uncertainty in this exceptionally long excited state lifetime by one order of magnitude wi… ▽ More We report a measurement of the radiative lifetime of the $^2F_{7/2}$ level of $^{171}$Yb$^+$ that is coupled to the $^2S_{1/2}$ ground state via an electric octupole transition. The radiative lifetime is determined to be $4.98(25)\times 10^7$ s, corresponding to 1.58(8) years. The result reduces the relative uncertainty in this exceptionally long excited state lifetime by one order of magnitude with respect to previous experimental estimates. Our method is based on the coherent excitation of the corresponding transition and avoids limitations through competing decay processes. The explicit dependence on the laser intensity is eliminated by simultaneously measuring the resonant Rabi frequency and the induced quadratic Stark shift. Combining the result with information on the dynamic differential polarizability permits a calculation of the transition matrix element to infer the radiative lifetime. △ Less

Submitted 23 July, 2021; originally announced July 2021.

Comments: 5 pages, 2 figures

Budget-based real-time Executor for Micro-ROS

Authors: Jan Staschulat, Ralph Lange, Dakshina Narahari Dasari

Abstract: The Robot Operating System (ROS) is a popular robotics middleware framework. In the last years, it underwent a redesign and reimplementation under the name ROS~2. It now features QoS-configurable communication and a flexible layered architecture. Micro-ROS is a variant developed specifically for resource-constrained microcontrollers (MCU). Such MCUs are commonly used in robotics for sensors and ac… ▽ More The Robot Operating System (ROS) is a popular robotics middleware framework. In the last years, it underwent a redesign and reimplementation under the name ROS~2. It now features QoS-configurable communication and a flexible layered architecture. Micro-ROS is a variant developed specifically for resource-constrained microcontrollers (MCU). Such MCUs are commonly used in robotics for sensors and actuators, for time-critical control functions, and for safety. While the execution management of ROS 2 has been addressed by an Executor concept, its lack of real-time capabilities make it unsuitable for industrial use. Neither defining an execution order nor the assignment of scheduling parameters to tasks is possible, despite the fact that advanced real-time scheduling algorithms are well-known and available in modern RTOS's. For example, the NuttX RTOS supports a variant of the reservation-based scheduling which is very attractive for industrial applications: It allows to assign execution time budgets to software components so that a system integrator can thereby guarantee the real-time requirements of the entire system. This paper presents for the first time a ROS~2 Executor design which enables the real-time scheduling capabilities of the operating system. In particular, we successfully demonstrate the budget-based scheduling of the NuttX RTOS with a micro-ROS application on an STM32 microcontroller. △ Less

Submitted 18 May, 2021; v1 submitted 12 May, 2021; originally announced May 2021.

Comments: 4 pages, 5 figures, submitted to RTAS conference

On Lottery Tickets and Minimal Task Representations in Deep Reinforcement Learning

Authors: Marc Aurel Vischer, Robert Tjarko Lange, Henning Sprekeler

Abstract: The lottery ticket hypothesis questions the role of overparameterization in supervised deep learning. But how is the performance of winning lottery tickets affected by the distributional shift inherent to reinforcement learning problems? In this work, we address this question by comparing sparse agents who have to address the non-stationarity of the exploration-exploitation problem with supervised… ▽ More The lottery ticket hypothesis questions the role of overparameterization in supervised deep learning. But how is the performance of winning lottery tickets affected by the distributional shift inherent to reinforcement learning problems? In this work, we address this question by comparing sparse agents who have to address the non-stationarity of the exploration-exploitation problem with supervised agents trained to imitate an expert. We show that feed-forward networks trained with behavioural cloning compared to reinforcement learning can be pruned to higher levels of sparsity without performance degradation. This suggests that in order to solve the RL-specific distributional shift agents require more degrees of freedom. Using a set of carefully designed baseline conditions, we find that the majority of the lottery ticket effect in both learning paradigms can be attributed to the identified mask rather than the weight initialization. The input layer mask selectively prunes entire input dimensions that turn out to be irrelevant for the task at hand. At a moderate level of sparsity the mask identified by iterative magnitude pruning yields minimal task-relevant representations, i.e., an interpretable inductive bias. Finally, we propose a simple initialization rescaling which promotes the robust identification of sparse task representations in low-dimensional control tasks. △ Less

Submitted 10 May, 2022; v1 submitted 4 May, 2021; originally announced May 2021.

Comments: 18 pages, 15 figures

Eliciting judgements about dependent quantities of interest: The SHELF extension and copula methods illustrated using an asthma case study

Authors: Björn Holzhauer, Lisa V. Hampson, John Paul Gosling, Björn Bornkamp, Joseph Kahn, Markus R. Lange, Wen-Lin Luo, Caterina Brindicci, David Lawrence, Steffen Ballerstedt, Anthony O'Hagan

Abstract: Pharmaceutical companies regularly need to make decisions about drug development programs based on the limited knowledge from early stage clinical trials. In this situation, eliciting the judgements of experts is an attractive approach for synthesising evidence on the unknown quantities of interest. When calculating the probability of success for a drug development program, multiple quantities of… ▽ More Pharmaceutical companies regularly need to make decisions about drug development programs based on the limited knowledge from early stage clinical trials. In this situation, eliciting the judgements of experts is an attractive approach for synthesising evidence on the unknown quantities of interest. When calculating the probability of success for a drug development program, multiple quantities of interest - such as the effect of a drug on different endpoints - should not be treated as unrelated. We discuss two approaches for establishing a multivariate distribution for several related quantities within the SHeffield ELicitation Framework (SHELF). The first approach elicits experts' judgements about a quantity of interest conditional on knowledge about another one. For the second approach, we first elicit marginal distributions for each quantity of interest. Then, for each pair of quantities, we elicit the concordance probability that both lie on the same side of their respective elicited medians. This allows us to specify a copula to obtain the joint distribution of the quantities of interest. We show how these approaches were used in an elicitation workshop that was performed to assess the probability of success of the registrational program of an asthma drug. The judgements of the experts, which were obtained prior to completion of the pivotal studies, were well aligned with the final trial results. △ Less

Submitted 15 February, 2021; v1 submitted 4 February, 2021; originally announced February 2021.

Comments: 29 pages, 7 figures

MSC Class: 62P10; 62P30; 62C99

Improving the assessment of the probability of success in late stage drug development

Authors: Lisa V Hampson, Björn Bornkamp, Björn Holzhauer, Joseph Kahn, Markus R Lange, Wen-Lin Luo, Giovanni Della Cioppa, Kelvin Stott, Steffen Ballerstedt

Abstract: There are several steps to confirming the safety and efficacy of a new medicine. A sequence of trials, each with its own objectives, is usually required. Quantitative risk metrics can be useful for informing decisions about whether a medicine should transition from one stage of development to the next. To obtain an estimate of the probability of regulatory approval, pharmaceutical companies may st… ▽ More There are several steps to confirming the safety and efficacy of a new medicine. A sequence of trials, each with its own objectives, is usually required. Quantitative risk metrics can be useful for informing decisions about whether a medicine should transition from one stage of development to the next. To obtain an estimate of the probability of regulatory approval, pharmaceutical companies may start with industry-wide success rates and then apply to these subjective adjustments to reflect program-specific information. However, this approach lacks transparency and fails to make full use of data from previous clinical trials. We describe a quantitative Bayesian approach for calculating the probability of success (PoS) at the end of phase II which incorporates internal clinical data from one or more phase IIb studies, industry-wide success rates, and expert opinion or external data if needed. Using an example, we illustrate how PoS can be calculated accounting for differences between the phase IIb data and future phase III trials, and discuss how the methods can be extended to accommodate accelerated drug development pathways. △ Less

Submitted 21 October, 2021; v1 submitted 4 February, 2021; originally announced February 2021.

Comments: 22 pages, 9 figures, 3 tables, 45 references

The nanomorphology of cell surfaces of adhered osteoblasts

Authors: C. Voelkner, M. Wendt, R. Lange, M. Ulbrich, M. Gruening, S. Staehlke, J. B. Nebe, I. Barke, S. Speller

Abstract: Functionality of living cells is inherently linked to subunits with dimensions on the nanoscale. In case of osteoblasts the cell surface plays a particularly important role for adhesion and spreading which are crucial properties with regard to bone implants. Here we present a comprehensive characterization of the 3D nanomorphology of living as well as fixed osteoblastic cells using scanning ion co… ▽ More Functionality of living cells is inherently linked to subunits with dimensions on the nanoscale. In case of osteoblasts the cell surface plays a particularly important role for adhesion and spreading which are crucial properties with regard to bone implants. Here we present a comprehensive characterization of the 3D nanomorphology of living as well as fixed osteoblastic cells using scanning ion conductance microscopy (SICM) which is a nanoprobing method largely avoiding forces. Dynamic ruffles are observed, manifesting themselves in characteristic membrane protrusions. They contribute to the overall surface corrugation which we systematically study by introducing the relative 3D excess area as a function of projected adhesion area. A clear anticorrelation is found upon analysis of ~40 different cells on glass as well as on amine covered surfaces. At the rim of lamellipodia characteristic edge heights between 100 nm and ~300 nm are observed. Power spectral densities of membrane fluctuations show frequency-dependent decay exponents in excess of -2 on living osteoblasts. We discuss the capability of apical membrane features and fluctuation dynamics in aiding assessment of adhesion and migration properties on a single-cell basis. △ Less

Submitted 27 November, 2020; v1 submitted 16 November, 2020; originally announced November 2020.

Comments: 27 pages, 11 figures

Improved limits for violations of local position invariance from atomic clock comparisons

Authors: R. Lange, N. Huntemann, J. M. Rahm, C. Sanner, H. Shao, B. Lipphardt, Chr. Tamm, S. Weyers, E. Peik

Abstract: We compare two optical clocks based on the $^2$S$_{1/2}(F=0)\to {}^2$D$_{3/2}(F=2)$ electric quadrupole (E2) and the $^2$S$_{1/2}(F=0)\to {}^2$F$_{7/2}(F=3)$ electric octupole (E3) transition of $^{171}$Yb$^{+}$ and measure the frequency ratio $ν_{\mathrm{E3}}/ν_{\mathrm{E2}}=0.932\,829\,404\,530\,965\,376(32)$. We determine the transition frequency $ν_{E3}=642\,121\,496\,772\,645.10(8)$ Hz using… ▽ More We compare two optical clocks based on the $^2$S$_{1/2}(F=0)\to {}^2$D$_{3/2}(F=2)$ electric quadrupole (E2) and the $^2$S$_{1/2}(F=0)\to {}^2$F$_{7/2}(F=3)$ electric octupole (E3) transition of $^{171}$Yb$^{+}$ and measure the frequency ratio $ν_{\mathrm{E3}}/ν_{\mathrm{E2}}=0.932\,829\,404\,530\,965\,376(32)$. We determine the transition frequency $ν_{E3}=642\,121\,496\,772\,645.10(8)$ Hz using two caesium fountain clocks. Repeated measurements of both quantities over several years are analyzed for potential violations of local position invariance. We improve by factors of about 20 and 2 the limits for fractional temporal variations of the fine structure constant $α$ to $1.0(1.1)\times10^{-18}/\mathrm{yr}$ and of the proton-to-electron mass ratio $μ$ to $-8(36)\times10^{-18}/\mathrm{yr}$. Using the annual variation of the Sun's gravitational potential at Earth $Φ$, we improve limits for a potential coupling of both constants to gravity, $(c^2/α) (dα/dΦ)=14(11)\times 10^{-9}$ and $(c^2/μ) (dμ/dΦ)=7(45)\times 10^{-8}$. △ Less

Submitted 7 January, 2021; v1 submitted 13 October, 2020; originally announced October 2020.

Comments: 6 pages, 3 figures

Journal ref: Phys. Rev. Lett. 126, 011102 (2021)

Learning Not to Learn: Nature versus Nurture in Silico

Authors: Robert Tjarko Lange, Henning Sprekeler

Abstract: Animals are equipped with a rich innate repertoire of sensory, behavioral and motor skills, which allows them to interact with the world immediately after birth. At the same time, many behaviors are highly adaptive and can be tailored to specific environments by means of learning. In this work, we use mathematical analysis and the framework of meta-learning (or 'learning to learn') to answer when… ▽ More Animals are equipped with a rich innate repertoire of sensory, behavioral and motor skills, which allows them to interact with the world immediately after birth. At the same time, many behaviors are highly adaptive and can be tailored to specific environments by means of learning. In this work, we use mathematical analysis and the framework of meta-learning (or 'learning to learn') to answer when it is beneficial to learn such an adaptive strategy and when to hard-code a heuristic behavior. We find that the interplay of ecological uncertainty, task complexity and the agents' lifetime has crucial effects on the meta-learned amortized Bayesian inference performed by an agent. There exist two regimes: One in which meta-learning yields a learning algorithm that implements task-dependent information-integration and a second regime in which meta-learning imprints a heuristic or 'hard-coded' behavior. Further analysis reveals that non-adaptive behaviors are not only optimal for aspects of the environment that are stable across individuals, but also in situations where an adaptation to the environment would in fact be highly beneficial, but could not be done quickly enough to be exploited within the remaining lifetime. Hard-coded behaviors should hence not only be those that always work, but also those that are too complex to be learned within a reasonable time frame. △ Less

Submitted 1 May, 2022; v1 submitted 9 October, 2020; originally announced October 2020.

Optical frequency ratio of a ${}^{171}\mathrm{Yb}^+$ single-ion clock and a ${}^{87}\mathrm{Sr}$ lattice clock

Authors: Sören Dörscher, Nils Huntemann, Roman Schwarz, Richard Lange, Erik Benkler, Burghard Lipphardt, Uwe Sterr, Ekkehard Peik, Christian Lisdat

Abstract: We report direct measurements of the frequency ratio of the 642 THz ${}^2S_{1/2} (F=0)$--${}^2F_{7/2} (F=3)$ electric octupole transition in ${}^{171}\mathrm{Yb}^+$ and the 429 THz ${}^1S_0$--${}^3P_0$ transition in ${}^{87}\mathrm{Sr}$. A series of 107 measurements has been performed at the Physikalisch-Technische Bundesanstalt between December 2012 and October 2019. Long-term variations of the r… ▽ More We report direct measurements of the frequency ratio of the 642 THz ${}^2S_{1/2} (F=0)$--${}^2F_{7/2} (F=3)$ electric octupole transition in ${}^{171}\mathrm{Yb}^+$ and the 429 THz ${}^1S_0$--${}^3P_0$ transition in ${}^{87}\mathrm{Sr}$. A series of 107 measurements has been performed at the Physikalisch-Technische Bundesanstalt between December 2012 and October 2019. Long-term variations of the ratio are larger than expected from the individual measurement uncertainties of few $10^{-17}$. The cause of these variations remains unknown. Even taking these into account, we find a fractional uncertainty of the frequency ratio of $2.5 \times 10^{-17}$, which improves upon previous knowledge by one order of magnitude. The average frequency ratio is $ν_{\mathrm{Yb}^+} / ν_{\mathrm{Sr}} = 1.495\,991\,618\,544\,900\,537(38)$. This represents one of the most accurate measurements between two different atomic species to date. △ Less

Submitted 11 September, 2020; originally announced September 2020.

Comments: 9 pages, 4 figures, 1 table

Bellman filtering and smoothing for state-space models

Authors: Rutger-Jan Lange

Abstract: This paper presents a new filter for state-space models based on Bellman's dynamic-programming principle, allowing for nonlinearity, non-Gaussianity and degeneracy in the observation and/or state-transition equations. The resulting Bellman filter is a direct generalisation of the (iterated and extended) Kalman filter, enabling scalability to higher dimensions while remaining computationally inexpe… ▽ More This paper presents a new filter for state-space models based on Bellman's dynamic-programming principle, allowing for nonlinearity, non-Gaussianity and degeneracy in the observation and/or state-transition equations. The resulting Bellman filter is a direct generalisation of the (iterated and extended) Kalman filter, enabling scalability to higher dimensions while remaining computationally inexpensive. It can also be extended to enable smoothing. Under suitable conditions, the Bellman-filtered states are stable over time and contractive towards a region around the true state at every time step. Static (hyper)parameters are estimated by maximising a filter-implied pseudo log-likelihood decomposition. In univariate simulation studies, the Bellman filter performs on par with state-of-the-art simulation-based techniques at a fraction of the computational cost. In two empirical applications, involving up to 150 spatial dimensions or highly degenerate/nonlinear state dynamics, the Bellman filter outperforms competing methods in both accuracy and speed. △ Less

Submitted 28 November, 2023; v1 submitted 26 August, 2020; originally announced August 2020.

Comments: 60 pages

MSC Class: 62M20; 60G35; 93E11 ACM Class: G.3

Search for $hep$ solar neutrinos and the diffuse supernova neutrino background using all three phases of the Sudbury Neutrino Observatory

Authors: B. Aharmim, S. N. Ahmed, A. E. Anthony, N. Barros, E. W. Beier, A. Bellerive, B. Beltran, M. Bergevin, S. D. Biller, E. Blucher, R. Bonventre, K. Boudjemline, M. G. Boulay, B. Cai, E. J. Callaghan, J. Caravaca, Y. D. Chan, D. Chauhan, M. Chen, B. T. Cleveland, G. A. Cox, X. Dai, H. Deng, F. B. Descamps, J. A. Detwiler , et al. (107 additional authors not shown)

Abstract: A search has been performed for neutrinos from two sources, the $hep$ reaction in the solar $pp$ fusion chain and the $ν_e$ component of the diffuse supernova neutrino background (DSNB), using the full dataset of the Sudbury Neutrino Observatory with a total exposure of 2.47 kton-years after fiducialization. The $hep$ search is performed using both a single-bin counting analysis and a likelihood f… ▽ More A search has been performed for neutrinos from two sources, the $hep$ reaction in the solar $pp$ fusion chain and the $ν_e$ component of the diffuse supernova neutrino background (DSNB), using the full dataset of the Sudbury Neutrino Observatory with a total exposure of 2.47 kton-years after fiducialization. The $hep$ search is performed using both a single-bin counting analysis and a likelihood fit. We find a best-fit flux that is compatible with solar model predictions while remaining consistent with zero flux, and set a one-sided upper limit of $Φ_{hep} < 30\times10^{3}~\mathrm{cm}^{-2}~\mathrm{s}^{-1}$ [90% credible interval (CI)]. No events are observed in the DSNB search region, and we set an improved upper bound on the $ν_e$ component of the DSNB flux of $Φ^\mathrm{DSNB}_{ν_e} < 19~\textrm{cm}^{-2}~\textrm{s}^{-1}$ (90% CI) in the energy range $22.9 < E_ν< 36.9$~MeV. △ Less

Submitted 12 November, 2020; v1 submitted 15 July, 2020; originally announced July 2020.

Comments: 11 pages, 6 figures

Journal ref: Phys. Rev. D 102, 062006 (2020)

Coherent excitation of the highly forbidden electric octupole transition in ${}^{172}$Yb$^+$

Authors: Henning A. Fürst, Chih-Han Yeh, Dimitri Kalincev, André P. Kulosa, Laura S. Dreissen, Richard Lange, Erik Benkler, Nils Huntemann, Ekkehard Peik, Tanja E. Mehlstäubler

Abstract: We report on the first coherent excitation of the highly forbidden $^2S_{1/2}\rightarrow{}^2F_{7/2}$ electric octupole (E3) transition in a single trapped ${}^{172}$Yb$^+$ ion, an isotope without nuclear spin. Using the transition in ${}^{171}$Yb$^+$ as a reference, we determine the transition frequency to be $642\,116\,784\,950\,887.6(2.4)\,$Hz. We map out the magnetic field environment using the… ▽ More We report on the first coherent excitation of the highly forbidden $^2S_{1/2}\rightarrow{}^2F_{7/2}$ electric octupole (E3) transition in a single trapped ${}^{172}$Yb$^+$ ion, an isotope without nuclear spin. Using the transition in ${}^{171}$Yb$^+$ as a reference, we determine the transition frequency to be $642\,116\,784\,950\,887.6(2.4)\,$Hz. We map out the magnetic field environment using the forbidden $^2S_{1/2} \rightarrow{}^2D_{5/2}$ electric quadrupole (E2) transition and determine its frequency to be $729\,476\,867\,027\,206.8(4.4)\,$Hz. Our results are a factor of $1\times10^5$ ($3\times10^{5}$) more accurate for the E2 (E3) transition compared to previous measurements. The results open up the way to search for new physics via precise isotope shift measurements and improved tests of local Lorentz invariance using the metastable $^2F_{7/2}$ state of Yb$^+$. △ Less

Submitted 25 June, 2020; originally announced June 2020.

Comments: 6 pages, 5 figures

Journal ref: Phys. Rev. Lett. 125, 163001 (2020)

Coherent suppression of tensor frequency shifts through magnetic field rotation

Authors: R. Lange, N. Huntemann, C. Sanner, H. Shao, B. Lipphardt, Chr. Tamm, E. Peik

Abstract: We introduce a scheme to coherently suppress second-rank tensor frequency shifts in atomic clocks, relying on the continuous rotation of an external magnetic field during the free atomic state evolution in a Ramsey sequence. The method retrieves the unperturbed frequency within a single interrogation cycle and is readily applicable to various atomic clock systems. For the frequency shift due to th… ▽ More We introduce a scheme to coherently suppress second-rank tensor frequency shifts in atomic clocks, relying on the continuous rotation of an external magnetic field during the free atomic state evolution in a Ramsey sequence. The method retrieves the unperturbed frequency within a single interrogation cycle and is readily applicable to various atomic clock systems. For the frequency shift due to the electric quadrupole interaction, we experimentally demonstrate suppression by more than two orders of magnitude for the ${}^2S_{1/2} \to {}^2D_{3/2}$ transition of a single trapped ${}^{171}\text{Yb}^+$ ion. The scheme provides particular advantages in the case of the ${}^{171}\text{Yb}^+$ ${}^2S_{1/2} \to {}^2F_{7/2}$ electric octupole (E3) transition. For an improved estimate of the residual quadrupole shift for this transition, we measure the excited state electric quadrupole moments $Θ({}^2D_{3/2}) = 1.95(1)~ea_0^2$ and $Θ({}^2F_{7/2}) = -0.0297(5)~ea_0^2$ with $e$ the elementary charge and $a_0$ the Bohr radius, improving the measurement uncertainties by one order of magnitude. △ Less

Submitted 29 May, 2020; originally announced May 2020.

Comments: 6 pages, 3 figures

Journal ref: Phys. Rev. Lett. 125, 143201 (2020)

Generalized excitation of atomic multipole transitions by twisted light modes

Authors: S. A. -L. Schulz, A. A. Peshkov, R. A. Müller, R. Lange, N. Huntemann, Chr. Tamm, E. Peik, A. Surzhykov

Abstract: A theoretical study is performed for the excitation of a single atom localized in the center of twisted light modes. Here we present the explicit dependence of excitation rates on critical parameters, such as the polarization of light, its orbital angular momentum projection, and the orientation of its propagation axis with respect to the atomic quantization axis. The effect of a spatial spread of… ▽ More A theoretical study is performed for the excitation of a single atom localized in the center of twisted light modes. Here we present the explicit dependence of excitation rates on critical parameters, such as the polarization of light, its orbital angular momentum projection, and the orientation of its propagation axis with respect to the atomic quantization axis. The effect of a spatial spread of the atom is also considered in detail. The expressions for transition rates obtained in this work can be used for any atom of arbitrary electronic configuration. For definiteness we apply them to the specific case of $^{2}S_{1/2} (F=0) \rightarrow\; ^{2}F_{7/2} (F=3, M=0)$ electric octupole (E3) transition in $^{171}$Yb$^{+}$ ion. Our analytical and numerical results are suitable for the analysis and planning of future experiments on the excitation of electric-dipole-forbidden transitions by twisted light modes in optical atomic clocks. △ Less

Submitted 6 March, 2020; originally announced March 2020.

Journal ref: Phys. Rev. A 102, 012812 (2020)

Reinforcement Learning with Structured Hierarchical Grammar Representations of Actions

Authors: Petros Christodoulou, Robert Tjarko Lange, Ali Shafti, A. Aldo Faisal

Abstract: From a young age humans learn to use grammatical principles to hierarchically combine words into sentences. Action grammars is the parallel idea, that there is an underlying set of rules (a "grammar") that govern how we hierarchically combine actions to form new, more complex actions. We introduce the Action Grammar Reinforcement Learning (AG-RL) framework which leverages the concept of action gra… ▽ More From a young age humans learn to use grammatical principles to hierarchically combine words into sentences. Action grammars is the parallel idea, that there is an underlying set of rules (a "grammar") that govern how we hierarchically combine actions to form new, more complex actions. We introduce the Action Grammar Reinforcement Learning (AG-RL) framework which leverages the concept of action grammars to consistently improve the sample efficiency of Reinforcement Learning agents. AG-RL works by using a grammar inference algorithm to infer the "action grammar" of an agent midway through training. The agent's action space is then augmented with macro-actions identified by the grammar. We apply this framework to Double Deep Q-Learning (AG-DDQN) and a discrete action version of Soft Actor-Critic (AG-SAC) and find that it improves performance in 8 out of 8 tested Atari games (median +31%, max +668%) and 19 out of 20 tested Atari games (median +96%, maximum +3,756%) respectively without substantive hyperparameter tuning. We also show that AG-SAC beats the model-free state-of-the-art for sample efficiency in 17 out of the 20 tested Atari games (median +62%, maximum +13,140%), again without substantive hyperparameter tuning. △ Less

Submitted 23 October, 2019; v1 submitted 7 October, 2019; originally announced October 2019.

Cosmogenic Neutron Production at the Sudbury Neutrino Observatory

Authors: B. Aharmim, S. N. Ahmed, A. E. Anthony, N. Barros, E. W. Beier, A. Bellerive, B. Beltran, M. Bergevin, S. D. Biller, R. Bonventre, K. Boudjemline, M. G. Boulay, B. Cai, E. J. Callaghan, J. Caravaca, Y. D. Chan, D. Chauhan, M. Chen, B. T. Cleveland, G. A. Cox, R. Curley, X. Dai, H. Deng, F. B. Descamps, J. A. Detwiler , et al. (106 additional authors not shown)

Abstract: Neutrons produced in nuclear interactions initiated by cosmic-ray muons present an irreducible background to many rare-event searches, even in detectors located deep underground. Models for the production of these neutrons have been tested against previous experimental data, but the extrapolation to deeper sites is not well understood. Here we report results from an analysis of cosmogenically prod… ▽ More Neutrons produced in nuclear interactions initiated by cosmic-ray muons present an irreducible background to many rare-event searches, even in detectors located deep underground. Models for the production of these neutrons have been tested against previous experimental data, but the extrapolation to deeper sites is not well understood. Here we report results from an analysis of cosmogenically produced neutrons at the Sudbury Neutrino Observatory. A specific set of observables are presented, which can be used to benchmark the validity of GEANT4 physics models. In addition, the cosmogenic neutron yield, in units of $10^{-4}\;\text{cm}^{2}/\left(\text{g}\cdotμ\right)$, is measured to be $7.28 \pm 0.09\;\text{stat.} ^{+1.59}_{-1.12}\;\text{syst.}$ in pure heavy water and $7.30 \pm 0.07\;\text{stat.} ^{+1.40}_{-1.02}\;\text{syst.}$ in NaCl-loaded heavy water. These results provide unique insights into this potential background source for experiments at SNOLAB. △ Less

Submitted 25 September, 2019; originally announced September 2019.

Journal ref: Phys. Rev. D 100, 112005 (2019)

Semantic RL with Action Grammars: Data-Efficient Learning of Hierarchical Task Abstractions

Authors: Robert Tjarko Lange, Aldo Faisal

Abstract: Hierarchical Reinforcement Learning algorithms have successfully been applied to temporal credit assignment problems with sparse reward signals. However, state-of-the-art algorithms require manual specification of sub-task structures, a sample inefficient exploration phase or lack semantic interpretability. Humans, on the other hand, efficiently detect hierarchical sub-structures induced by their… ▽ More Hierarchical Reinforcement Learning algorithms have successfully been applied to temporal credit assignment problems with sparse reward signals. However, state-of-the-art algorithms require manual specification of sub-task structures, a sample inefficient exploration phase or lack semantic interpretability. Humans, on the other hand, efficiently detect hierarchical sub-structures induced by their surroundings. It has been argued that this inference process universally applies to language, logical reasoning as well as motor control. Therefore, we propose a cognitive-inspired Reinforcement Learning architecture which uses grammar induction to identify sub-goal policies. By treating an on-policy trajectory as a sentence sampled from the policy-conditioned language of the environment, we identify hierarchical constituents with the help of unsupervised grammatical inference. The resulting set of temporal abstractions is called action grammar (Pastra & Aloimonos, 2012) and unifies symbolic and connectionist approaches to Reinforcement Learning. It can be used to facilitate efficient imitation, transfer and online learning. △ Less

Submitted 23 September, 2019; v1 submitted 29 July, 2019; originally announced July 2019.

Comments: 11 pages, 8 figures

Search for transient variations of the fine structure constant and dark matter using fiber-linked optical atomic clocks

Authors: B. M. Roberts, P. Delva, A. Al-Masoudi, A. Amy-Klein, C. Bærentsen, C. F. A. Baynham, E. Benkler, S. Bilicki, S. Bize, W. Bowden, J. Calvert, V. Cambier, E. Cantin, E. A. Curtis, S. Dörscher, M. Favier, F. Frank, P. Gill, R. M. Godun, G. Grosche, C. Guo, A. Hees, I. R. Hill, R. Hobson, N. Huntemann , et al. (29 additional authors not shown)

Abstract: We search for transient variations of the fine structure constant using data from a European network of fiber-linked optical atomic clocks. By searching for coherent variations in the recorded clock frequency comparisons across the network, we significantly improve the constraints on transient variations of the fine structure constant. For example, we constrain the variation in alpha to <5*10^-17… ▽ More We search for transient variations of the fine structure constant using data from a European network of fiber-linked optical atomic clocks. By searching for coherent variations in the recorded clock frequency comparisons across the network, we significantly improve the constraints on transient variations of the fine structure constant. For example, we constrain the variation in alpha to <5*10^-17 for transients of duration 10^3 s. This analysis also presents a possibility to search for dark matter, the mysterious substance hypothesised to explain galaxy dynamics and other astrophysical phenomena that is thought to dominate the matter density of the universe. At the current sensitivity level, we find no evidence for dark matter in the form of topological defects (or, more generally, any macroscopic objects), and we thus place constraints on certain potential couplings between the dark matter and standard model particles, substantially improving upon the existing constraints, particularly for large (>~10^4 km) objects. △ Less

Submitted 8 July, 2019; v1 submitted 4 July, 2019; originally announced July 2019.

Journal ref: New J. Phys. 22, 093010 (2020)

Measurement of neutron production in atmospheric neutrino interactions at the Sudbury Neutrino Observatory

Authors: SNO Collaboration, B. Aharmim, S. N. Ahmed, A. E. Anthony, N. Barros, E. W. Beier, A. Bellerive, B. Beltran, M. Bergevin, S. D. Biller, R. Bonventre, K. Boudjemline, M. G. Boulay, B. Cai, E. J. Callaghan, J. Caravaca, Y. D. Chan, D. Chauhan, M. Chen, B. T. Cleveland, G. A. Cox, X. Dai, H. Deng, F. B. Descamps, J. A. Detwiler , et al. (107 additional authors not shown)

Abstract: Neutron production in GeV-scale neutrino interactions is a poorly studied process. We have measured the neutron multiplicities in atmospheric neutrino interactions in the Sudbury Neutrino Observatory experiment and compared them to the prediction of a Monte Carlo simulation using GENIE and a minimally modified version of GEANT4. We analyzed 837 days of exposure corresponding to Phase I, using pure… ▽ More Neutron production in GeV-scale neutrino interactions is a poorly studied process. We have measured the neutron multiplicities in atmospheric neutrino interactions in the Sudbury Neutrino Observatory experiment and compared them to the prediction of a Monte Carlo simulation using GENIE and a minimally modified version of GEANT4. We analyzed 837 days of exposure corresponding to Phase I, using pure heavy water, and Phase II, using a mixture of Cl in heavy water. Neutrons produced in atmospheric neutrino interactions were identified with an efficiency of $15.3\%$ and $44.3\%$, for Phase I and II respectively. The neutron production is measured as a function of the visible energy of the neutrino interaction and, for charged current quasi-elastic interaction candidates, also as a function of the neutrino energy. This study is also performed classifying the complete sample into two pairs of event categories: charged current quasi-elastic and non charged current quasi-elastic, and $ν_μ$ and $ν_e$. Results show good overall agreement between data and Monte Carlo for both phases, with some small tension with a statistical significance below $2σ$ for some intermediate energies. △ Less

Submitted 19 June, 2019; v1 submitted 1 April, 2019; originally announced April 2019.

Journal ref: Phys. Rev. D 99, 112007 (2019)

A Constraint Programming Approach to Simultaneous Task Allocation and Motion Scheduling for Industrial Dual-Arm Manipulation Tasks

Authors: Jan Kristof Behrens, Ralph Lange, Masoumeh Mansouri

Abstract: Modern lightweight dual-arm robots bring the physical capabilities to quickly take over tasks at typical industrial workplaces designed for workers. In times of mass-customization, low setup times including the instructing/specifying of new tasks are crucial to stay competitive. We propose a constraint programming approach to simultaneous task allocation and motion scheduling for such industrial m… ▽ More Modern lightweight dual-arm robots bring the physical capabilities to quickly take over tasks at typical industrial workplaces designed for workers. In times of mass-customization, low setup times including the instructing/specifying of new tasks are crucial to stay competitive. We propose a constraint programming approach to simultaneous task allocation and motion scheduling for such industrial manipulation and assembly tasks. The proposed approach covers dual-arm and even multi-arm robots as well as connected machines. The key concept are Ordered Visiting Constraints, a descriptive and extensible model to specify such tasks with their spatiotemporal requirements and task-specific combinatorial or ordering constraints. Our solver integrates such task models and robot motion models into constraint optimization problems and solves them efficiently using various heuristics to produce makespan-optimized robot programs. The proposed task model is robot independent and thus can easily be deployed to other robotic platforms. Flexibility and portability of our proposed model is validated through several experiments on different simulated robot platforms. We benchmarked our search strategy against a general-purpose heuristic. For large manipulation tasks with 200 objects, our solver implemented using Google's Operations Research tools and ROS requires less than a minute to compute usable plans. △ Less

Submitted 23 January, 2019; originally announced January 2019.

Comments: 8 pages, 8 figures, submitted to ICRA'19

Constraints on Neutrino Lifetime from the Sudbury Neutrino Observatory

Authors: SNO Collaboration, B. Aharmim, S. N. Ahmed, A. E. Anthony, N. Barros, E. W. Beier, A. Bellerive, B. Beltran, M. Bergevin, S. D. Biller, R. Bonventre, K. Boudjemline, M. G. Boulay, B. Cai, E. J. Callaghan, J. Caravaca, Y. D. Chan, D. Chauhan, M. Chen, B. T. Cleveland, G. A. Cox, X. Dai, H. Deng, F. B. Descamps, J. A. Detwiler , et al. (106 additional authors not shown)

Abstract: The long baseline between the Earth and the Sun makes solar neutrinos an excellent test beam for exploring possible neutrino decay. The signature of such decay would be an energy-dependent distortion of the traditional survival probability which can be fit for using well-developed and high precision analysis methods. Here a model including neutrino decay is fit to all three phases of $^8$B solar n… ▽ More The long baseline between the Earth and the Sun makes solar neutrinos an excellent test beam for exploring possible neutrino decay. The signature of such decay would be an energy-dependent distortion of the traditional survival probability which can be fit for using well-developed and high precision analysis methods. Here a model including neutrino decay is fit to all three phases of $^8$B solar neutrino data taken by the Sudbury Neutrino Observatory. This fit constrains the lifetime of neutrino mass state $ν_2$ to be ${>8.08\times10^{-5}}$ s/eV at $90\%$ confidence. An analysis combining this SNO result with those from other solar neutrino experiments results in a combined limit for the lifetime of mass state $ν_2$ of ${>1.04\times10^{-3}}$ s/eV at $99\%$ confidence. △ Less

Submitted 3 December, 2018; originally announced December 2018.

Journal ref: Phys. Rev. D 99, 032013 (2019)

A probabilistic population code based on neural samples

Authors: Sabyasachi Shivkumar, Richard D. Lange, Ankani Chattoraj, Ralf M. Haefner

Abstract: Sensory processing is often characterized as implementing probabilistic inference: networks of neurons compute posterior beliefs over unobserved causes given the sensory inputs. How these beliefs are computed and represented by neural responses is much-debated (Fiser et al. 2010, Pouget et al. 2013). A central debate concerns the question of whether neural responses represent samples of latent var… ▽ More Sensory processing is often characterized as implementing probabilistic inference: networks of neurons compute posterior beliefs over unobserved causes given the sensory inputs. How these beliefs are computed and represented by neural responses is much-debated (Fiser et al. 2010, Pouget et al. 2013). A central debate concerns the question of whether neural responses represent samples of latent variables (Hoyer & Hyvarinnen 2003) or parameters of their distributions (Ma et al. 2006) with efforts being made to distinguish between them (Grabska-Barwinska et al. 2013). A separate debate addresses the question of whether neural responses are proportionally related to the encoded probabilities (Barlow 1969), or proportional to the logarithm of those probabilities (Jazayeri & Movshon 2006, Ma et al. 2006, Beck et al. 2012). Here, we show that these alternatives - contrary to common assumptions - are not mutually exclusive and that the very same system can be compatible with all of them. As a central analytical result, we show that modeling neural responses in area V1 as samples from a posterior distribution over latents in a linear Gaussian model of the image implies that those neural responses form a linear Probabilistic Population Code (PPC, Ma et al. 2006). In particular, the posterior distribution over some experimenter-defined variable like "orientation" is part of the exponential family with sufficient statistics that are linear in the neural sampling-based firing rates. △ Less

Submitted 23 November, 2018; originally announced November 2018.

Comments: First three contributed equally to the work