-
Laboratory Experiments of Model-based Reinforcement Learning for Adaptive Optics Control
Authors:
Jalo Nousiainen,
Byron Engler,
Markus Kasper,
Chang Rajani,
Tapio Helin,
Cédric T. Heritier,
Sascha P. Quanz,
Adrian M. Glauser
Abstract:
Direct imaging of Earth-like exoplanets is one of the most prominent scientific drivers of the next generation of ground-based telescopes. Typically, Earth-like exoplanets are located at small angular separations from their host stars, making their detection difficult. Consequently, the adaptive optics (AO) system's control algorithm must be carefully designed to distinguish the exoplanet from the…
▽ More
Direct imaging of Earth-like exoplanets is one of the most prominent scientific drivers of the next generation of ground-based telescopes. Typically, Earth-like exoplanets are located at small angular separations from their host stars, making their detection difficult. Consequently, the adaptive optics (AO) system's control algorithm must be carefully designed to distinguish the exoplanet from the residual light produced by the host star.
A new promising avenue of research to improve AO control builds on data-driven control methods such as Reinforcement Learning (RL). RL is an active branch of the machine learning research field, where control of a system is learned through interaction with the environment. Thus, RL can be seen as an automated approach to AO control, where its usage is entirely a turnkey operation. In particular, model-based reinforcement learning (MBRL) has been shown to cope with both temporal and misregistration errors. Similarly, it has been demonstrated to adapt to non-linear wavefront sensing while being efficient in training and execution.
In this work, we implement and adapt an RL method called Policy Optimization for AO (PO4AO) to the GHOST test bench at ESO headquarters, where we demonstrate a strong performance of the method in a laboratory environment. Our implementation allows the training to be performed parallel to inference, which is crucial for on-sky operation. In particular, we study the predictive and self-calibrating aspects of the method. The new implementation on GHOST running PyTorch introduces only around 700 microseconds in addition to hardware, pipeline, and Python interface latency. We open-source well-documented code for the implementation and specify the requirements for the RTC pipeline. We also discuss the important hyperparameters of the method, the source of the latency, and the possible paths for a lower latency implementation.
△ Less
Submitted 30 December, 2023;
originally announced January 2024.
-
Towards on-sky adaptive optics control using reinforcement learning
Authors:
J. Nousiainen,
C. Rajani,
M. Kasper,
T. Helin,
S. Y. Haffert,
C. Vérinaud,
J. R. Males,
K. Van Gorkom,
L. M. Close,
J. D. Long,
A. D. Hedglen,
O. Guyon,
L. Schatz,
M. Kautz,
J. Lumbres,
A. Rodack,
J. M. Knight,
K. Miller
Abstract:
The direct imaging of potentially habitable Exoplanets is one prime science case for the next generation of high contrast imaging instruments on ground-based extremely large telescopes. To reach this demanding science goal, the instruments are equipped with eXtreme Adaptive Optics (XAO) systems which will control thousands of actuators at a framerate of kilohertz to several kilohertz. Most of the…
▽ More
The direct imaging of potentially habitable Exoplanets is one prime science case for the next generation of high contrast imaging instruments on ground-based extremely large telescopes. To reach this demanding science goal, the instruments are equipped with eXtreme Adaptive Optics (XAO) systems which will control thousands of actuators at a framerate of kilohertz to several kilohertz. Most of the habitable exoplanets are located at small angular separations from their host stars, where the current XAO systems' control laws leave strong residuals.Current AO control strategies like static matrix-based wavefront reconstruction and integrator control suffer from temporal delay error and are sensitive to mis-registration, i.e., to dynamic variations of the control system geometry. We aim to produce control methods that cope with these limitations, provide a significantly improved AO correction and, therefore, reduce the residual flux in the coronagraphic point spread function.
We extend previous work in Reinforcement Learning for AO. The improved method, called PO4AO, learns a dynamics model and optimizes a control neural network, called a policy. We introduce the method and study it through numerical simulations of XAO with Pyramid wavefront sensing for the 8-m and 40-m telescope aperture cases. We further implemented PO4AO and carried out experiments in a laboratory environment using MagAO-X at the Steward laboratory. PO4AO provides the desired performance by improving the coronagraphic contrast in numerical simulations by factors 3-5 within the control region of DM and Pyramid WFS, in simulation and in the laboratory. The presented method is also quick to train, i.e., on timescales of typically 5-10 seconds, and the inference time is sufficiently small (< ms) to be used in real-time control for XAO with currently available hardware even for extremely large telescopes.
△ Less
Submitted 16 May, 2022;
originally announced May 2022.
-
Unsupervised Metric Relocalization Using Transform Consistency Loss
Authors:
Mike Kasper,
Fernando Nobre,
Christoffer Heckman,
Nima Keivan
Abstract:
Training networks to perform metric relocalization traditionally requires accurate image correspondences. In practice, these are obtained by restricting domain coverage, employing additional sensors, or capturing large multi-view datasets. We instead propose a self-supervised solution, which exploits a key insight: localizing a query image within a map should yield the same absolute pose, regardle…
▽ More
Training networks to perform metric relocalization traditionally requires accurate image correspondences. In practice, these are obtained by restricting domain coverage, employing additional sensors, or capturing large multi-view datasets. We instead propose a self-supervised solution, which exploits a key insight: localizing a query image within a map should yield the same absolute pose, regardless of the reference image used for registration. Guided by this intuition, we derive a novel transform consistency loss. Using this loss function, we train a deep neural network to infer dense feature and saliency maps to perform robust metric relocalization in dynamic environments. We evaluate our framework on synthetic and real-world data, showing our approach outperforms other supervised methods when a limited amount of ground-truth information is available.
△ Less
Submitted 1 November, 2020;
originally announced November 2020.
-
A Model Order Reduction Algorithm for Estimating the Absorption Spectrum
Authors:
Roel Van Beeumen,
David B. Williams-Young,
Joseph M. Kasper,
Chao Yang,
Esmond G. Ng,
Xiaosong Li
Abstract:
The ab initio description of the spectral interior of the absorption spectrum poses both a theoretical and computational challenge for modern electronic structure theory. Due to the often spectrally dense character of this domain in the quantum propagator's eigenspectrum for medium-to-large sized systems, traditional approaches based on the partial diagonalization of the propagator often encounter…
▽ More
The ab initio description of the spectral interior of the absorption spectrum poses both a theoretical and computational challenge for modern electronic structure theory. Due to the often spectrally dense character of this domain in the quantum propagator's eigenspectrum for medium-to-large sized systems, traditional approaches based on the partial diagonalization of the propagator often encounter oscillatory and stagnating convergence. Electronic structure methods which solve the molecular response problem through the solution of spectrally shifted linear systems, such as the complex polarization propagator, offer an alternative approach which is agnostic to the underlying spectral density or domain location. This generality comes at a seemingly high computational cost associated with solving a large linear system for each spectral shift in some discretization of the spectral domain of interest. We present a novel, adaptive solution based on model order reduction techniques via interpolation. Model order reduction reduces the computational complexity of mathematical models and is ubiquitous in the simulation of dynamical systems. The efficiency and effectiveness of the proposed algorithm in the ab initio prediction of X-Ray absorption spectra is demonstrated using a test set of challenging water clusters which are spectrally dense in the neighborhood of the oxygen K-edge. Based on a single, user defined tolerance we automatically determine the order of the reduced models and approximate the absorption spectrum up to the given tolerance. We also illustrate that the automatically determined model order increases logarithmically with the problem dimension, compared to a linear increase of the number of eigenvalues within the energy window. Furthermore, we observed that the computational cost of the proposed algorithm only scales quadratically with respect to the problem dimension.
△ Less
Submitted 30 August, 2017; v1 submitted 19 April, 2017;
originally announced April 2017.
-
Light Source Estimation with Analytical Path-tracing
Authors:
Mike Kasper,
Nima Keivan,
Gabe Sibley,
Christoffer Heckman
Abstract:
We present a novel algorithm for light source estimation in scenes reconstructed with a RGB-D camera based on an analytically-derived formulation of path-tracing. Our algorithm traces the reconstructed scene with a custom path-tracer and computes the analytical derivatives of the light transport equation from principles in optics. These derivatives are then used to perform gradient descent, minimi…
▽ More
We present a novel algorithm for light source estimation in scenes reconstructed with a RGB-D camera based on an analytically-derived formulation of path-tracing. Our algorithm traces the reconstructed scene with a custom path-tracer and computes the analytical derivatives of the light transport equation from principles in optics. These derivatives are then used to perform gradient descent, minimizing the photometric error between one or more captured reference images and renders of our current lighting estimation using an environment map parameterization for light sources. We show that our approach of modeling all light sources as points at infinity approximates lights located near the scene with surprising accuracy. Due to the analytical formulation of derivatives, optimization to the solution is considerably accelerated. We verify our algorithm using both real and synthetic data.
△ Less
Submitted 15 January, 2017;
originally announced January 2017.
-
On the deployment of Mobile Trusted Modules
Authors:
Andreas U. Schmidt,
Nicolai Kuntze,
Michael Kasper
Abstract:
In its recently published TCG Mobile Reference Architecture, the TCG Mobile Phone Work Group specifies a new concept to enable trust into future mobile devices. For this purpose, the TCG devises a trusted mobile platform as a set of trusted engines on behalf of different stakeholders supported by a physical trust-anchor. In this paper, we present our perception on this emerging specification. We…
▽ More
In its recently published TCG Mobile Reference Architecture, the TCG Mobile Phone Work Group specifies a new concept to enable trust into future mobile devices. For this purpose, the TCG devises a trusted mobile platform as a set of trusted engines on behalf of different stakeholders supported by a physical trust-anchor. In this paper, we present our perception on this emerging specification. We propose an approach for the practical design and implementation of this concept and how to deploy it to a trustworthy operating platform. In particular we propose a method for the take-ownership of a device by the user and the migration (i.e., portability) of user credentials between devices.
△ Less
Submitted 13 December, 2007;
originally announced December 2007.
