Showing 1–44 of 44 results for author: Bertram, T

Contrastive Learning of Preferences with a Contextual InfoNCE Loss

Authors: Timo Bertram, Johannes Fürnkranz, Martin Müller

Abstract: A common problem in contextual preference ranking is that a single preferred action is compared against several choices, thereby blowing up the complexity and skewing the preference distribution. In this work, we show how one can solve this problem via a suitable adaptation of the CLIP framework.This adaptation is not entirely straight-forward, because although the InfoNCE loss used by CLIP has ac… ▽ More A common problem in contextual preference ranking is that a single preferred action is compared against several choices, thereby blowing up the complexity and skewing the preference distribution. In this work, we show how one can solve this problem via a suitable adaptation of the CLIP framework.This adaptation is not entirely straight-forward, because although the InfoNCE loss used by CLIP has achieved great success in computer vision and multi-modal domains, its batch-construction technique requires the ability to compare arbitrary items, and is not well-defined if one item has multiple positive associations in the same batch. We empirically demonstrate the utility of our adapted version of the InfoNCE loss in the domain of collectable card games, where we aim to learn an embedding space that captures the associations between single cards and whole card pools based on human selections. Such selection data only exists for restricted choices, thus generating concrete preferences of one item over a set of other items rather than a perfect fit between the card and the pool. Our results show that vanilla CLIP does not perform well due to the aforementioned intuitive issues. However, by adapting CLIP to the problem, we receive a model outperforming previous work trained with the triplet loss, while also alleviating problems associated with mining triplets. △ Less

Submitted 8 July, 2024; originally announced July 2024.

Learning With Generalised Card Representations for "Magic: The Gathering"

Authors: Timo Bertram, Johannes Fürnkranz, Martin Müller

Abstract: A defining feature of collectable card games is the deck building process prior to actual gameplay, in which players form their decks according to some restrictions. Learning to build decks is difficult for players and models alike due to the large card variety and highly complex semantics, as well as requiring meaningful card and deck representations when aiming to utilise AI. In addition, regula… ▽ More A defining feature of collectable card games is the deck building process prior to actual gameplay, in which players form their decks according to some restrictions. Learning to build decks is difficult for players and models alike due to the large card variety and highly complex semantics, as well as requiring meaningful card and deck representations when aiming to utilise AI. In addition, regular releases of new card sets lead to unforeseeable fluctuations in the available card pool, thus affecting possible deck configurations and requiring continuous updates. Previous Game AI approaches to building decks have often been limited to fixed sets of possible cards, which greatly limits their utility in practice. In this work, we explore possible card representations that generalise to unseen cards, thus greatly extending the real-world utility of AI-based deck building for the game "Magic: The Gathering".We study such representations based on numerical, nominal, and text-based features of cards, card images, and meta information about card usage from third-party services. Our results show that while the particular choice of generalised input representation has little effect on learning to predict human card selections among known cards, the performance on new, unseen cards can be greatly improved. Our generalised model is able to predict 55\% of human choices on completely unseen cards, thus showing a deep understanding of card quality and strategy. △ Less

Submitted 8 July, 2024; originally announced July 2024.

Comments: Best paper award nominee at IEEE Conference on Games 2024

Journal ref: IEEE Conference on Games 2024

Efficiently Training Neural Networks for Imperfect Information Games by Sampling Information Sets

Authors: Timo Bertram, Johannes Fürnkranz, Martin Müller

Abstract: In imperfect information games, the evaluation of a game state not only depends on the observable world but also relies on hidden parts of the environment. As accessing the obstructed information trivialises state evaluations, one approach to tackle such problems is to estimate the value of the imperfect state as a combination of all states in the information set, i.e., all possible states that ar… ▽ More In imperfect information games, the evaluation of a game state not only depends on the observable world but also relies on hidden parts of the environment. As accessing the obstructed information trivialises state evaluations, one approach to tackle such problems is to estimate the value of the imperfect state as a combination of all states in the information set, i.e., all possible states that are consistent with the current imperfect information. In this work, the goal is to learn a function that maps from the imperfect game information state to its expected value. However, constructing a perfect training set, i.e. an enumeration of the whole information set for numerous imperfect states, is often infeasible. To compute the expected values for an imperfect information game like \textit{Reconnaissance Blind Chess}, one would need to evaluate thousands of chess positions just to obtain the training target for a single state. Still, the expected value of a state can already be approximated with appropriate accuracy from a much smaller set of evaluations. Thus, in this paper, we empirically investigate how a budget of perfect information game evaluations should be distributed among training samples to maximise the return. Our results show that sampling a small number of states, in our experiments roughly 3, for a larger number of separate positions is preferable over repeatedly sampling a smaller quantity of states. Thus, we find that in our case, the quantity of different samples seems to be more important than higher target quality. △ Less

Submitted 8 July, 2024; originally announced July 2024.

Comments: KI 2024 - 47th German Conference on Artificial Intelligence

Journal ref: KI 2024 - 47th German Conference on Artificial Intelligence

Neural Network-based Information Set Weighting for Playing Reconnaissance Blind Chess

Authors: Timo Bertram, Johannes Fürnkranz, Martin Müller

Abstract: In imperfect information games, the game state is generally not fully observable to players. Therefore, good gameplay requires policies that deal with the different information that is hidden from each player. To combat this, effective algorithms often reason about information sets; the sets of all possible game states that are consistent with a player's observations. While there is no way to dist… ▽ More In imperfect information games, the game state is generally not fully observable to players. Therefore, good gameplay requires policies that deal with the different information that is hidden from each player. To combat this, effective algorithms often reason about information sets; the sets of all possible game states that are consistent with a player's observations. While there is no way to distinguish between the states within an information set, this property does not imply that all states are equally likely to occur in play. We extend previous research on assigning weights to the states in an information set in order to facilitate better gameplay in the imperfect information game of Reconnaissance Blind Chess. For this, we train two different neural networks which estimate the likelihood of each state in an information set from historical game data. Experimentally, we find that a Siamese neural network is able to achieve higher accuracy and is more efficient than a classical convolutional neural network for the given domain. Finally, we evaluate an RBC-playing agent that is based on the generated weightings and compare different parameter settings that influence how strongly it should rely on them. The resulting best player is ranked 5th on the public leaderboard. △ Less

Submitted 8 July, 2024; originally announced July 2024.

Comments: Extended version of IEEE Conference on Games 2023 paper

Journal ref: IEEE Transactions on Games 2024

Multi-Object Tracking based on Imaging Radar 3D Object Detection

Authors: Patrick Palmer, Martin Krüger, Richard Altendorfer, Torsten Bertram

Abstract: Effective tracking of surrounding traffic participants allows for an accurate state estimation as a necessary ingredient for prediction of future behavior and therefore adequate planning of the ego vehicle trajectory. One approach for detecting and tracking surrounding traffic participants is the combination of a learning based object detector with a classical tracking algorithm. Learning based ob… ▽ More Effective tracking of surrounding traffic participants allows for an accurate state estimation as a necessary ingredient for prediction of future behavior and therefore adequate planning of the ego vehicle trajectory. One approach for detecting and tracking surrounding traffic participants is the combination of a learning based object detector with a classical tracking algorithm. Learning based object detectors have been shown to work adequately on lidar and camera data, while learning based object detectors using standard radar data input have proven to be inferior. Recently, with the improvements to radar sensor technology in the form of imaging radars, the object detection performance on radar was greatly improved but is still limited compared to lidar sensors due to the sparsity of the radar point cloud. This presents a unique challenge for the task of multi-object tracking. The tracking algorithm must overcome the limited detection quality while generating consistent tracks. To this end, a comparison between different multi-object tracking methods on imaging radar data is required to investigate its potential for downstream tasks. The work at hand compares multiple approaches and analyzes their limitations when applied to imaging radar data. Furthermore, enhancements to the presented approaches in the form of probabilistic association algorithms are considered for this task. △ Less

Submitted 3 June, 2024; originally announced June 2024.

Comments: Presented at: 9. International ATZ-Live Automated Driving 2024

Energy-based Potential Games for Joint Motion Forecasting and Control

Authors: Christopher Diehl, Tobias Klosek, Martin Krüger, Nils Murzyn, Timo Osterburg, Torsten Bertram

Abstract: This work uses game theory as a mathematical framework to address interaction modeling in multi-agent motion forecasting and control. Despite its interpretability, applying game theory to real-world robotics, like automated driving, faces challenges such as unknown game parameters. To tackle these, we establish a connection between differential games, optimal control, and energy-based models, demo… ▽ More This work uses game theory as a mathematical framework to address interaction modeling in multi-agent motion forecasting and control. Despite its interpretability, applying game theory to real-world robotics, like automated driving, faces challenges such as unknown game parameters. To tackle these, we establish a connection between differential games, optimal control, and energy-based models, demonstrating how existing approaches can be unified under our proposed Energy-based Potential Game formulation. Building upon this, we introduce a new end-to-end learning application that combines neural networks for game-parameter inference with a differentiable game-theoretic optimization layer, acting as an inductive bias. The analysis provides empirical evidence that the game-theoretic layer adds interpretability and improves the predictive performance of various neural network backbones using two simulations and two real-world driving datasets. △ Less

Submitted 4 December, 2023; originally announced December 2023.

Comments: Conference on Robot Learning, CoRL 2023

Simulating METIS SCAO System

Authors: Markus Feldt, Horst Steuer, Carlos Correia, Andreas Obereder, Stefan Raffetseder, Thomas Bertram, Julia Shatokina, Faustine Cantalloube

Abstract: METIS, the Mid-Infrared ELT Imager and Spectrograph, is one of the four first-generation ELT instruments scheduled to see first light in 2028. Its two main science modules are supported by an adaptive optics system featuring a pyramid sensor with 90x90 subapertures working in the H and K bands. During the PDR and FDR phases, extensive simulations were carried out to support the sensing, reconstruc… ▽ More METIS, the Mid-Infrared ELT Imager and Spectrograph, is one of the four first-generation ELT instruments scheduled to see first light in 2028. Its two main science modules are supported by an adaptive optics system featuring a pyramid sensor with 90x90 subapertures working in the H and K bands. During the PDR and FDR phases, extensive simulations were carried out to support the sensing, reconstruction, and control concept of METIS single-conjugate adaptive optics (SCAO) system. We present details on the implementation of the COMPASS-based environment used for the simulations, the metrics used for analyzing our performance expectations, an overview of the main results, and some details on special cases like non-common path aberrations (NCPA) and water vapor seeing, as well as the low-wind effect. △ Less

Submitted 22 November, 2023; originally announced November 2023.

Comments: 17 pages, 14 Figures, AO4ELT VII Conference - Avignon - June 2023

Time-Optimal Trajectory Planning in Highway Scenarios using Basis-Spline Parameterization

Authors: Philip Dorpmüller, Thomas Schmitz, Naveen Bejagam, Torsten Bertram

Abstract: Basis splines enable a time-continuous feasibility check with a finite number of constraints. Constraints apply to the whole trajectory for motion planning applications that require a collision-free and dynamically feasible trajectory. Existing motion planners that rely on gradient-based optimization apply time scaling to implement a shrinking planning horizon. They neither guarantee a recursively… ▽ More Basis splines enable a time-continuous feasibility check with a finite number of constraints. Constraints apply to the whole trajectory for motion planning applications that require a collision-free and dynamically feasible trajectory. Existing motion planners that rely on gradient-based optimization apply time scaling to implement a shrinking planning horizon. They neither guarantee a recursively feasible trajectory nor enable reaching two terminal manifold parts at different time scales. This paper proposes a nonlinear optimization problem that addresses the drawbacks of existing approaches. Therefore, the spline breakpoints are included in the optimization variables. Transformations between spline bases are implemented so a sparse problem formulation is achieved. A strategy for breakpoint removal enables the convergence into a terminal manifold. The evaluation in an overtaking scenario shows the influence of the breakpoint number on the solution quality and the time required for optimization. △ Less

Submitted 5 October, 2023; originally announced October 2023.

Comments: Accepted for 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC)

On a Connection between Differential Games, Optimal Control, and Energy-based Models for Multi-Agent Interactions

Authors: Christopher Diehl, Tobias Klosek, Martin Krüger, Nils Murzyn, Torsten Bertram

Abstract: Game theory offers an interpretable mathematical framework for modeling multi-agent interactions. However, its applicability in real-world robotics applications is hindered by several challenges, such as unknown agents' preferences and goals. To address these challenges, we show a connection between differential games, optimal control, and energy-based models and demonstrate how existing approache… ▽ More Game theory offers an interpretable mathematical framework for modeling multi-agent interactions. However, its applicability in real-world robotics applications is hindered by several challenges, such as unknown agents' preferences and goals. To address these challenges, we show a connection between differential games, optimal control, and energy-based models and demonstrate how existing approaches can be unified under our proposed Energy-based Potential Game formulation. Building upon this formulation, this work introduces a new end-to-end learning application that combines neural networks for game-parameter inference with a differentiable game-theoretic optimization layer, acting as an inductive bias. The experiments using simulated mobile robot pedestrian interactions and real-world automated driving data provide empirical evidence that the game-theoretic layer improves the predictive performance of various neural network backbones. △ Less

Submitted 16 October, 2023; v1 submitted 31 August, 2023; originally announced August 2023.

Comments: International Conference on Machine Learning, Workshop on New Frontiers in Learning, Control, and Dynamical Systems (ICML 2023 Frontiers4LCD); added further related work in Energy-based Model sections in V2

Ego-Motion Estimation and Dynamic Motion Separation from 3D Point Clouds for Accumulating Data and Improving 3D Object Detection

Authors: Patrick Palmer, Martin Krueger, Richard Altendorfer, Torsten Bertram

Abstract: New 3+1D high-resolution radar sensors are gaining importance for 3D object detection in the automotive domain due to their relative affordability and improved detection compared to classic low-resolution radar sensors. One limitation of high-resolution radar sensors, compared to lidar sensors, is the sparsity of the generated point cloud. This sparsity could be partially overcome by accumulating… ▽ More New 3+1D high-resolution radar sensors are gaining importance for 3D object detection in the automotive domain due to their relative affordability and improved detection compared to classic low-resolution radar sensors. One limitation of high-resolution radar sensors, compared to lidar sensors, is the sparsity of the generated point cloud. This sparsity could be partially overcome by accumulating radar point clouds of subsequent time steps. This contribution analyzes limitations of accumulating radar point clouds on the View-of-Delft dataset. By employing different ego-motion estimation approaches, the dataset's inherent constraints, and possible solutions are analyzed. Additionally, a learning-based instance motion estimation approach is deployed to investigate the influence of dynamic motion on the accumulated point cloud for object detection. Experiments document an improved object detection performance by applying an ego-motion estimation and dynamic motion correction approach. △ Less

Submitted 29 August, 2023; originally announced August 2023.

Comments: Published at: AmE 2023 - Automotive meets Electronics; 14. GMM Symposium (https://ieeexplore.ieee.org/document/10227711)

Reviewing 3D Object Detectors in the Context of High-Resolution 3+1D Radar

Authors: Patrick Palmer, Martin Krueger, Richard Altendorfer, Ganesh Adam, Torsten Bertram

Abstract: Recent developments and the beginning market introduction of high-resolution imaging 4D (3+1D) radar sensors have initialized deep learning-based radar perception research. We investigate deep learning-based models operating on radar point clouds for 3D object detection. 3D object detection on lidar point cloud data is a mature area of 3D vision. Many different architectures have been proposed, ea… ▽ More Recent developments and the beginning market introduction of high-resolution imaging 4D (3+1D) radar sensors have initialized deep learning-based radar perception research. We investigate deep learning-based models operating on radar point clouds for 3D object detection. 3D object detection on lidar point cloud data is a mature area of 3D vision. Many different architectures have been proposed, each with strengths and weaknesses. Due to similarities between 3D lidar point clouds and 3+1D radar point clouds, those existing 3D object detectors are a natural basis to start deep learning-based 3D object detection on radar data. Thus, the first step is to analyze the detection performance of the existing models on the new data modality and evaluate them in depth. In order to apply existing 3D point cloud object detectors developed for lidar point clouds to the radar domain, they need to be adapted first. While some detectors, such as PointPillars, have already been adapted to be applicable to radar data, we have adapted others, e.g., Voxel R-CNN, SECOND, PointRCNN, and PV-RCNN. To this end, we conduct a cross-model validation (evaluating a set of models on one particular data set) as well as a cross-data set validation (evaluating all models in the model set on several data sets). The high-resolution radar data used are the View-of-Delft and Astyx data sets. Finally, we evaluate several adaptations of the models and their training procedures. We also discuss major factors influencing the detection performance on radar data and propose possible solutions indicating potential future research avenues. △ Less

Submitted 10 August, 2023; originally announced August 2023.

Comments: Published at CVPR 2023 Workshop on 3D Vision and Robotics (https://drive.google.com/file/d/1xj4R5ucH3PaR7QdRDJbbkjS-3iBUsruR/view)

Molecular Insights into Chemical Reactions at Aqueous Aerosol Interfaces

Authors: David T. Limmer, Andreas W. Götz, Timothy H. Bertram, Gilbert M. Nathanson

Abstract: Atmospheric aerosols facilitate reactions between ambient gases and dissolved species. Here, we review our efforts to interrogate the uptake of these gases and the mechanisms of their reactions both theoretically and experimentally. We highlight the fascinating behavior of $\mathrm{N}_2\mathrm{O}_5$ in solutions ranging from pure water to complex mixtures, chosen because its aerosol-mediated react… ▽ More Atmospheric aerosols facilitate reactions between ambient gases and dissolved species. Here, we review our efforts to interrogate the uptake of these gases and the mechanisms of their reactions both theoretically and experimentally. We highlight the fascinating behavior of $\mathrm{N}_2\mathrm{O}_5$ in solutions ranging from pure water to complex mixtures, chosen because its aerosol-mediated reactions significantly impact global ozone, hydroxyl, and methane concentrations. As a hydrophobic, weakly soluble, and highly reactive species, $\mathrm{N}_2\mathrm{O}_5$ is a sensitive probe of the chemical and physical properties of aerosol interfaces. We employ contemporary theory to disentangle the fate of $\mathrm{N}_2\mathrm{O}_5$ as it approaches pure and salty water, starting with adsorption and ending with hydrolysis to HNO$_3$, chlorination to $\mathrm{ClNO}_2$, or evaporation. Flow reactor and gas-liquid scattering experiments probe even greater complexity as added ions, organic molecules, and surfactants alter interfacial composition and reaction rates. Together, we reveal a new perspective on multiphase chemistry in the atmosphere. △ Less

Submitted 23 June, 2023; originally announced June 2023.

Comments: Posted with permission from the Annual Review of Physical Chemistry, Volume 75, copyright 2024 Annual Reviews, http://www.annualreviews.org

Differentiable Constrained Imitation Learning for Robot Motion Planning and Control

Authors: Christopher Diehl, Janis Adamek, Martin Krüger, Frank Hoffmann, Torsten Bertram

Abstract: Motion planning and control are crucial components of robotics applications like automated driving. Here, spatio-temporal hard constraints like system dynamics and safety boundaries (e.g., obstacles) restrict the robot's motions. Direct methods from optimal control solve a constrained optimization problem. However, in many applications finding a proper cost function is inherently difficult because… ▽ More Motion planning and control are crucial components of robotics applications like automated driving. Here, spatio-temporal hard constraints like system dynamics and safety boundaries (e.g., obstacles) restrict the robot's motions. Direct methods from optimal control solve a constrained optimization problem. However, in many applications finding a proper cost function is inherently difficult because of the weighting of partially conflicting objectives. On the other hand, Imitation Learning (IL) methods such as Behavior Cloning (BC) provide an intuitive framework for learning decision-making from offline demonstrations and constitute a promising avenue for planning and control in complex robot applications. Prior work primarily relied on soft constraint approaches, which use additional auxiliary loss terms describing the constraints. However, catastrophic safety-critical failures might occur in out-of-distribution (OOD) scenarios. This work integrates the flexibility of IL with hard constraint handling in optimal control. Our approach constitutes a general framework for constraint robotic motion planning and control, as well as traffic agent simulation, whereas we focus on mobile robot and automated driving applications. Hard constraints are integrated into the learning problem in a differentiable manner, via explicit completion and gradient-based correction. Simulated experiments of mobile robot navigation and automated driving provide evidence for the performance of the proposed method. △ Less

Submitted 28 August, 2023; v1 submitted 21 October, 2022; originally announced October 2022.

Comments: International Conference on Intelligent Robots and Systems Agents4AD Workshop, IROS 2023

Supervised and Reinforcement Learning from Observations in Reconnaissance Blind Chess

Authors: Timo Bertram, Johannes Fürnkranz, Martin Müller

Abstract: In this work, we adapt a training approach inspired by the original AlphaGo system to play the imperfect information game of Reconnaissance Blind Chess. Using only the observations instead of a full description of the game state, we first train a supervised agent on publicly available game records. Next, we increase the performance of the agent through self-play with the on-policy reinforcement le… ▽ More In this work, we adapt a training approach inspired by the original AlphaGo system to play the imperfect information game of Reconnaissance Blind Chess. Using only the observations instead of a full description of the game state, we first train a supervised agent on publicly available game records. Next, we increase the performance of the agent through self-play with the on-policy reinforcement learning algorithm Proximal Policy Optimization. We do not use any search to avoid problems caused by the partial observability of game states and only use the policy network to generate moves when playing. With this approach, we achieve an ELO of 1330 on the RBC leaderboard, which places our agent at position 27 at the time of this writing. We see that self-play significantly improves performance and that the agent plays acceptably well without search and without making assumptions about the true game state. △ Less

Submitted 3 August, 2022; originally announced August 2022.

Comments: 4 Pages, IEEE Conference on Games 2022 short paper

Quantity vs Quality: Investigating the Trade-Off between Sample Size and Label Reliability

Authors: Timo Bertram, Johannes Fürnkranz, Martin Müller

Abstract: In this paper, we study learning in probabilistic domains where the learner may receive incorrect labels but can improve the reliability of labels by repeatedly sampling them. In such a setting, one faces the problem of whether the fixed budget for obtaining training examples should rather be used for obtaining all different examples or for improving the label quality of a smaller number of exampl… ▽ More In this paper, we study learning in probabilistic domains where the learner may receive incorrect labels but can improve the reliability of labels by repeatedly sampling them. In such a setting, one faces the problem of whether the fixed budget for obtaining training examples should rather be used for obtaining all different examples or for improving the label quality of a smaller number of examples by re-sampling their labels. We motivate this problem in an application to compare the strength of poker hands where the training signal depends on the hidden community cards, and then study it in depth in an artificial setting where we insert controlled noise levels into the MNIST database. Our results show that with increasing levels of noise, resampling previous examples becomes increasingly more important than obtaining new examples, as classifier performance deteriorates when the number of incorrect labels is too high. In addition, we propose two different validation strategies; switching from lower to higher validations over the course of training and using chi-square statistics to approximate the confidence in obtained labels. △ Less

Submitted 20 April, 2022; originally announced April 2022.

Comments: Preliminary work under review for ICML 2022

LINC-NIRVANA Commissioning at the Large Binocular Telescope -- Lessons Learned

Authors: Kalyan Kumar Radhakrishnan Santhakumari, Carmelo Arcidiacono, Maria Bergomi, Thomas Bertram, Florian Briegel, Jacopo Farinato, Thomas M. Herbst, John M. Hill, Micah Klettk, Luca Marafatto, Rosalie C. McGurk, Roberto Ragazzoni, Fabio P. Santos, Valentina Viotto

Abstract: LINC-NIRVANA (LN) is one of the instruments on-board the Large Binocular Telescope (LBT). LN is a high-resolution, near-infrared imager equipped with an advanced adaptive optics module. LN implements layer-oriented Multi-Conjugate Adaptive Optics (MCAO) approach using two independent wavefront sensors per side of the binocular telescope measuring the turbulence volume above the telescope. The capa… ▽ More LINC-NIRVANA (LN) is one of the instruments on-board the Large Binocular Telescope (LBT). LN is a high-resolution, near-infrared imager equipped with an advanced adaptive optics module. LN implements layer-oriented Multi-Conjugate Adaptive Optics (MCAO) approach using two independent wavefront sensors per side of the binocular telescope measuring the turbulence volume above the telescope. The capability of acquiring up to 20 Natural Guide Stars simultaneously from two distinct fields of view, and using them for wavefront sensing with 20 separate pyramids per side of the telescope makes the LN MCAO system one of a kind. Commissioning of the left MCAO channel is almost complete, while that of the right arm is on-going. The Science Verification on the left side is expected to start soon after the MCAO performance is optimised for faint guide stars. In this article, we put together the lessons learned during the commissioning of the LN MCAO module. We hope and believe that this article will help the future MCAO instrument commissioning teams. △ Less

Submitted 2 December, 2021; originally announced December 2021.

Comments: AO$ELT6 Conference Proceeding (http://ao4elt6.copl.ulaval.ca/proceedings/401-GR5j-251.pdf)

UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning Leveraging Planning

Authors: Christopher Diehl, Timo Sievernich, Martin Krüger, Frank Hoffmann, Torsten Bertram

Abstract: Offline reinforcement learning (RL) provides a framework for learning decision-making from offline data and therefore constitutes a promising approach for real-world applications as automated driving. Self-driving vehicles (SDV) learn a policy, which potentially even outperforms the behavior in the sub-optimal data set. Especially in safety-critical applications as automated driving, explainabilit… ▽ More Offline reinforcement learning (RL) provides a framework for learning decision-making from offline data and therefore constitutes a promising approach for real-world applications as automated driving. Self-driving vehicles (SDV) learn a policy, which potentially even outperforms the behavior in the sub-optimal data set. Especially in safety-critical applications as automated driving, explainability and transferability are key to success. This motivates the use of model-based offline RL approaches, which leverage planning. However, current state-of-the-art methods often neglect the influence of aleatoric uncertainty arising from the stochastic behavior of multi-agent systems. This work proposes a novel approach for Uncertainty-aware Model-Based Offline REinforcement Learning Leveraging plAnning (UMBRELLA), which solves the prediction, planning, and control problem of the SDV jointly in an interpretable learning-based fashion. A trained action-conditioned stochastic dynamics model captures distinctively different future evolutions of the traffic scene. The analysis provides empirical evidence for the effectiveness of our approach in challenging automated driving simulations and based on a real-world public dataset. △ Less

Submitted 28 June, 2022; v1 submitted 22 November, 2021; originally announced November 2021.

Comments: Best Paper Award @ Advances in Neural Information Processing Systems - Machine Learning for Autonomous Driving Workshop (NeurIPS 2021 ML4AD)

Suboptimal nonlinear model predictive control with input move-blocking

Authors: Artemi Makarow, Christoph Rösmann, Torsten Bertram

Abstract: This paper deals with the integration of input move-blocking into the framework of suboptimal model predictive control. The blocked input parameterization is explicitly considered as a source of suboptimality. A straightforward integration approach is to hold back a manually generated stabilizing fallback solution in some buffer for the case that the optimizer does not find a better input move-blo… ▽ More This paper deals with the integration of input move-blocking into the framework of suboptimal model predictive control. The blocked input parameterization is explicitly considered as a source of suboptimality. A straightforward integration approach is to hold back a manually generated stabilizing fallback solution in some buffer for the case that the optimizer does not find a better input move-blocked solution. An extended approach superimposes the manually generated stabilizing warm-start by the move-blocked control sequence and enables a stepwise improvement of the control performance. In addition, this contribution provides a detailed review of the literature on input move-blocked model predictive control and combines important results with the findings of suboptimal model predictive control. A numerical example supports the theoretical results and shows the effectiveness of the proposed approach. △ Less

Submitted 27 June, 2022; v1 submitted 25 September, 2021; originally announced September 2021.

Comments: Added missing condition to Eq. (11); Added reference to difference inclusion (12) in Prop. 7 and Prop. 10; Removed typos;

Journal ref: International Journal of Control (2022)

A Comparison of Contextual and Non-Contextual Preference Ranking for Set Addition Problems

Authors: Timo Bertram, Johannes Fürnkranz, Martin Müller

Abstract: In this paper, we study the problem of evaluating the addition of elements to a set. This problem is difficult, because it can, in the general case, not be reduced to unconditional preferences between the choices. Therefore, we model preferences based on the context of the decision. We discuss and compare two different Siamese network architectures for this task: a twin network that compares the t… ▽ More In this paper, we study the problem of evaluating the addition of elements to a set. This problem is difficult, because it can, in the general case, not be reduced to unconditional preferences between the choices. Therefore, we model preferences based on the context of the decision. We discuss and compare two different Siamese network architectures for this task: a twin network that compares the two sets resulting after the addition, and a triplet network that models the contribution of each candidate to the existing set. We evaluate the two settings on a real-world task; learning human card preferences for deck building in the collectible card game Magic: The Gathering. We show that the triplet approach achieves a better result than the twin network and that both outperform previous results on this task. △ Less

Submitted 26 April, 2022; v1 submitted 9 July, 2021; originally announced July 2021.

Comments: arXiv admin note: substantial text overlap with arXiv:2105.11864

Journal ref: SubSetML: Subset Selection in Machine Learning: From Theory to Practice @ ICML 2021

Conference proceedings KI4Industry AI for SMEs -- The online congress for practical entry into AI for SMEs

Authors: Michael Arnemann, Per Olof Beckemeier, Thomas Bertram, Michael Eder, Maximilian Erschig, Matthias Feiner, Francisco Javier Fernandez Garcia, Frederic Foerster, Ruediger Haas, Martin Kipfmueller, Jan Kotschenreuther, Bernd Langer, Ivan Lozada Rodriguez, Thomas Meibert, Simon Ottenhaus, Stefan Paschek, Lars Pfotzer, Michael M. Roth, Tim Schanz, Philip Scherer, Janine Schwienke, Martin Simon, Robin Tenscher-Philipp

Abstract: The Institute of Materials and Processes, IMP, of the University of Applied Sciences in Karlsruhe, Germany in cooperation with VDI Verein Deutscher Ingenieure e.V, AEN Automotive Engineering Network and their cooperation partners present their competences of AI-based solution approaches in the production engineering field. The online congress KI 4 Industry on November 12 and 13, 2020, showed what… ▽ More The Institute of Materials and Processes, IMP, of the University of Applied Sciences in Karlsruhe, Germany in cooperation with VDI Verein Deutscher Ingenieure e.V, AEN Automotive Engineering Network and their cooperation partners present their competences of AI-based solution approaches in the production engineering field. The online congress KI 4 Industry on November 12 and 13, 2020, showed what opportunities the use of artificial intelligence offers for medium-sized manufacturing companies, SMEs, and where potential fields of application lie. The main purpose of KI 4 Industry is to increase the transfer of knowledge, research and technology from universities to small and medium-sized enterprises, to demystify the term AI and to encourage companies to use AI-based solutions in their own value chain or in their products. △ Less

Submitted 5 August, 2021; v1 submitted 14 June, 2021; originally announced June 2021.

Comments: Editors: Matthias Feiner and Manuel Schoellhorn, 72 pages, 48 figures, in German, Conference proceedings KI 4 Industry, 79 pages in total

Predicting Human Card Selection in Magic: The Gathering with Contextual Preference Ranking

Authors: Timo Bertram, Johannes Fürnkranz, Martin Müller

Abstract: Drafting, i.e., the selection of a subset of items from a larger candidate set, is a key element of many games and related problems. It encompasses team formation in sports or e-sports, as well as deck selection in many modern card games. The key difficulty of drafting is that it is typically not sufficient to simply evaluate each item in a vacuum and to select the best items. The evaluation of an… ▽ More Drafting, i.e., the selection of a subset of items from a larger candidate set, is a key element of many games and related problems. It encompasses team formation in sports or e-sports, as well as deck selection in many modern card games. The key difficulty of drafting is that it is typically not sufficient to simply evaluate each item in a vacuum and to select the best items. The evaluation of an item depends on the context of the set of items that were already selected earlier, as the value of a set is not just the sum of the values of its members - it must include a notion of how well items go together. In this paper, we study drafting in the context of the card game Magic: The Gathering. We propose the use of a contextual preference network, which learns to compare two possible extensions of a given deck of cards. We demonstrate that the resulting network is better able to evaluate card decks in this game than previous attempts. △ Less

Submitted 29 June, 2021; v1 submitted 25 May, 2021; originally announced May 2021.

Comments: IEEE Conference on Games 2021 version

Journal ref: 3rd IEEE Conference on Games 2021

The adaptive optics simulation analysis tool(kit) (AOSAT)

Authors: Markus Feldt, Stefan Hippler, Faustine Cantalloube, Thomas Bertram, Andreas Obereder, Horst Steuer, Olivier Absil, Miska Le Louarn

Abstract: AOSAT is a python package for the analysis of single-conjugate adaptive optics (SCAO) simulation results. Python is widely used in the astronomical community these days, and AOSAT may be used stand-alone, integrated into a simulation environment, or can easily be extended according to a user's needs. Standalone operation requires the user to provide the residual wavefront frames provided by the SC… ▽ More AOSAT is a python package for the analysis of single-conjugate adaptive optics (SCAO) simulation results. Python is widely used in the astronomical community these days, and AOSAT may be used stand-alone, integrated into a simulation environment, or can easily be extended according to a user's needs. Standalone operation requires the user to provide the residual wavefront frames provided by the SCAO simulation package used, the aperture mask (pupil) used for the simulation, and a custom setup file describing the simulation/analysis configuration. In its standard form, AOSAT's "tearsheet" functionality will then run all standard analyzers, providing an informative plot collection on properties such as the point-spread function (PSF) and its quality, residual tip-tilt, the impact of pupil fragmentation, residual optical aberration modes both static and dynamic, the expected high-contrast performance of suitable instrumentation with and without coronagraphs, and the power spectral density of residual wavefront errors. AOSAT fills the gap between the simple numerical outputs provided by most simulation packages, and the full-scale deployment of instrument simulators and data reduction suites operating on SCAO residual wavefronts. It enables instrument designers and end-users to quickly judge the impact of design or configuration decisions on the final performance of down-stream instrumentation. △ Less

Submitted 5 February, 2021; originally announced February 2021.

Comments: 18 pages, 11 figures

Journal ref: Proceedings of the SPIE, Volume 11448, id. 114483S 18 pp. (2020)

Radar-based Dynamic Occupancy Grid Map** and Object Detection

Authors: Christopher Diehl, Eduard Feicho, Alexander Schwambach, Thomas Dammeier, Eric Mares, Torsten Bertram

Abstract: Environment modeling utilizing sensor data fusion and object tracking is crucial for safe automated driving. In recent years, the classical occupancy grid map approach, which assumes a static environment, has been extended to dynamic occupancy grid maps, which maintain the possibility of a low-level data fusion while also estimating the position and velocity distribution of the dynamic local envir… ▽ More Environment modeling utilizing sensor data fusion and object tracking is crucial for safe automated driving. In recent years, the classical occupancy grid map approach, which assumes a static environment, has been extended to dynamic occupancy grid maps, which maintain the possibility of a low-level data fusion while also estimating the position and velocity distribution of the dynamic local environment. This paper presents the further development of a previous approach. To the best of the author's knowledge, there is no publication about dynamic occupancy grid map** with subsequent analysis based only on radar data. Therefore in this work, the data of multiple radar sensors are fused, and a grid-based object tracking and map** method is applied. Subsequently, the clustering of dynamic areas provides high-level object information. For comparison, also a lidar-based method is developed. The approach is evaluated qualitatively and quantitatively with real-world data from a moving vehicle in urban environments. The evaluation illustrates the advantages of the radar-based dynamic occupancy grid map, considering different comparison metrics. △ Less

Submitted 9 August, 2020; originally announced August 2020.

Comments: Accepted to be published as part of the 23rd IEEE International Conference on Intelligent Transportation Systems (ITSC), Rhodes, Greece, September 20-23, 2020

Online Motion Planning based on Nonlinear Model Predictive Control with Non-Euclidean Rotation Groups

Authors: Christoph Rösmann, Artemi Makarow, Torsten Bertram

Abstract: This paper proposes a novel online motion planning approach to robot navigation based on nonlinear model predictive control. Common approaches rely on pure Euclidean optimization parameters. In robot navigation, however, state spaces often include rotational components which span over non-Euclidean rotation groups. The proposed approach applies nonlinear increment and difference operators in the e… ▽ More This paper proposes a novel online motion planning approach to robot navigation based on nonlinear model predictive control. Common approaches rely on pure Euclidean optimization parameters. In robot navigation, however, state spaces often include rotational components which span over non-Euclidean rotation groups. The proposed approach applies nonlinear increment and difference operators in the entire optimization scheme to explicitly consider these groups. Realizations include but are not limited to quadratic form and time-optimal objectives. A complex parking scenario for the kinematic bicycle model demonstrates the effectiveness and practical relevance of the approach. In case of simpler robots (e.g. differential drive), a comparative analysis in a hierarchical planning setting reveals comparable computation times and performance. The approach is available in a modular and highly configurable open-source C++ software framework. △ Less

Submitted 5 June, 2020; originally announced June 2020.

Journal ref: 2021 European Control Conference (ECC), 2021, pp. 1583-1590

Time-optimal control with direct collocation and variable discretization

Authors: Christoph Rösmann, Artemi Makarow, Torsten Bertram

Abstract: This paper deals with time-optimal control of nonlinear continuous-time systems based on direct collocation. The underlying discretization grid is variable in time, as the time intervals are subject to optimization. This technique differs from approaches that are usually based on a time transformation. Hermite-Simpson collocation is selected as common representative in the field of optimal control… ▽ More This paper deals with time-optimal control of nonlinear continuous-time systems based on direct collocation. The underlying discretization grid is variable in time, as the time intervals are subject to optimization. This technique differs from approaches that are usually based on a time transformation. Hermite-Simpson collocation is selected as common representative in the field of optimal control and trajectory optimization. Hereby, quadratic splines approximate the system dynamics. Several splines of different order are suitable for the control parameterization. A comparative analysis reveals that increasing the degrees of freedom in control, e.g. quadratic splines, is not suitable for time-optimal control problems due to constraint violation and inherent oscillations. However, choosing constant or linear control splines points out to be very effective. A major advantage is that the implicit solution of the system dynamics is suited for stiff systems and often requires smaller grid sizes in practice. △ Less

Submitted 25 May, 2020; originally announced May 2020.

Stabilizing Quasi-Time-Optimal Nonlinear Model Predictive Control with Variable Discretization

Authors: Christoph Rösmann, Artemi Makarow, Torsten Bertram

Abstract: This paper deals with the development and analysis of novel time-optimal point-to-point model predictive control concepts for nonlinear systems. Recent approaches in the literature apply a time transformation, however, which do not maintain recursive feasibility for piecewise constant control parameterization. The key idea in this paper is to introduce uniform grids with variable discretization. A… ▽ More This paper deals with the development and analysis of novel time-optimal point-to-point model predictive control concepts for nonlinear systems. Recent approaches in the literature apply a time transformation, however, which do not maintain recursive feasibility for piecewise constant control parameterization. The key idea in this paper is to introduce uniform grids with variable discretization. A shrinking-horizon grid adaptation scheme ensures convergence to a specific region around the target state and recursive feasibility. The size of the region is configurable by design parameters. This facilitates the systematic dual-mode design for quasi-time-optimal control to restore asymptotic stability and establish a smooth stabilization. Two nonlinear program formulations with different sparsity patterns are introduced to realize and implement the underlying optimal control problem. For a class of numerical integration schemes, even nominal asymptotic stability and true time-optimality are achieved without dual-mode. A comparative analysis as well as experimental results demonstrate the effectiveness of the proposed techniques. △ Less

Submitted 20 April, 2020; originally announced April 2020.

Journal ref: International Journal of Control (2021)

High contrast imaging with ELT/METIS: The wind driven halo, from SPHERE to METIS

Authors: Faustine Cantalloube, Olivier Absil, Thomas Bertram, Wolfgang Brandner, Christian Delacroix, Markus Feldt, Matthew Kenworthy, Martin Kulas, Julien Milli, Philip Neureuther, Gilles Orban de Xivry, Prashant Pathak, Emiel Por, Silvia Scheithauer, Horst Steuer, Roy van Boekel

Abstract: METIS is one of the three first-light instruments planned for the ELT, mainly dedicated to high contrast imaging in the mid-infrared. On the SPHERE high-contrast instrument currently installed at the VLT, we observe that one of the main contrast limitations is the wind driven halo, due to the limited AO running speed with respect to the atmospheric turbulence temporal evolution. From this observat… ▽ More METIS is one of the three first-light instruments planned for the ELT, mainly dedicated to high contrast imaging in the mid-infrared. On the SPHERE high-contrast instrument currently installed at the VLT, we observe that one of the main contrast limitations is the wind driven halo, due to the limited AO running speed with respect to the atmospheric turbulence temporal evolution. From this observation, we extrapolate this signature to the ELT/METIS instrument, which is equipped with a single conjugated adaptive optics system and with several coronagraphic devices. By making use of an analytic AO simulator, we compare the amount of wind driven halo observed with SPHERE and with METIS, under the same turbulence conditions. △ Less

Submitted 25 November, 2019; originally announced November 2019.

Comments: AO4ELT6 Conference Proceedings, 12 pages, 8 figures

Single conjugate adaptive optics for the ELT instrument METIS

Authors: Stefan Hippler, Markus Feldt, Thomas Bertram, Wolfgang Brandner, Faustine Cantalloube, Brunella Carlomagno, Olivier Absil, Andreas Obereder, Iuliia Shatokhina, Remko Stuik

Abstract: The ELT is a 39m large, ground-based optical and near- to mid-infrared telescope under construction in the Chilean Atacama desert. Operation is planned to start around the middle of the next decade. All first light instruments will come with wavefront sensing devices that allow control of the ELT's intrinsic M4 and M5 wavefront correction units, thus building an adaptive optics (AO) system. To tak… ▽ More The ELT is a 39m large, ground-based optical and near- to mid-infrared telescope under construction in the Chilean Atacama desert. Operation is planned to start around the middle of the next decade. All first light instruments will come with wavefront sensing devices that allow control of the ELT's intrinsic M4 and M5 wavefront correction units, thus building an adaptive optics (AO) system. To take advantage of the ELT's optical performance, full diffraction-limited operation is required and only a high performance AO system can deliver this. Further technically challenging requirements for the AO come from the exoplanet research field, where the task to resolve the very small angular separations between host star and planet, has also to take into account the high-contrast ratio between the two objects. We present in detail the results of our simulations and their impact on high-contrast imaging in order to find the optimal wavefront sensing device for the METIS instrument. METIS is the mid-infrared imager and spectrograph for the ELT with specialised high-contrast, coronagraphic imaging capabilities, whose performance strongly depends on the AO residual wavefront errors. We examined the sky and target sample coverage of a generic wavefront sensor in two spectral regimes, visible and near-infrared, to pre-select the spectral range for the more detailed wavefront sensor type analysis. We find that the near-infrared regime is the most suitable for METIS. We then analysed the performance of Shack-Hartmann and pyramid wavefront sensors under realistic conditions at the ELT, did a balancing with our scientific requirements, and concluded that a pyramid wavefront sensor is the best choice for METIS. For this choice we additionally examined the impact of non-common path aberrations, of vibrations, and the long-term stability of the SCAO system including high-contrast imaging performance. △ Less

Submitted 22 January, 2019; v1 submitted 17 October, 2018; originally announced October 2018.

Comments: 37 pages, 27 figures, accepted for publication in Experimental Astronomy

Journal ref: Exp Astron (November 2018)

Alignment and preliminary outcomes of an ELT-size instrument to a very large telescope: LINC-NIRVANA at LBT

Authors: M. Bergomi, L. Marafatto, V. Viotto, C. Arcidiacono, J. Farinato, K. K. R. Santhakumari, R. McGurk, M. Dima, H. Baumeister, T. Bertram, J. Berwein, P. Bizenberger, F. Briegel, F. Kittman, A. Conrad, R. Ragazzoni, T. M. Herbst

Abstract: LINC-NIRVANA (LN) is a high resolution, near infrared imager that uses a multiple field-of-view, layer-oriented, multi-conjugate AO system, consisting of four multi-pyramid wavefront sensors (two for each arm of the Large Binocular Telescope, each conjugated to a different altitude). The system employs up to 40 star probes, looking at up to 20 natural guide stars simultaneously. Its final goal is… ▽ More LINC-NIRVANA (LN) is a high resolution, near infrared imager that uses a multiple field-of-view, layer-oriented, multi-conjugate AO system, consisting of four multi-pyramid wavefront sensors (two for each arm of the Large Binocular Telescope, each conjugated to a different altitude). The system employs up to 40 star probes, looking at up to 20 natural guide stars simultaneously. Its final goal is to perform Fizeau interferometric imaging, thereby achieving ELT-like spatial resolution (22.8 m baseline resolution). For this reason, LN is also equipped with a fringe tracker, a beam combiner and a NIR science camera, for a total of more than 250 optical components and an overall size of approximately 6x4x4.5 meters. This paper describes the tradeoffs evaluated in order to achieve the alignment of the system to the telescope. We note that LN is comparable in size to planned ELT instrumentation. The impact of such alignment strategies will be compared and the selected procedure, where the LBT telescope is, in fact, aligned to the instrument, will be described. Furthermore, results coming from early night-time commissioning of the system will be presented. △ Less

Submitted 1 August, 2018; originally announced August 2018.

Comments: 8 pages, 6 pages, AO4ELT5 Proceedings, 2017

Journal ref: AO4ELT5 Proceedings, 2017

The calibration procedure of the LINC-NIRVANA ground and high layer WFS

Authors: Carmelo Arcidiacono, Kalyan Kumar Radhakrishnan Santhakumari, Valentina Viotto, Maria Bergomi, Florian Briegel, Thomas Bertram, Luca Marafatto, Tom Herbst, Jacopo Farinato, Roberto Ragazzoni, Ralph Hofferbert, Martin Kuerster, Frank Kittman, Juergen Berwein, Harald Baumeister

Abstract: LINC--NIRVANA (LN) is an MCAO module currently mounted on the Rear Bent Gregorian focus of the Large Binocular Telescope (LBT). It mounts a camera originally designed to realize the interferometric imaging focal station of the telescopes. LN follows the LBT binocular strategy having two twin channels: a double Layer Oriented Multi-Conjugate Adaptive Optics system assisting the two arms, supplies h… ▽ More LINC--NIRVANA (LN) is an MCAO module currently mounted on the Rear Bent Gregorian focus of the Large Binocular Telescope (LBT). It mounts a camera originally designed to realize the interferometric imaging focal station of the telescopes. LN follows the LBT binocular strategy having two twin channels: a double Layer Oriented Multi-Conjugate Adaptive Optics system assisting the two arms, supplies high order wave-front correction. In order to counterbalance the field rotation, a mechanical derotation is applied for the two ground wave-front sensors, and an optical (K-mirror) one for the two high layers sensors, fixing the positions of the focal planes with respect to the pyramids aboard the wavefront sensors. The derotation introduces a pupil images rotation on the wavefront sensors, changing the projection of the deformable mirrors on the sensor consequently. △ Less

Submitted 4 July, 2018; originally announced July 2018.

Comments: 9 pages, 6 figures, proceeding of the SPIE Astronomical Telescopes + Instrumentation meeting, Conference Adaptive Optics Systems VI held in Austin Convention Center, Austin, Texas, United States, 10 - 15 June 2018

Pathfinder first light: alignment, calibration, and commissioning of the LINC-NIRVANA ground-layer adaptive optics subsystem

Authors: Derek Kopon, Al Conrad, Carmelo Arcidiacono, Tom Herbst, Valentina Viotto, Jacopo Farinato, Maria Bergomi, Roberto Ragazzoni, Luca Marafatto, Harald Baumeister, Thomas Bertram, Jürgen Berwein, Florian Briegel, Ralph Hofferbert, Frank Kittmann, Martin Kürster, Lars Mohr, Kalyan Radhakrishnan

Abstract: We present descriptions of the alignment and calibration tests of the Pathfinder, which achieved first light during our 2013 commissioning campaign at the LBT. The full LINC-NIRVANA instrument is a Fizeau interferometric imager with fringe tracking and 2-layer natural guide star multi-conjugate adaptive optics (MCAO) systems on each eye of the LBT. The MCAO correction for each side is achieved usi… ▽ More We present descriptions of the alignment and calibration tests of the Pathfinder, which achieved first light during our 2013 commissioning campaign at the LBT. The full LINC-NIRVANA instrument is a Fizeau interferometric imager with fringe tracking and 2-layer natural guide star multi-conjugate adaptive optics (MCAO) systems on each eye of the LBT. The MCAO correction for each side is achieved using a ground layer wavefront sensor that drives the LBT adaptive secondary mirror and a mid-high layer wavefront sensor that drives a Xinetics 349 actuator DM conjugated to an altitude of 7.1 km. When the LINC-NIRVANA MCAO system is commissioned, it will be one of only two such systems on an 8-meter telescope and the only such system in the northern hemisphere. In order to mitigate risk, we take a modular approach to commissioning by decoupling and testing the LINC-NIRVANA subsystems individually. The Pathfinder is the ground-layer wavefront sensor for the DX eye of the LBT. It uses 12 pyramid wavefront sensors to optically co-add light from natural guide stars in order to make four pupil images that sense ground layer turbulence. Pathfinder is now the first LINC-NIRVANA subsystem to be fully integrated with the telescope and commissioned on sky. Our 2013 commissioning campaign consisted of 7 runs at the LBT with the tasks of assembly, integration and communication with the LBT telescope control system, alignment to the telescope optical axis, off-sky closed loop AO calibration, and finally closed loop on-sky AO. We present the programmatics of this campaign, along with the novel designs of our alignment scheme and our off-sky calibration test, which lead to the Pathfinder's first on-sky closed loop images. △ Less

Submitted 22 July, 2014; originally announced July 2014.

Journal ref: Proceedings of the SPIE, Astronomical Instrumentation 2014

Laboratory Results and Status Update for Pathfinder at LBT, The LINC-NIRVANA NGS Ground- Layer AO Subsystem

Authors: Derek Kopon, Al Conrad, Thomas Bertram, Tom Herbst, Martin Kürster, Jürgen Berwein, Roberto Ragazzoni, Jacopo Farinato, Valentina Viotto, Maria Bergomi, Ralf-Rainer Rohloff, Harald Baumeister, Fulvio De Bonis, Ralph Hofferbert, Alessandro Brunelli, Carmelo Arcidiacono, Jorg-Uwe Pott, Peter Bizenberger, Florian Briegel, Daniel Meschke, Lars Mohr, Xianyu Zhang, Frank Kittmann

Abstract: The full LINC-NIRVANA instrument will be one of the most complex ground-based astronomical systems ever built. It will consist of multiple subsystems, including two multi-conjugate ground layer AO systems (MCAO) that drive the LBT adaptive secondaries, two mid-high layer AO systems with their own Xynetics 349 actuator DM's , a fringe tracker, a beam combiner, and the NIR science camera. In order t… ▽ More The full LINC-NIRVANA instrument will be one of the most complex ground-based astronomical systems ever built. It will consist of multiple subsystems, including two multi-conjugate ground layer AO systems (MCAO) that drive the LBT adaptive secondaries, two mid-high layer AO systems with their own Xynetics 349 actuator DM's , a fringe tracker, a beam combiner, and the NIR science camera. In order to mitigate risk, we take a modular approach to instrument testing and commissioning by decoupling these subsystems individually. The first subsystem tested on-sky will be one of the ground-layer AO systems, part of a test-bed known as the Pathfinder. The Pathfinder consists of a 12-star pyramid wavefront sensor (PWFS) that drives one of the LBT's adaptive secondaries, a support structure known as "The Foot," and the infrared test camera (IRTC), which is used for acquisition and alignment. The 12 natural guide stars are acquired by moveable arms called "star enlargers," each of which contains its own optical path. The Pathfinder was shipped from MPIA in Heidelberg, Germany to the LBT mountain lab on Mt. Graham, Arizona in February 2013. The system was unpacked, assembled in the LBT clean room, and internally optically aligned. We present the results of our system tests, including star enlarger alignment and system alignment. We also present our immediate plans for on-sky closed loop tests on the LBT scheduled for late Fall. Because plans for all ELTs call for ground layer correction, the Pathfinder provides valuable preliminary information not only for the full LINC-NIRVANA system, but also for future advanced MCAO systems. △ Less

Submitted 22 July, 2014; originally announced July 2014.

Comments: 2013 AO4ELT Proceedings

Numerical control matrix rotation for the LINC-NIRVANA Multi-Conjugate Adaptive Optics system

Authors: Carmelo Arcidiacono, Thomas Bertram, Roberto Ragazzoni, Jacopo Farinato, Simone Esposito, Armando Riccardi, Enrico Pinna, Alfio Puglisi, Luca Fini, Marco Xompero, Lorenzo Busoni, Fernando Quiros-Pacheco, Runa Briguglio

Abstract: LINC-NIRVANA will realize the interferometric imaging focal station of the Large Binocular Telescope. A double Layer Oriented multi-conjugate adaptive optics system assists the two arms of the interferometer, supplying high order wave-front correction. In order to counterbalance the field rotation, mechanical derotation for the two ground wave-front sensors, and optical derotators for the mid-high… ▽ More LINC-NIRVANA will realize the interferometric imaging focal station of the Large Binocular Telescope. A double Layer Oriented multi-conjugate adaptive optics system assists the two arms of the interferometer, supplying high order wave-front correction. In order to counterbalance the field rotation, mechanical derotation for the two ground wave-front sensors, and optical derotators for the mid-high layers sensors fix the positions of the focal planes with respect to the pyramids aboard the wave-front sensors. The derotation introduces pupil images rotation on the wavefront sensors: the projection of the deformable mirrors on the sensor consequently change. The proper adjustment of the control matrix will be applied in real-time through numerical computation of the new matrix. In this paper we investigate the temporal and computational aspects related to the pupils rotation, explicitly computing the wave-front errors that may be generated. △ Less

Submitted 26 August, 2010; originally announced August 2010.

Comments: 6 pages, 2 figures, presented at SPIE Symposium "Astronomical Telescopes and Instrumentation'' conference "Adaptive Optics Systems II'',Sunday 27 June 2010, San Diego, California, USA

Journal ref: Proc. of SPIE 7736 (2010) 77364J

Simultaneous NIR/sub-mm observation of flare emission from SgrA*

Authors: A. Eckart, R. Schoedel, M. Garcia-Marin, G. Witzel, A. Weiss, F. K. Baganoff, M. R. Morris, T. Bertram, M. Dovciak, W. J. Duschl, V. Karas, S. Koenig, T. P. Krichbaum, M. Krips, D. Kunneriath, R. S. Lu, S. Markoff, J. Mauerhan, L. Meyer, J. Moultaka, K. Muzic, F. Najarro, J. -U. Pott, K. F. Schuster, L. O. Sjouwerman , et al. (6 additional authors not shown)

Abstract: We report on a successful, simultaneous observation and modeling of the sub-millimeter to near-infrared flare emission of the Sgr A* counterpart associated with the super-massive black hole at the Galactic center. Our modeling is based on simultaneous observations that have been carried out on 03 June, 2008 using the NACO adaptive optics (AO) instrument at the ESO VLT and the LABOCA bolometer at… ▽ More We report on a successful, simultaneous observation and modeling of the sub-millimeter to near-infrared flare emission of the Sgr A* counterpart associated with the super-massive black hole at the Galactic center. Our modeling is based on simultaneous observations that have been carried out on 03 June, 2008 using the NACO adaptive optics (AO) instrument at the ESO VLT and the LABOCA bolometer at the APEX telescope. Inspection and modeling of the light curves show that the sub-mm follows the NIR emission with a delay of 1.5+/-0.5 hours. We explain the flare emission delay by an adiabatic expansion of the source components. △ Less

Submitted 17 November, 2008; originally announced November 2008.

Comments: 12 pages, 9 figures, 3 tables, in press with A&A

Coordinated mm/sub-mm observations of Sagittarius A* in May 2007

Authors: D. Kunneriath, A. Eckart, S. Vogel, L. Sjouwerman, H. Wiesemeyer, R. Schoedel, F. K. Baganoff, M. Morris, T. Bertram, M. Dovciak, D. Downes, W. J. Duschl, V. Karas, S. Koenig, T. Krichbaum, M. Krips, R. -S. Lu, S. Markoff, J. Mauerhan, L. Meyer, J. Moultaka, K. Muzic, F. Najarro, K. Schuster, C. Straubmeier , et al. (4 additional authors not shown)

Abstract: At the center of the Milky Way, with a distance of ~8 kpc, the compact source Sagittarius A* (SgrA*) can be associated with a super massive black hole of ~4x10^6 solar masses. SgrA* shows strong variability from the radio to the X-ray wavelength domains. Here we report on simultaneous NIR/sub-millimeter/X-ray observations from May 2007 that involved the NACO adaptive optics (AO) instrument at th… ▽ More At the center of the Milky Way, with a distance of ~8 kpc, the compact source Sagittarius A* (SgrA*) can be associated with a super massive black hole of ~4x10^6 solar masses. SgrA* shows strong variability from the radio to the X-ray wavelength domains. Here we report on simultaneous NIR/sub-millimeter/X-ray observations from May 2007 that involved the NACO adaptive optics (AO) instrument at the European Southern Observatory's Very Large Telescope, the Australian Telescope Compact Array (ATCA), the US mm-array CARMA, the IRAM 30m mm-telescope, and other telescopes. We concentrate on the time series of mm/sub-mm data from CARMA, ATCA, and the MAMBO bolometer at the IRAM 30m telescope. △ Less

Submitted 1 October, 2008; originally announced October 2008.

Comments: 7 pages, 3 figures, contribution for the conference "The Universe under the Microscope" (AHAR 2008), to be published in Journal of Physics: Conference Series by Institute of Physics Publishing

Journal ref: J.Phys.Conf.Ser.131:012006,2008

Coordinated multi-wavelength observations of Sgr A*

Authors: A. Eckart, R. Schoedel, F. K. Baganoff, M. Morris, T. Bertram, M. Dovciak, D. Downes, W. J. Duschl, V. Karas, S. Koenig, T. Krichbaum, M. Krips, D. Kunneriath, R-S. Lu, S. Markoff, J. Mauerhan, L. Meyer, J. Moultaka, K. Muzic, F. Najarro, K. Schuster, L. Sjouwerman, C. Straubmeier, C. Thum, S. Vogel , et al. (4 additional authors not shown)

Abstract: We report on recent near-infrared (NIR) and X-ray observations of Sagittarius A* (Sgr A*), the electromagnetic manifestation of the ~4x10^6 solar masses super-massive black hole (SMBH) at the Galactic Center. The goal of these coordinated multi-wavelength observations is to investigate the variable emission from Sgr A* in order to obtain a better understanding of the underlying physical processe… ▽ More We report on recent near-infrared (NIR) and X-ray observations of Sagittarius A* (Sgr A*), the electromagnetic manifestation of the ~4x10^6 solar masses super-massive black hole (SMBH) at the Galactic Center. The goal of these coordinated multi-wavelength observations is to investigate the variable emission from Sgr A* in order to obtain a better understanding of the underlying physical processes in the accretion flow/outflow. The observations have been carried out using the NACO adaptive optics (AO) instrument at the European Southern Observatory's Very Large Telescope (July 2005, May 2007) and the ACIS-I instrument aboard the Chandra X-ray Observatory (July 2005). We report on a polarized NIR flare synchronous to a 8x1033 erg/s X-ray flare in July 2005, and a further flare in May 2007 that shows the highest sub-flare to flare contrast observed until now. The observations can be interpreted in the framework of a model involving a temporary disk with a short jet. In the disk component flux density variations can be explained due to hot spots on relativistic orbits around the central SMBH. The variations of the sub-structures of the May 2007 flare are interpreted as a variation of the hot spot structure due to differential rotation within the disk. △ Less

Submitted 1 October, 2008; originally announced October 2008.

Comments: 15 pages, 7 figures, contribution for the conference "The Universe under the Microscope" (AHAR 2008), to be published in Journal of Physics: Conference Series by Institute of Physics Publishing

Journal ref: J.Phys.Conf.Ser.131:012002,2008

Molecular gas in nearby low-luminosity QSO host galaxies

Authors: T. Bertram, A. Eckart, S. Fischer, J. Zuther, C. Straubmeier, L. Wisotzki, M. Krips

Abstract: This paper addresses the global molecular gas properties of a representative sample of galaxies hosting low-luminosity quasistellar objects. An abundant supply of gas is necessary to fuel both the active galactic nucleus and any circum-nuclear starburst activity of QSOs. We selected a sample of nearby low-luminosity QSO host galaxies that is free of infrared excess biases. All objects are draw… ▽ More This paper addresses the global molecular gas properties of a representative sample of galaxies hosting low-luminosity quasistellar objects. An abundant supply of gas is necessary to fuel both the active galactic nucleus and any circum-nuclear starburst activity of QSOs. We selected a sample of nearby low-luminosity QSO host galaxies that is free of infrared excess biases. All objects are drawn from the Hamburg-ESO survey for bright UV-excess QSOs, have DEC>-30 degrees and redshifts that do not exceed z=0.06. The IRAM 30m telescope was used to measure the CO(1-0) and CO(2-1) transition in parallel. 27 out of 39 galaxies in the sample have been detected. The molecular gas masses of the detected sources range from 0.4E9 M_sun to 9.7E9 M_sun. We can confirm that the majority of galaxies hosting low-luminosity QSOs are rich in molecular gas. The properties of galaxies hosting brighter type I AGN and circumnuclear starformation regions differ from the properties of galaxies with fainter central regions. The overall supply of molecular gas and the spread of the line width distribution is larger. When comparing the far-infrared with the CO luminosities, the distribution can be separated into two different power-laws: one describing the lower activity Seyfert I population and the second describing the luminous QSO population. The separation in the L_FIR/L'_CO behavior may be explainable with differing degrees of compactness of the emission regions. We provide a simple model to describe the two power-laws. The sample studied in this paper is located in a transition region between the two populations. △ Less

Submitted 19 June, 2007; originally announced June 2007.

Mrk 609: resolving the circum-nuclear structure with near-infrared integral field spectroscopy

Authors: J. Zuther, C. Iserlohe, J. -U. Pott, T. Bertram, S. Fischer, W. Voges, G. Hasinger, A. Eckart

Abstract: We present first results of near infrared J and H+K ESO-SINFONI integral field spectroscopy of the composite starburst/Seyfert 1.8 galaxy Mrk 609. The data were taken during the science verification period of SINFONI. We aim to investigate the morphology and excitation conditions within the central 2 kpc. Additional Nobeyama 45 m CO(1-0) data are presented, which we used to estimate the molecula… ▽ More We present first results of near infrared J and H+K ESO-SINFONI integral field spectroscopy of the composite starburst/Seyfert 1.8 galaxy Mrk 609. The data were taken during the science verification period of SINFONI. We aim to investigate the morphology and excitation conditions within the central 2 kpc. Additional Nobeyama 45 m CO(1-0) data are presented, which we used to estimate the molecular gas mass. The source was selected from a sample of adaptive optics suitable, SDSS/ROSAT based, X-ray bright AGN with redshifts of 0.03 < z < 1. This sample allows for a detailed study of the NIR properties of the nuclear and host environments with high spectral and spatial resolution. Our NIR data reveal a complex emission-line morphology, possibly associated with a nuclear bar seen in the reconstructed continuum images. The detections of [SiVI] and a broad Pa alpha component are clear indicators for the presence of an accreting super-massive black hole at the center of Mrk 609. In agreement with previous observations we find that the circum-nuclear emission is not significantly extincted. The analysis of the high angular resolution rotational-vibrational molecular hydrogen and forbidden [FeII] emission reveals a LINER character of the nucleus. The large H_2 gas mass deduced from the CO(1-0) observation provides the fuel needed to feed the starburst and Seyfert activity in Mrk 609. High angular resolution imaging spectroscopy provides an ideal tool to resolve the nuclear and starburst contribution in active galaxies. We show that Mrk 609 exhibits LINER features, that appear to be hidden in larger aperture visible/NIR spectra. △ Less

Submitted 16 April, 2007; v1 submitted 2 January, 2007; originally announced January 2007.

Comments: published by A&A, 19 pages, 16 figures, version with high resolution figures is available via http://www.ph1.uni-koeln.de/~zuther/mrk609.pdf

Barred CO emission in HE1029-1831

Authors: M. Krips, A. Eckart, R. Neri, T. Bertram, C. Straubmeier, S. Fischer, J. G. Staguhn, S. N. Vogel

Abstract: We present CO(1-0) and CO(2-1) line emission maps of the barred spiral active galaxy HE1029-1831 (z=0.0403) obtained with the IRAM Plateau de Bure Interferometer (PdBI) and in part by the Berkeley-Illinois-Maryland Association (BIMA) observatory. The CO emission is well associated with the optical bar and extended along it. The FWHM of the CO emission is estimated to be ~(6+-2)kpc. The CO emissi… ▽ More We present CO(1-0) and CO(2-1) line emission maps of the barred spiral active galaxy HE1029-1831 (z=0.0403) obtained with the IRAM Plateau de Bure Interferometer (PdBI) and in part by the Berkeley-Illinois-Maryland Association (BIMA) observatory. The CO emission is well associated with the optical bar and extended along it. The FWHM of the CO emission is estimated to be ~(6+-2)kpc. The CO emission shows a strong velocity gradient along the minor axis of the bar (PA=90degree). The molecular gas mass is estimated to be ~1.2x10^(10)Msun which indicates a very gas rich host galaxy. Most of the molecular gas appears to be subthermally excited and cold but we also find weak evidence for a warmer and/or denser gas component at the southern part of the bar emission, about \~4kpc from the galactic nucleus. △ Less

Submitted 11 December, 2006; originally announced December 2006.

Comments: accepted for publication in A&A; 6 pages and 7 figures

Nearby AGN and their hosts in the near infrared

Authors: S. Fischer, C. Iserlohe, J. Zuther, T. Bertram, C. Straubmeier, R. Schoedel, A. Eckart

Abstract: We present near infrared ISAAC VLT observations of nine (0.01<z<0.06) Active Galactic Nuclei selected from the Hamburg/ESO Survey and the Veron-Cetty & Veron catalog. Hydrogen recombination lines Pa$α$ and Br$γ$ are observed in seven of the nine sources of which five show a broad component. In three sources, extended 1-0S(1) rotational-vibrational molecular hydrogen emission is detected. Stellar… ▽ More We present near infrared ISAAC VLT observations of nine (0.01<z<0.06) Active Galactic Nuclei selected from the Hamburg/ESO Survey and the Veron-Cetty & Veron catalog. Hydrogen recombination lines Pa$α$ and Br$γ$ are observed in seven of the nine sources of which five show a broad component. In three sources, extended 1-0S(1) rotational-vibrational molecular hydrogen emission is detected. Stellar CO absorption is seen in four sources. In one of these objects, an upper limit of the central mass can be determined from the stellar velocity field. H- and Ks-band imaging allow us to determine the morphology class of the host galaxies. Colors (with supplementary J-band 2MASS images) show that the four galaxies with detected CO absorption are characterized by an overall strong stellar contribution. One galaxy shows an increased extinction towards the nucleus. After removal of the nuclear point source, the host galaxies show colors typical for non-active spiral galaxies. △ Less

Submitted 13 March, 2006; originally announced March 2006.

Comments: Accepted by A&A, for higher resolution version, see http://www.ph1.uni-koeln.de/~fischer/fischerAGNinNIR.pdf

Molecular gas in QSO host galaxies

Authors: T. Bertram, A. Eckart, M. Krips, C. Straubmeier, S. Fischer, J. G. Staguhn

Abstract: We present the results of a survey for CO line emission from a sample of nearby QSO hosts taken from the Hamburg/ESO survey (HES) and the Veron-Cetty and Veron quasar catalogue. From a total of 39 observed sources we clearly detected 5 objects with >10sigma signals (HE 0108-4743, HE 0224-2834, J035818.7-612407, HE 1029-1831, HE 2211-3903). Further 6 sources show marginal detections on the 2sigma… ▽ More We present the results of a survey for CO line emission from a sample of nearby QSO hosts taken from the Hamburg/ESO survey (HES) and the Veron-Cetty and Veron quasar catalogue. From a total of 39 observed sources we clearly detected 5 objects with >10sigma signals (HE 0108-4743, HE 0224-2834, J035818.7-612407, HE 1029-1831, HE 2211-3903). Further 6 sources show marginal detections on the 2sigma level. △ Less

Submitted 15 November, 2005; originally announced November 2005.

Comments: 4 pages, 1 figure, submitted to "QSO Hosts: Evolution and Environment", P.D. Barthel, D.B. Sanders, eds., August 2005, Leiden University (The Netherlands), New Astr. Rev

Journal ref: New Astron.Rev.50:712-715,2006

NIR Imaging and Spectroscopy of AGN hosts at z < 0.06

Authors: S. Fischer, C. Iserlohe, J. Zuther, T. Bertram, C. Straubmeier, A. Eckart

Abstract: We created a sample of nearby QSOs selected from the Hamburg/ESO survey and the Veron-Cetty & Veron catalog, with a limiting redshift of z<0.06, which consists of 63 objects. In this contribution, we present the results of our ISAAC Ks-band spectroscopic and H and Ks-band photometric observations of 9 sources of this sample. In seven sources we find hydrogen recombination lines Paalpha and Brgam… ▽ More We created a sample of nearby QSOs selected from the Hamburg/ESO survey and the Veron-Cetty & Veron catalog, with a limiting redshift of z<0.06, which consists of 63 objects. In this contribution, we present the results of our ISAAC Ks-band spectroscopic and H and Ks-band photometric observations of 9 sources of this sample. In seven sources we find hydrogen recombination lines Paalpha and Brgamma of which five galaxies show a broad component. In three sources, extended molecular hydrogen emission is detected in the 1-0S(1) line. The stellar CO-absorption feature is only detectable in 5 objects, those sources with J-H and H-K colors closest to those of ordinary galaxies. △ Less

Submitted 3 November, 2005; originally announced November 2005.

Comments: 4 pages, 3 figures, submitted to "QSO Hosts: Evolution and Environment", P.D. Barthel, D.B. Sanders, eds., August 2005, Leiden University (The Netherlands), New Astr. Rev

Journal ref: New Astron.Rev.50:736-738,2006

Molecular gas in the galaxy cluster Abell 262. CO observations of UGC 1347 and other galaxies of the cluster

Authors: T. Bertram, A. Eckart, M. Krips, J. G. Staguhn, W. Hackenberg

Abstract: We present millimeter CO line emission observations of 12 galaxies within the Abell 262 cluster, together with L_FIR data, in the context of a possible molecular gas deficiency within the region of the cluster center. Several indications of the presence of such a deficiency are highlighted and connected to a model of cirrus-like cloud strip**. The model predicts a drop in the average 100 micro… ▽ More We present millimeter CO line emission observations of 12 galaxies within the Abell 262 cluster, together with L_FIR data, in the context of a possible molecular gas deficiency within the region of the cluster center. Several indications of the presence of such a deficiency are highlighted and connected to a model of cirrus-like cloud strip**. The model predicts a drop in the average 100 micron flux density of galaxies in the core of the cluster compared to the average 100 micron flux density in the outer regions, which is actually indicated in the IRAS data of the cluster members. This drop is explained by the decrease in the total hydrogen column density N(H) and, therefore, also includes a decrease in the molecular gas content. In addition to results for the global CO content of the galaxy sample, high-resolution interferometric CO(1-0) observations of one of the cluster members, UGC 1347, exemplify the spatial distribution of the molecular gas in a galaxy of the cluster. With these observations, it was possible to confirm the existence of a bright off-nuclear CO-emission source and to derive molecular masses and line ratios for this source and the nucleus. △ Less

Submitted 18 October, 2005; originally announced October 2005.

Comments: 14 pages, 14 figures, accepted by A&A

Journal ref: Astron.Astrophys. 448 (2006) 29-42

Selection of extragalactic Targets for AO and VLTI Observations

Authors: J. Zuther, A. Eckart, W. Voges, T. Bertram, C. Straubmeier

Abstract: In this contribution we would like to emphasize the usefulness of data mining multiwavelength surveys like the Sloan Digital Sky Survey (SDSS) or the ROSAT All Sky Survey (RASS) -- which have become available to the public recently -- in order to find interesting objects suitable for adaptive optics (AO) or interferometric (VLTI) observations in the infrared. We will present a sample of extragal… ▽ More In this contribution we would like to emphasize the usefulness of data mining multiwavelength surveys like the Sloan Digital Sky Survey (SDSS) or the ROSAT All Sky Survey (RASS) -- which have become available to the public recently -- in order to find interesting objects suitable for adaptive optics (AO) or interferometric (VLTI) observations in the infrared. We will present a sample of extragalactic X-ray sources having an optical counterpart (based on SDSS data release 1) which are suitable for AO/VLTI observations using a natural guide star in their vicinity. △ Less

Submitted 29 October, 2003; originally announced October 2003.

Comments: 3 pages, 1 figure, to appear in "Science with Adaptive Optics", ed. W. Brandner and M. Kasper (Springer-Verlag, ESO Astrophysics Symposia)