Showing 1–50 of 104 results for author: Frey, J

Leveraging small language models for Text2SPARQL tasks to improve the resilience of AI assistance

Authors: Felix Brei, Johannes Frey, Lars-Peter Meyer

Abstract: In this work we will show that language models with less than one billion parameters can be used to translate natural language to SPARQL queries after fine-tuning. Using three different datasets ranging from academic to real world, we identify prerequisites that the training data must fulfill in order for the training to be successful. The goal is to empower users of semantic web technology to use… ▽ More In this work we will show that language models with less than one billion parameters can be used to translate natural language to SPARQL queries after fine-tuning. Using three different datasets ranging from academic to real world, we identify prerequisites that the training data must fulfill in order for the training to be successful. The goal is to empower users of semantic web technology to use AI assistance with affordable commodity hardware, making them more resilient against external factors. △ Less

Submitted 27 May, 2024; originally announced May 2024.

Comments: To appear in Proceedings of the Workshop on Linked Data-driven Resilience Research 2024 (D2R2) co-located with Extended Semantic Web Conference 2024 (ESWC 2024)

Autonomous Forest Inventory with Legged Robots: System Design and Field Deployment

Authors: Matías Mattamala, Nived Chebrolu, Benoit Casseau, Leonard Freißmuth, Jonas Frey, Turcan Tuna, Marco Hutter, Maurice Fallon

Abstract: We present a solution for autonomous forest inventory with a legged robotic platform. Compared to their wheeled and aerial counterparts, legged platforms offer an attractive balance of endurance and low soil impact for forest applications. In this paper, we present the complete system architecture of our forest inventory solution which includes state estimation, navigation, mission planning, and r… ▽ More We present a solution for autonomous forest inventory with a legged robotic platform. Compared to their wheeled and aerial counterparts, legged platforms offer an attractive balance of endurance and low soil impact for forest applications. In this paper, we present the complete system architecture of our forest inventory solution which includes state estimation, navigation, mission planning, and real-time tree segmentation and trait estimation. We present preliminary results for three campaigns in forests in Finland and the UK and summarize the main outcomes, lessons, and challenges. Our UK experiment at the Forest of Dean with the ANYmal D legged platform, achieved an autonomous survey of a 0.96 hectare plot in 20 min, identifying over 100 trees with typical DBH accuracy of 2 cm. △ Less

Submitted 22 April, 2024; originally announced April 2024.

Comments: Accepted to the IEEE ICRA Workshop on Field Robotics 2024

Learning with 3D rotations, a hitchhiker's guide to SO(3)

Authors: A. René Geist, Jonas Frey, Mikel Zobro, Anna Levina, Georg Martius

Abstract: Many settings in machine learning require the selection of a rotation representation. However, choosing a suitable representation from the many available options is challenging. This paper acts as a survey and guide through rotation representations. We walk through their properties that harm or benefit deep learning with gradient-based optimization. By consolidating insights from rotation-based le… ▽ More Many settings in machine learning require the selection of a rotation representation. However, choosing a suitable representation from the many available options is challenging. This paper acts as a survey and guide through rotation representations. We walk through their properties that harm or benefit deep learning with gradient-based optimization. By consolidating insights from rotation-based learning, we provide a comprehensive overview of learning functions with rotation representations. We provide guidance on selecting representations based on whether rotations are in the model's input or output and whether the data primarily comprises small angles. △ Less

Submitted 19 June, 2024; v1 submitted 17 April, 2024; originally announced April 2024.

Comments: Published at ICML 2024

Wild Visual Navigation: Fast Traversability Learning via Pre-Trained Models and Online Self-Supervision

Authors: Matías Mattamala, Jonas Frey, Piotr Libera, Nived Chebrolu, Georg Martius, Cesar Cadena, Marco Hutter, Maurice Fallon

Abstract: Natural environments such as forests and grasslands are challenging for robotic navigation because of the false perception of rigid obstacles from high grass, twigs, or bushes. In this work, we present Wild Visual Navigation (WVN), an online self-supervised learning system for visual traversability estimation. The system is able to continuously adapt from a short human demonstration in the field,… ▽ More Natural environments such as forests and grasslands are challenging for robotic navigation because of the false perception of rigid obstacles from high grass, twigs, or bushes. In this work, we present Wild Visual Navigation (WVN), an online self-supervised learning system for visual traversability estimation. The system is able to continuously adapt from a short human demonstration in the field, only using onboard sensing and computing. One of the key ideas to achieve this is the use of high-dimensional features from pre-trained self-supervised models, which implicitly encode semantic information that massively simplifies the learning task. Further, the development of an online scheme for supervision generator enables concurrent training and inference of the learned model in the wild. We demonstrate our approach through diverse real-world deployments in forests, parks, and grasslands. Our system is able to bootstrap the traversable terrain segmentation in less than 5 min of in-field training time, enabling the robot to navigate in complex, previously unseen outdoor terrains. Code: https://bit.ly/498b0CV - Project page:https://bit.ly/3M6nMHH △ Less

Submitted 10 April, 2024; originally announced April 2024.

Comments: Extended version of arXiv:2305.08510

Uniform Preorders and Partial Combinatory Algebras

Authors: Jonas Frey

Abstract: Uniform preorders are a class of combinatory representations of Set-indexed preorders that generalize Pieter Hofstra's basic relational objects. An indexed preorder is representable by a uniform preorder if and only if it has as generic predicate. We study the $\exists$-completion of indexed preorders on the level of uniform preorders, and identify a combinatory condition (called 'relational compl… ▽ More Uniform preorders are a class of combinatory representations of Set-indexed preorders that generalize Pieter Hofstra's basic relational objects. An indexed preorder is representable by a uniform preorder if and only if it has as generic predicate. We study the $\exists$-completion of indexed preorders on the level of uniform preorders, and identify a combinatory condition (called 'relational completeness') which characterizes those uniform preorders with finite meets whose $\exists$-completions are triposes. The class of triposes obtained this way contains relative realizability triposes, for which we derive a characterization as a fibrational analogue of the characterization of realizability toposes given in earlier work. Besides relative partial combinatory algebras, the class of relationally complete uniform preorders contains filtered ordered partial combinatory algebras, and it is unclear if there are any others. △ Less

Submitted 25 March, 2024; originally announced March 2024.

Comments: 21 pages

MSC Class: 03G30

Fast Generation of Feasible Trajectories in Direct Optimal Control

Authors: David Kiessling, Katrin Baumgärtner, Jonathan Frey, Wilm Decré, Jan Swevers, Moritz Diehl

Abstract: This paper examines the question of finding feasible points to discrete-time optimal control problems. The optimization problem of finding a feasible trajectory is transcribed to an unconstrained optimal control problem. An efficient algorithm, called FP-DDP, is proposed that solves the resulting problem using Differential Dynamic Programming preserving feasibility with respect to the system dynam… ▽ More This paper examines the question of finding feasible points to discrete-time optimal control problems. The optimization problem of finding a feasible trajectory is transcribed to an unconstrained optimal control problem. An efficient algorithm, called FP-DDP, is proposed that solves the resulting problem using Differential Dynamic Programming preserving feasibility with respect to the system dynamics in every iteration. Notably, FP-DDP admits global and rapid local convergence properties induced by a combination of a Levenberg-Marquardt method and an Armijo-type line search. The efficiency of FP-DDP is demonstrated against established methods such as Direct Multiple Shooting, Direct Single Shooting, and state-of-the-art solvers. △ Less

Submitted 4 July, 2024; v1 submitted 15 March, 2024; originally announced March 2024.

Advanced-Step Real-time Iterations with Four Levels -- New Error Bounds and Fast Implementation in acados

Authors: Jonathan Frey, Armin Nurkanovic, Moritz Diehl

Abstract: The Real-Time Iteration (RTI) is an online nonlinear model predictive control algorithm that performs a single Sequential Quadratic Programming (SQP) per sampling time. The algorithm is split into a preparation and a feedback phase, where the latter one performs as little computations as possible solving a single prepared quadratic program. To further improve the accuracy of this method, the Advan… ▽ More The Real-Time Iteration (RTI) is an online nonlinear model predictive control algorithm that performs a single Sequential Quadratic Programming (SQP) per sampling time. The algorithm is split into a preparation and a feedback phase, where the latter one performs as little computations as possible solving a single prepared quadratic program. To further improve the accuracy of this method, the Advanced-Step RTI (AS-RTI) performs additional Multi-Level Iterations (MLI) in the preparation phase, such as inexact or zero-order SQP iterations on a problem with a predicted state estimate. This paper extends and streamlines the existing local convergence analysis of AS-RTI, such as analyzing MLI of level A and B for the first time, and significantly simplifying the proofs for levels C and D. Moreover, this paper provides an efficient open-source implementation in acados, making it widely accessible to practitioners. △ Less

Submitted 4 July, 2024; v1 submitted 11 March, 2024; originally announced March 2024.

Comments: 6 pages, 2 figures, accepted for L-CSS

RoadRunner - Learning Traversability Estimation for Autonomous Off-road Driving

Authors: Jonas Frey, Shehryar Khattak, Manthan Patel, Deegan Atha, Julian Nubert, Curtis Padgett, Marco Hutter, Patrick Spieler

Abstract: Autonomous navigation at high speeds in off-road environments necessitates robots to comprehensively understand their surroundings using onboard sensing only. The extreme conditions posed by the off-road setting can cause degraded camera image quality due to poor lighting and motion blur, as well as limited sparse geometric information available from LiDAR sensing when driving at high speeds. In t… ▽ More Autonomous navigation at high speeds in off-road environments necessitates robots to comprehensively understand their surroundings using onboard sensing only. The extreme conditions posed by the off-road setting can cause degraded camera image quality due to poor lighting and motion blur, as well as limited sparse geometric information available from LiDAR sensing when driving at high speeds. In this work, we present RoadRunner, a novel framework capable of predicting terrain traversability and an elevation map directly from camera and LiDAR sensor inputs. RoadRunner enables reliable autonomous navigation, by fusing sensory information, handling of uncertainty, and generation of contextually informed predictions about the geometry and traversability of the terrain while operating at low latency. In contrast to existing methods relying on classifying handcrafted semantic classes and using heuristics to predict traversability costs, our method is trained end-to-end in a self-supervised fashion. The RoadRunner network architecture builds upon popular sensor fusion network architectures from the autonomous driving domain, which embed LiDAR and camera information into a common Bird's Eye View perspective. Training is enabled by utilizing an existing traversability estimation stack to generate training data in hindsight in a scalable manner from real-world off-road driving datasets. Furthermore, RoadRunner improves the system latency by a factor of roughly 4, from 500 ms to 140 ms, while improving the accuracy for traversability costs and elevation map predictions. We demonstrate the effectiveness of RoadRunner in enabling safe and reliable off-road navigation at high speeds in multiple real-world driving scenarios through unstructured desert environments. △ Less

Submitted 3 March, 2024; v1 submitted 29 February, 2024; originally announced February 2024.

Comments: under review for Field Robotics

Efficient Zero-Order Robust Optimization for Real-Time Model Predictive Control with acados

Authors: Jonathan Frey, Yunfan Gao, Florian Messerer, Amon Lahr, Melanie Zeilinger, Moritz Diehl

Abstract: Robust and stochastic optimal control problem (OCP) formulations allow a systematic treatment of uncertainty, but are typically associated with a high computational cost. The recently proposed zero-order robust optimization (zoRO) algorithm mitigates the computational cost of uncertainty-aware MPC by propagating the uncertainties outside of the MPC problem. This paper details the combination of zo… ▽ More Robust and stochastic optimal control problem (OCP) formulations allow a systematic treatment of uncertainty, but are typically associated with a high computational cost. The recently proposed zero-order robust optimization (zoRO) algorithm mitigates the computational cost of uncertainty-aware MPC by propagating the uncertainties outside of the MPC problem. This paper details the combination of zoRO with the real-time iteration (RTI) scheme and presents an efficient open-source implementation in acados, utilizing BLASFEO for the linear algebra operations. In addition to the scaling advantages posed by the zoRO algorithm, the efficient implementation drastically reduces the computational overhead, and, combined with an RTI scheme, enables the use of tube-based MPC for a wider range of applications. The flexibility, usability and effectiveness of the proposed implementation is demonstrated on two examples. On the practical example of a differential drive robot, the proposed implementation results in a tenfold reduction of computation time with respect to the previously available zoRO implementation. △ Less

Submitted 8 November, 2023; originally announced November 2023.

Comments: 7 pages, 4 figures, submitted to ECC 2024

Gauss-Newton Runge-Kutta Integration for Efficient Discretization of Optimal Control Problems with Long Horizons and Least-Squares Costs

Authors: Jonathan Frey, Katrin Baumgärtner, Moritz Diehl

Abstract: This work proposes an efficient treatment of continuous-time optimal control problem (OCP) with long horizons and nonlinear least-squares costs. The Gauss-Newton Runge-Kutta (GNRK) integrator is presented which provides a high-order cost integration. Crucially, the Hessian of the cost terms required within an SQP-type algorithm is approximated with a Gauss-Newton Hessian. Moreover, L2 penalty form… ▽ More This work proposes an efficient treatment of continuous-time optimal control problem (OCP) with long horizons and nonlinear least-squares costs. The Gauss-Newton Runge-Kutta (GNRK) integrator is presented which provides a high-order cost integration. Crucially, the Hessian of the cost terms required within an SQP-type algorithm is approximated with a Gauss-Newton Hessian. Moreover, L2 penalty formulations for constraints are shown to be particularly effective for optimization with GNRK. An efficient implementation of GNRK is provided in the open-source software framework acados. We demonstrate the effectiveness of the proposed approach and its implementation on an illustrative example showing a reduction of relative suboptimality by a factor greater than 10 while increasing the runtime by only 10 %. △ Less

Submitted 31 October, 2023; originally announced October 2023.

Comments: 7 pages, 3 Figures, submitted to ECC 2024

Resilient Legged Local Navigation: Learning to Traverse with Compromised Perception End-to-End

Authors: ** **, Chong Zhang, Jonas Frey, Nikita Rudin, Matias Mattamala, Cesar Cadena, Marco Hutter

Abstract: Autonomous robots must navigate reliably in unknown environments even under compromised exteroceptive perception, or perception failures. Such failures often occur when harsh environments lead to degraded sensing, or when the perception algorithm misinterprets the scene due to limited generalization. In this paper, we model perception failures as invisible obstacles and pits, and train a reinforce… ▽ More Autonomous robots must navigate reliably in unknown environments even under compromised exteroceptive perception, or perception failures. Such failures often occur when harsh environments lead to degraded sensing, or when the perception algorithm misinterprets the scene due to limited generalization. In this paper, we model perception failures as invisible obstacles and pits, and train a reinforcement learning (RL) based local navigation policy to guide our legged robot. Unlike previous works relying on heuristics and anomaly detection to update navigational information, we train our navigation policy to reconstruct the environment information in the latent space from corrupted perception and react to perception failures end-to-end. To this end, we incorporate both proprioception and exteroception into our policy inputs, thereby enabling the policy to sense collisions on different body parts and pits, prompting corresponding reactions. We validate our approach in simulation and on the real quadruped robot ANYmal running in real-time (<10 ms CPU inference). In a quantitative comparison with existing heuristic-based locally reactive planners, our policy increases the success rate over 30% when facing perception failures. Project Page: https://bit.ly/45NBTuh. △ Less

Submitted 5 October, 2023; originally announced October 2023.

Comments: Website and videos are available at our Project Page: https://bit.ly/45NBTuh

Benchmarking the Abilities of Large Language Models for RDF Knowledge Graph Creation and Comprehension: How Well Do LLMs Speak Turtle?

Authors: Johannes Frey, Lars-Peter Meyer, Natanael Arndt, Felix Brei, Kirill Bulert

Abstract: Large Language Models (LLMs) are advancing at a rapid pace, with significant improvements at natural language processing and coding tasks. Yet, their ability to work with formal languages representing data, specifically within the realm of knowledge graph engineering, remains under-investigated. To evaluate the proficiency of various LLMs, we created a set of five tasks that probe their ability to… ▽ More Large Language Models (LLMs) are advancing at a rapid pace, with significant improvements at natural language processing and coding tasks. Yet, their ability to work with formal languages representing data, specifically within the realm of knowledge graph engineering, remains under-investigated. To evaluate the proficiency of various LLMs, we created a set of five tasks that probe their ability to parse, understand, analyze, and create knowledge graphs serialized in Turtle syntax. These tasks, each embodying distinct degrees of complexity and being able to scale with the size of the problem, have been integrated into our automated evaluation system, the LLM-KG-Bench. The evaluation encompassed four commercially available LLMs - GPT-3.5, GPT-4, Claude 1.3, and Claude 2.0, as well as two freely accessible offline models, GPT4All Vicuna and GPT4All Falcon 13B. This analysis offers an in-depth understanding of the strengths and shortcomings of LLMs in relation to their application within RDF knowledge graph engineering workflows utilizing Turtle representation. While our findings show that the latest commercial models outperform their forerunners in terms of proficiency with the Turtle language, they also reveal an apparent weakness. These models fall short when it comes to adhering strictly to the output formatting constraints, a crucial requirement in this context. △ Less

Submitted 29 September, 2023; originally announced September 2023.

Comments: accepted for proceedings of DL4KG Workshop @ ISWC 2023 at ceur-ws.org

MEM: Multi-Modal Elevation Map** for Robotics and Learning

Authors: Gian Erni, Jonas Frey, Takahiro Miki, Matias Mattamala, Marco Hutter

Abstract: Elevation maps are commonly used to represent the environment of mobile robots and are instrumental for locomotion and navigation tasks. However, pure geometric information is insufficient for many field applications that require appearance or semantic information, which limits their applicability to other platforms or domains. In this work, we extend a 2.5D robot-centric elevation map** framewo… ▽ More Elevation maps are commonly used to represent the environment of mobile robots and are instrumental for locomotion and navigation tasks. However, pure geometric information is insufficient for many field applications that require appearance or semantic information, which limits their applicability to other platforms or domains. In this work, we extend a 2.5D robot-centric elevation map** framework by fusing multi-modal information from multiple sources into a popular map representation. The framework allows inputting data contained in point clouds or images in a unified manner. To manage the different nature of the data, we also present a set of fusion algorithms that can be selected based on the information type and user requirements. Our system is designed to run on the GPU, making it real-time capable for various robotic and learning tasks. We demonstrate the capabilities of our framework by deploying it on multiple robots with varying sensor configurations and showcasing a range of applications that utilize multi-modal layers, including line detection, human detection, and colorization. △ Less

Submitted 28 September, 2023; originally announced September 2023.

Comments: Accapted for IROS2023. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Learning Risk-Aware Quadrupedal Locomotion using Distributional Reinforcement Learning

Authors: Lukas Schneider, Jonas Frey, Takahiro Miki, Marco Hutter

Abstract: Deployment in hazardous environments requires robots to understand the risks associated with their actions and movements to prevent accidents. Despite its importance, these risks are not explicitly modeled by currently deployed locomotion controllers for legged robots. In this work, we propose a risk sensitive locomotion training method employing distributional reinforcement learning to consider s… ▽ More Deployment in hazardous environments requires robots to understand the risks associated with their actions and movements to prevent accidents. Despite its importance, these risks are not explicitly modeled by currently deployed locomotion controllers for legged robots. In this work, we propose a risk sensitive locomotion training method employing distributional reinforcement learning to consider safety explicitly. Instead of relying on a value expectation, we estimate the complete value distribution to account for uncertainty in the robot's interaction with the environment. The value distribution is consumed by a risk metric to extract risk sensitive value estimates. These are integrated into Proximal Policy Optimization (PPO) to derive our method, Distributional Proximal Policy Optimization (DPPO). The risk preference, ranging from risk-averse to risk-seeking, can be controlled by a single parameter, which enables to adjust the robot's behavior dynamically. Importantly, our approach removes the need for additional reward function tuning to achieve risk sensitivity. We show emergent risk sensitive locomotion behavior in simulation and on the quadrupedal robot ANYmal. Videos of the experiments and code are available at https://sites.google.com/leggedrobotics.com/risk-aware-locomotion. △ Less

Submitted 3 May, 2024; v1 submitted 25 September, 2023; originally announced September 2023.

Develo** a Scalable Benchmark for Assessing Large Language Models in Knowledge Graph Engineering

Authors: Lars-Peter Meyer, Johannes Frey, Kurt Junghanns, Felix Brei, Kirill Bulert, Sabine Gründer-Fahrer, Michael Martin

Abstract: As the field of Large Language Models (LLMs) evolves at an accelerated pace, the critical need to assess and monitor their performance emerges. We introduce a benchmarking framework focused on knowledge graph engineering (KGE) accompanied by three challenges addressing syntax and error correction, facts extraction and dataset generation. We show that while being a useful tool, LLMs are yet unfit t… ▽ More As the field of Large Language Models (LLMs) evolves at an accelerated pace, the critical need to assess and monitor their performance emerges. We introduce a benchmarking framework focused on knowledge graph engineering (KGE) accompanied by three challenges addressing syntax and error correction, facts extraction and dataset generation. We show that while being a useful tool, LLMs are yet unfit to assist in knowledge graph generation with zero-shot prompting. Consequently, our LLM-KG-Bench framework provides automatic evaluation and storage of LLM responses as well as statistical data and visualization tools to support tracking of prompt engineering and model performance. △ Less

Submitted 31 August, 2023; originally announced August 2023.

Comments: To be published in SEMANTICS 2023 poster track proceedings. SEMANTICS 2023 EU: 19th International Conference on Semantic Systems, September 20-22, 2023, Leipzig, Germany

Duality for Clans: an Extension of Gabriel-Ulmer Duality

Authors: Jonas Frey

Abstract: Clans are representations of generalized algebraic theories that contain more information than the finite-limit categories associated to the locally finitely presentable categories of models via Gabriel-Ulmer duality. Extending Gabriel-Ulmer duality to account for this additional information, we present a duality theory between clans and locally finitely presentable categories equipped with a weak… ▽ More Clans are representations of generalized algebraic theories that contain more information than the finite-limit categories associated to the locally finitely presentable categories of models via Gabriel-Ulmer duality. Extending Gabriel-Ulmer duality to account for this additional information, we present a duality theory between clans and locally finitely presentable categories equipped with a weak factorization system of a certain kind. △ Less

Submitted 29 October, 2023; v1 submitted 23 August, 2023; originally announced August 2023.

Comments: 38 pages

MSC Class: 03G30; 03B38

The Future of Fundamental Science Led by Generative Closed-Loop Artificial Intelligence

Authors: Hector Zenil, Jesper Tegnér, Felipe S. Abrahão, Alexander Lavin, Vipin Kumar, Jeremy G. Frey, Adrian Weller, Larisa Soldatova, Alan R. Bundy, Nicholas R. Jennings, Koichi Takahashi, Lawrence Hunter, Saso Dzeroski, Andrew Briggs, Frederick D. Gregory, Carla P. Gomes, Jon Rowe, James Evans, Hiroaki Kitano, Ross King

Abstract: Recent advances in machine learning and AI, including Generative AI and LLMs, are disrupting technological innovation, product development, and society as a whole. AI's contribution to technology can come from multiple approaches that require access to large training data sets and clear performance evaluation criteria, ranging from pattern recognition and classification to generative models. Yet,… ▽ More Recent advances in machine learning and AI, including Generative AI and LLMs, are disrupting technological innovation, product development, and society as a whole. AI's contribution to technology can come from multiple approaches that require access to large training data sets and clear performance evaluation criteria, ranging from pattern recognition and classification to generative models. Yet, AI has contributed less to fundamental science in part because large data sets of high-quality data for scientific practice and model discovery are more difficult to access. Generative AI, in general, and Large Language Models in particular, may represent an opportunity to augment and accelerate the scientific discovery of fundamental deep science with quantitative models. Here we explore and investigate aspects of an AI-driven, automated, closed-loop approach to scientific discovery, including self-driven hypothesis generation and open-ended autonomous exploration of the hypothesis space. Integrating AI-driven automation into the practice of science would mitigate current problems, including the replication of findings, systematic production of data, and ultimately democratisation of the scientific process. Realising these possibilities requires a vision for augmented AI coupled with a diversity of AI approaches able to deal with fundamental aspects of causality analysis and model discovery while enabling unbiased search across the space of putative explanations. These advances hold the promise to unleash AI's potential for searching and discovering the fundamental structure of our world beyond what human scientists have been able to achieve. Such a vision would push the boundaries of new fundamental science rather than automatize current workflows and instead open doors for technological innovation to tackle some of the greatest challenges facing humanity today. △ Less

Submitted 29 August, 2023; v1 submitted 9 July, 2023; originally announced July 2023.

Comments: 35 pages, first draft of the final report from the Alan Turing Institute on AI for Scientific Discovery

LLM-assisted Knowledge Graph Engineering: Experiments with ChatGPT

Authors: Lars-Peter Meyer, Claus Stadler, Johannes Frey, Norman Radtke, Kurt Junghanns, Roy Meissner, Gordian Dziwis, Kirill Bulert, Michael Martin

Abstract: Knowledge Graphs (KG) provide us with a structured, flexible, transparent, cross-system, and collaborative way of organizing our knowledge and data across various domains in society and industrial as well as scientific disciplines. KGs surpass any other form of representation in terms of effectiveness. However, Knowledge Graph Engineering (KGE) requires in-depth experiences of graph structures, we… ▽ More Knowledge Graphs (KG) provide us with a structured, flexible, transparent, cross-system, and collaborative way of organizing our knowledge and data across various domains in society and industrial as well as scientific disciplines. KGs surpass any other form of representation in terms of effectiveness. However, Knowledge Graph Engineering (KGE) requires in-depth experiences of graph structures, web technologies, existing models and vocabularies, rule sets, logic, as well as best practices. It also demands a significant amount of work. Considering the advancements in large language models (LLMs) and their interfaces and applications in recent years, we have conducted comprehensive experiments with ChatGPT to explore its potential in supporting KGE. In this paper, we present a selection of these experiments and their results to demonstrate how ChatGPT can assist us in the development and management of KGs. △ Less

Submitted 13 July, 2023; originally announced July 2023.

Comments: to appear in conference proceedings of AI-Tomorrow-23, 29.+30.6.2023 in Leipzig, Germany

Journal ref: Informatik aktuell. First Working Conference on Artificial Intelligence Development for a Resilient and Sustainable Tomorrow 2023. AIDRST 2023. p. 103-115

Finite Elements with Switch Detection for Numerical Optimal Control of Nonsmooth Dynamical Systems with Set-Valued Heaviside Step Functions

Authors: Armin Nurkanović, Anton Pozharskiy, Jonathan Frey, Moritz Diehl

Abstract: This paper develops high-accuracy methods for numerically solving optimal control problems subject to nonsmooth differential equations with set-valued step functions. A notable subclass of these systems are Filippov systems. The set-valued step functions are here written as the solution map of a linear program. Using the optimality conditions of this problem we rewrite the initial nonsmooth system… ▽ More This paper develops high-accuracy methods for numerically solving optimal control problems subject to nonsmooth differential equations with set-valued step functions. A notable subclass of these systems are Filippov systems. The set-valued step functions are here written as the solution map of a linear program. Using the optimality conditions of this problem we rewrite the initial nonsmooth system into a equivalent dynamic complementarity systems (DCS). We extend the Finite Elements with Switch Detection (FESD) method [Nurkanović et al., 2024], initially developed for Filippov systems transformed via Stewart's reformulation into DCS [Stewart, 1990], to the class of nonsmooth systems with set-valued step functions. The key ideas are to start with a standard Runge-Kutta method for the obtained DCS and to let the integration step sizes to be degrees of freedom. Next, we introduce additional conditions to enable implicit but exact switch detection and to remove possible spurious degrees of freedom if no switches occur. The theoretical properties of the method are studied. Its favorable properties are illustrated on numerical simulation and optimal control examples. All methods introduced in this paper are implemented in the open-source software package NOSNOC. △ Less

Submitted 6 May, 2024; v1 submitted 7 July, 2023; originally announced July 2023.

Comments: submitted to Nonlinear Analysis: Hybrid Systems, Special Issue on Nonsmooth Dynamical Systems: Analysis, Control and Optimization

Collision-free Motion Planning for Mobile Robots by Zero-order Robust Optimization-based MPC

Authors: Yunfan Gao, Florian Messerer, Jonathan Frey, Niels van Duijkeren, Moritz Diehl

Abstract: This paper presents an implementation of robust model predictive control (MPC) for collision-free reference trajectory tracking for mobile robots. The presented approach considers the robot motion to be subject to process noise bounded by ellipsoidal sets. In order to efficiently handle the evolution of the disturbance ellipsoids within the MPC, the zero-order robust optimization (zoRO) scheme is… ▽ More This paper presents an implementation of robust model predictive control (MPC) for collision-free reference trajectory tracking for mobile robots. The presented approach considers the robot motion to be subject to process noise bounded by ellipsoidal sets. In order to efficiently handle the evolution of the disturbance ellipsoids within the MPC, the zero-order robust optimization (zoRO) scheme is applied. The idea is to fix the disturbance ellipsoids within one optimization iteration and solve the problem repeatedly with updated disturbance ellipsoid trajectories. The zero-order approach is suboptimal in general. However, we show that it does not impair convergence to the reference trajectory in the absence of obstacles. The experiments on an industrial mobile robot prototype demonstrate the performance of the controller. △ Less

Submitted 30 June, 2023; originally announced June 2023.

SMUG Planner: A Safe Multi-Goal Planner for Mobile Robots in Challenging Environments

Authors: Changan Chen, Jonas Frey, Philip Arm, Marco Hutter

Abstract: Robotic exploration or monitoring missions require mobile robots to autonomously and safely navigate between multiple target locations in potentially challenging environments. Currently, this type of multi-goal mission often relies on humans designing a set of actions for the robot to follow in the form of a path or waypoints. In this work, we consider the multi-goal problem of visiting a set of p… ▽ More Robotic exploration or monitoring missions require mobile robots to autonomously and safely navigate between multiple target locations in potentially challenging environments. Currently, this type of multi-goal mission often relies on humans designing a set of actions for the robot to follow in the form of a path or waypoints. In this work, we consider the multi-goal problem of visiting a set of pre-defined targets, each of which could be visited from multiple potential locations. To increase autonomy in these missions, we propose a safe multi-goal (SMUG) planner that generates an optimal motion path to visit those targets. To increase safety and efficiency, we propose a hierarchical state validity checking scheme, which leverages robot-specific traversability learned in simulation. We use LazyPRM* with an informed sampler to accelerate collision-free path generation. Our iterative dynamic programming algorithm enables the planner to generate a path visiting more than ten targets within seconds. Moreover, the proposed hierarchical state validity checking scheme reduces the planning time by 30% compared to pure volumetric collision checking and increases safety by avoiding high-risk regions. We deploy the SMUG planner on the quadruped robot ANYmal and show its capability to guide the robot in multi-goal missions fully autonomously on rough terrain. △ Less

Submitted 8 June, 2023; originally announced June 2023.

Two Warm Neptunes transiting HIP 9618 revealed by TESS & Cheops

Authors: Hugh P. Osborn, Grzegorz Nowak, Guillaume Hébrard, Thomas Masseron, J. Lillo-Box, Enric Pallé, Anja Bekkelien, Hans-Gustav Florén, Pascal Guterman, Attila E. Simon, V. Adibekyan, Allyson Bieryla, Luca Borsato, Alexis Brandeker, David R. Ciardi, Andrew Collier Cameron, Karen A. Collins, Jo A. Egger, Davide Gandolfi, Matthew J. Hooton, David W. Latham, Monika Lendl, Elisabeth C. Matthews, Amy Tuson, Solène Ulmer-Moll , et al. (104 additional authors not shown)

Abstract: HIP 9618 (HD 12572, TOI-1471, TIC 306263608) is a bright ($G=9.0$ mag) solar analogue. TESS photometry revealed the star to have two candidate planets with radii of $3.9 \pm 0.044$ $R_\oplus$ (HIP 9618 b) and $3.343 \pm 0.039$ $R_\oplus$ (HIP 9618 c). While the 20.77291 day period of HIP 9618 b was measured unambiguously, HIP 9618 c showed only two transits separated by a 680-day gap in the time s… ▽ More HIP 9618 (HD 12572, TOI-1471, TIC 306263608) is a bright ($G=9.0$ mag) solar analogue. TESS photometry revealed the star to have two candidate planets with radii of $3.9 \pm 0.044$ $R_\oplus$ (HIP 9618 b) and $3.343 \pm 0.039$ $R_\oplus$ (HIP 9618 c). While the 20.77291 day period of HIP 9618 b was measured unambiguously, HIP 9618 c showed only two transits separated by a 680-day gap in the time series, leaving many possibilities for the period. To solve this issue, CHEOPS performed targeted photometry of period aliases to attempt to recover the true period of planet c, and successfully determined the true period to be 52.56349 d. High-resolution spectroscopy with HARPS-N, SOPHIE and CAFE revealed a mass of $10.0 \pm 3.1 M_\oplus$ for HIP 9618 b, which, according to our interior structure models, corresponds to a $6.8\pm1.4\%$ gas fraction. HIP 9618 c appears to have a lower mass than HIP 9618 b, with a 3-sigma upper limit of $< 18M_\oplus$. Follow-up and archival RV measurements also reveal a clear long-term trend which, when combined with imaging and astrometric information, reveal a low-mass companion ($0.08^{+0.12}_{-0.05} M_\odot$) orbiting at $26^{+19}_{-11}$ au. This detection makes HIP 9618 one of only five bright ($K<8$ mag) transiting multi-planet systems known to host a planet with $P>50$ d, opening the door for the atmospheric characterisation of warm ($T_{\rm eq}<750$ K) sub-Neptunes. △ Less

Submitted 7 June, 2023; originally announced June 2023.

Comments: 19 pages, 16 figures, 9 tables. Accepted at MNRAS. CHEOPS, RV and ground-based photometric data is available on CDS at https://cdsarc.cds.unistra.fr/viz-bin/cat/J/MNRAS/523/3069

Journal ref: MNRAS, Vol. 523, 2023, issue 2, pp 3069-3089

FESD-J: Finite Elements with Switch Detection for Numerical Optimal Control of Rigid Bodies with Impacts and Coulomb Friction

Authors: Armin Nurkanović, Jonathan Frey, Anton Pozharskiy, Moritz Diehl

Abstract: The Finite Elements with Switch Detection (FESD) is a high-accuracy method for the numerical simulation and solution of optimal control problems subject to discontinuous ODEs. In this article, we extend the FESD method [Nurkanović et al., 2022] to the dynamic equations of multiple rigid bodies that exhibit state jumps due to impacts and Coulomb friction. This new method is referred to as FESD with… ▽ More The Finite Elements with Switch Detection (FESD) is a high-accuracy method for the numerical simulation and solution of optimal control problems subject to discontinuous ODEs. In this article, we extend the FESD method [Nurkanović et al., 2022] to the dynamic equations of multiple rigid bodies that exhibit state jumps due to impacts and Coulomb friction. This new method is referred to as FESD with Jumps (FESD-J). Starting from the standard Runge-Kutta equations, we let the integration step sizes be degrees of freedom. Additional constraints are introduced to ensure exact switch detection and to remove spurious degrees of freedom if no switches occur. Moreover, at the boundaries of each finite element, we impose the impact equations in their complementarity form, at both the position and velocity level. They compute the normal and tangential impulses in case of contact making. Otherwise, they are reduced to the continuity conditions for the velocities. FESD-J treats multiple contacts, where each contact can have a different coefficient of restitution and friction. All methods introduced in this paper are implemented in the open-source software package NOSNOC. We illustrate the use of FESD-J in both simulation and optimal control examples. △ Less

Submitted 26 May, 2023; originally announced May 2023.

Fast Traversability Estimation for Wild Visual Navigation

Authors: Jonas Frey, Matias Mattamala, Nived Chebrolu, Cesar Cadena, Maurice Fallon, Marco Hutter

Abstract: Natural environments such as forests and grasslands are challenging for robotic navigation because of the false perception of rigid obstacles from high grass, twigs, or bushes. In this work, we propose Wild Visual Navigation (WVN), an online self-supervised learning system for traversability estimation which uses only vision. The system is able to continuously adapt from a short human demonstratio… ▽ More Natural environments such as forests and grasslands are challenging for robotic navigation because of the false perception of rigid obstacles from high grass, twigs, or bushes. In this work, we propose Wild Visual Navigation (WVN), an online self-supervised learning system for traversability estimation which uses only vision. The system is able to continuously adapt from a short human demonstration in the field. It leverages high-dimensional features from self-supervised visual transformer models, with an online scheme for supervision generation that runs in real-time on the robot. We demonstrate the advantages of our approach with experiments and ablation studies in challenging environments in forests, parks, and grasslands. Our system is able to bootstrap the traversable terrain segmentation in less than 5 min of in-field training time, enabling the robot to navigate in complex outdoor terrains - negotiating obstacles in high grass as well as a 1.4 km footpath following. While our experiments were executed with a quadruped robot, ANYmal, the approach presented can generalize to any ground robot. △ Less

Submitted 16 May, 2023; v1 submitted 15 May, 2023; originally announced May 2023.

Comments: Accepted for Robotics: Science and Systems 2023

Seeing Through the Grass: Semantic Pointcloud Filter for Support Surface Learning

Authors: Anqiao Li, Chenyu Yang, Jonas Frey, Joonho Lee, Cesar Cadena, Marco Hutter

Abstract: Mobile ground robots require perceiving and understanding their surrounding support surface to move around autonomously and safely. The support surface is commonly estimated based on exteroceptive depth measurements, e.g., from LiDARs. However, the measured depth fails to align with the true support surface in the presence of high grass or other penetrable vegetation. In this work, we present the… ▽ More Mobile ground robots require perceiving and understanding their surrounding support surface to move around autonomously and safely. The support surface is commonly estimated based on exteroceptive depth measurements, e.g., from LiDARs. However, the measured depth fails to align with the true support surface in the presence of high grass or other penetrable vegetation. In this work, we present the Semantic Pointcloud Filter (SPF), a Convolutional Neural Network (CNN) that learns to adjust LiDAR measurements to align with the underlying support surface. The SPF is trained in a semi-self-supervised manner and takes as an input a LiDAR pointcloud and RGB image. The network predicts a binary segmentation mask that identifies the specific points requiring adjustment, along with estimating their corresponding depth values. To train the segmentation task, 300 distinct images are manually labeled into rigid and non-rigid terrain. The depth estimation task is trained in a self-supervised manner by utilizing the future footholds of the robot to estimate the support surface based on a Gaussian process. Our method can correctly adjust the support surface prior to interacting with the terrain and is extensively tested on the quadruped robot ANYmal. We show the qualitative benefits of SPF in natural environments for elevation map** and traversability estimation compared to using raw sensor measurements and existing smoothing methods. Quantitative analysis is performed in various natural environments, and an improvement by 48% RMSE is achieved within a meadow terrain. △ Less

Submitted 13 May, 2023; originally announced May 2023.

Comments: 8 pages, 9 figures

Imitation Learning from Nonlinear MPC via the Exact Q-Loss and its Gauss-Newton Approximation

Authors: Andrea Ghezzi, Jasper Hoffman, Jonathan Frey, Joschka Boedecker, Moritz Diehl

Abstract: This work presents a novel loss function for learning nonlinear Model Predictive Control policies via Imitation Learning. Standard approaches to Imitation Learning neglect information about the expert and generally adopt a loss function based on the distance between expert and learned controls. In this work, we present a loss based on the Q-function directly embedding the performance objectives an… ▽ More This work presents a novel loss function for learning nonlinear Model Predictive Control policies via Imitation Learning. Standard approaches to Imitation Learning neglect information about the expert and generally adopt a loss function based on the distance between expert and learned controls. In this work, we present a loss based on the Q-function directly embedding the performance objectives and constraint satisfaction of the associated Optimal Control Problem (OCP). However, training a Neural Network with the Q-loss requires solving the associated OCP for each new sample. To alleviate the computational burden, we derive a second Q-loss based on the Gauss-Newton approximation of the OCP resulting in a faster training time. We validate our losses against Behavioral Cloning, the standard approach to Imitation Learning, on the control of a nonlinear system with constraints. The final results show that the Q-function-based losses significantly reduce the amount of constraint violations while achieving comparable or better closed-loop costs. △ Less

Submitted 3 April, 2023; originally announced April 2023.

Comments: Submitted to Conference on Decision and Control (CDC) 2023. The paper contains 6 pages

Clamped and sideband-resolved silicon optomechanical crystals

Authors: Johan Kolvik, Paul Burger, Joey Frey, Raphaël Van Laer

Abstract: Optomechanical crystals (OMCs) are a promising and versatile platform for transduction between mechanical and optical fields. However, the release from the substrate used in conventional suspended OMCs also prevents heat-carrying noise phonons from rapidly leaking away. Thermal anchoring may be improved by attaching the OMCs directly to the substrate. Previous work towards such clamped, i.e. non-s… ▽ More Optomechanical crystals (OMCs) are a promising and versatile platform for transduction between mechanical and optical fields. However, the release from the substrate used in conventional suspended OMCs also prevents heat-carrying noise phonons from rapidly leaking away. Thermal anchoring may be improved by attaching the OMCs directly to the substrate. Previous work towards such clamped, i.e. non-suspended, OMCs suffers from weak interaction rates and insufficient lifetimes. Here, we present a new class of clamped OMCs realizing -- for the first time -- optomechanical interactions in the resolved-sideband regime required for quantum transduction. Our approach leverages high-wavevector mechanical modes outside the continuum. We observe a record zero-point optomechanical coupling rate of $g_0/(2π) \approx 0.50$ MHz along with a sevenfold improvement in the single-photon cooperativity of clamped OMCs. Our devices operate at frequencies commonly used in superconducting qubits. This opens a new avenue using clamped OMCs in both classical and quantum communications, sensing, and computation through scalable mechanical circuitry that couples strongly to light. △ Less

Submitted 31 March, 2023; originally announced March 2023.

Comments: 7 pages, 6 figures; Johan Kolvik, and Paul Burger contributed equally to this work

Journal ref: Optica Vol. 10, Issue 7, pp. 913-916 (2023)

Finite Elements with Switch Detection for Direct Optimal Control of Nonsmooth Systems with Set-Valued Step Functions

Authors: Armin Nurkanović, Jonathan Frey, Anton Pozharskiy, Moritz Diehl

Abstract: This paper extends the Finite Elements with Switch Detection (FESD) method [Nurkanović et al., 2022] to optimal control problems with nonsmooth systems involving set-valued step functions. Logical relations and common nonsmooth functions within a dynamical system can be expressed using linear and nonlinear expressions involving step functions. A prominent subclass of these systems are Filippov sys… ▽ More This paper extends the Finite Elements with Switch Detection (FESD) method [Nurkanović et al., 2022] to optimal control problems with nonsmooth systems involving set-valued step functions. Logical relations and common nonsmooth functions within a dynamical system can be expressed using linear and nonlinear expressions involving step functions. A prominent subclass of these systems are Filippov systems. The set-valued step function can be expressed by the solution map of a linear program, and using its KKT conditions allows one to transform the initial system into an equivalent dynamic complementarity system (DCS). Standard Runge-Kutta (RK) methods applied to DCS have only first-order accuracy. The FESD discretization makes the step sizes degrees of freedom and adds further constraints that ensure exact switch detection to recover the high-accuracy properties that RK methods have for smooth ODEs. We use the novel FESD method for the direct transcription of optimal control problems. All methods and examples in this paper are implemented in the open-source software package NOSNOC. △ Less

Submitted 14 August, 2023; v1 submitted 31 March, 2023; originally announced March 2023.

Comments: accepted for publication at the 62nd IEEE Conference on Decision and Control

Resonant two-laser spin-state spectroscopy of a negatively charged quantum dot-microcavity system with a cold permanent magnet

Authors: P. Steindl, T. van der Ent, H. van der Meer, J. A. Frey, J. Norman, J. E. Bowers, D. Bouwmeester, W. Löffler

Abstract: A high-efficiency spin-photon interface is an essential piece of quantum hardware necessary for various quantum technologies. Self-assembled InGaAs quantum dots have excellent optical properties, if embedded into an optical micro-cavity they can show near-deterministic spin-photon entanglement and spin readout, but an external magnetic field is required to address the individual spin states, which… ▽ More A high-efficiency spin-photon interface is an essential piece of quantum hardware necessary for various quantum technologies. Self-assembled InGaAs quantum dots have excellent optical properties, if embedded into an optical micro-cavity they can show near-deterministic spin-photon entanglement and spin readout, but an external magnetic field is required to address the individual spin states, which usually is done using a superconducting magnet. Here, we show a compact cryogenically compatible SmCo magnet design that delivers 475 mT in-plane Voigt geometry magnetic field at 5 K, which is suitable to lift the energy degeneracy of the electron spin states and trion transitions of a single InGaAs quantum dot. This quantum dot is embedded in a birefringent high-finesse optical micro-cavity which enables efficient collection of single photons emitted by the quantum dot. We demonstrate spin-state manipulation by addressing the trion transitions with a single and two laser fields. The experimental data agrees well to our model which covers single- and two-laser cross-polarized resonance fluorescence, Purcell enhancement in a birefringent cavity, and variation of the laser powers. △ Less

Submitted 5 March, 2023; originally announced March 2023.

Comments: 7 pages, 5 figures. Supp. info: 7 pages, 9 figures

Cross-polarization extinction enhancement and spin-orbit coupling of light for quantum-dot cavity-QED spectroscopy

Authors: P. Steindl, J. A. Frey, J. Norman, J. E. Bowers, D. Bouwmeester, W. Löffler

Abstract: Resonant laser spectroscopy is essential for the characterization, operation, and manipulation of single quantum systems such as semiconductor quantum dots. The separation of the weak resonance fluorescence from the excitation laser is key for high-quality single- and entangled photon sources. This is often achieved by cross-polarization laser extinction, which is limited by the quality of the opt… ▽ More Resonant laser spectroscopy is essential for the characterization, operation, and manipulation of single quantum systems such as semiconductor quantum dots. The separation of the weak resonance fluorescence from the excitation laser is key for high-quality single- and entangled photon sources. This is often achieved by cross-polarization laser extinction, which is limited by the quality of the optical elements. Recently, it was discovered that Fresnel-reflection birefringence in combination with single-mode filtering counteracting spin-orbit coupling effects enables a three-order of magnitude improvement of polarization extinction [PRX 11, 021007 (2021)]. Here, we first investigate multiple reflections and analyze beam resha**, and observe that the single-reflection extinction enhancement is optimal. We then demonstrate this method for cross-polarization extinction enhancement for a resonantly excited semiconductor quantum dot in a birefringent optical micro cavity, and observe a 10x improvement of single-photon contrast. △ Less

Submitted 10 February, 2023; originally announced February 2023.

Comments: 7 pages, 6 figures

Journal ref: Phys. Rev. Applied 19, 064082 (2023)

Frenet-Cartesian Model Representations for Automotive Obstacle Avoidance within Nonlinear MPC

Authors: Rudolf Reiter, Armin Nurkanović, Jonathan Frey, Moritz Diehl

Abstract: In recent years, nonlinear model predictive control (NMPC) has been extensively used for solving automotive motion control and planning tasks. In order to formulate the NMPC problem, different coordinate systems can be used with different advantages. We propose and compare formulations for the NMPC related optimization problem, involving a Cartesian and a Frenet coordinate frame (CCF/ FCF) in a si… ▽ More In recent years, nonlinear model predictive control (NMPC) has been extensively used for solving automotive motion control and planning tasks. In order to formulate the NMPC problem, different coordinate systems can be used with different advantages. We propose and compare formulations for the NMPC related optimization problem, involving a Cartesian and a Frenet coordinate frame (CCF/ FCF) in a single nonlinear program (NLP). We specify costs and collision avoidance constraints in the more advantageous coordinate frame, derive appropriate formulations and compare different obstacle constraints. With this approach, we exploit the simpler formulation of opponent vehicle constraints in the CCF, as well as road aligned costs and constraints related to the FCF. Comparisons to other approaches in a simulation framework highlight the advantages of the proposed approaches. △ Less

Submitted 22 December, 2022; originally announced December 2022.

Unsupervised Continual Semantic Adaptation through Neural Rendering

Authors: Zhizheng Liu, Francesco Milano, Jonas Frey, Roland Siegwart, Hermann Blum, Cesar Cadena

Abstract: An increasing amount of applications rely on data-driven models that are deployed for perception tasks across a sequence of scenes. Due to the mismatch between training and deployment data, adapting the model on the new scenes is often crucial to obtain good performance. In this work, we study continual multi-scene adaptation for the task of semantic segmentation, assuming that no ground-truth lab… ▽ More An increasing amount of applications rely on data-driven models that are deployed for perception tasks across a sequence of scenes. Due to the mismatch between training and deployment data, adapting the model on the new scenes is often crucial to obtain good performance. In this work, we study continual multi-scene adaptation for the task of semantic segmentation, assuming that no ground-truth labels are available during deployment and that performance on the previous scenes should be maintained. We propose training a Semantic-NeRF network for each scene by fusing the predictions of a segmentation model and then using the view-consistent rendered semantic labels as pseudo-labels to adapt the model. Through joint training with the segmentation model, the Semantic-NeRF model effectively enables 2D-3D knowledge transfer. Furthermore, due to its compact size, it can be stored in a long-term memory and subsequently used to render data from arbitrary viewpoints to reduce forgetting. We evaluate our approach on ScanNet, where we outperform both a voxel-based baseline and a state-of-the-art unsupervised domain adaptation method. △ Less

Submitted 24 March, 2023; v1 submitted 25 November, 2022; originally announced November 2022.

Comments: Accepted by the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023. Zhizheng Liu and Francesco Milano share first authorship. Hermann Blum and Cesar Cadena share senior authorship. 18 pages, 8 figures, 9 tables

Fast integrators with sensitivity propagation for use in CasADi

Authors: Jonathan Frey, Jochem De Schutter, Moritz Diehl

Abstract: Efficient integrators with sensitivity propagation are an essential ingredient for the numerical solution of optimal control problems. This paper gives an overview on the acados integrators, their Python interface and presents a workflow that allows using them with their sensitivities within a nonlinear programming (NLP) solver interfaced by CasADi. The implementation is discussed, demonstrated an… ▽ More Efficient integrators with sensitivity propagation are an essential ingredient for the numerical solution of optimal control problems. This paper gives an overview on the acados integrators, their Python interface and presents a workflow that allows using them with their sensitivities within a nonlinear programming (NLP) solver interfaced by CasADi. The implementation is discussed, demonstrated and provided as open-source software. The computation times of the proposed integrator and its sensitivity computation are compared to the native CasADi collocation integrator, CVODES and IDAS on different examples. A speedup of one order of magnitude for simulation and of up to three orders of magnitude for the forward sensitivity propagation is shown for an airborne wind energy system model. △ Less

Submitted 3 November, 2022; originally announced November 2022.

Active Learning of Discrete-Time Dynamics for Uncertainty-Aware Model Predictive Control

Authors: Alessandro Saviolo, Jonathan Frey, Abhishek Rathod, Moritz Diehl, Giuseppe Loianno

Abstract: Model-based control requires an accurate model of the system dynamics for precisely and safely controlling the robot in complex and dynamic environments. Moreover, in the presence of variations in the operating conditions, the model should be continuously refined to compensate for dynamics changes. In this paper, we present a self-supervised learning approach that actively models the dynamics of n… ▽ More Model-based control requires an accurate model of the system dynamics for precisely and safely controlling the robot in complex and dynamic environments. Moreover, in the presence of variations in the operating conditions, the model should be continuously refined to compensate for dynamics changes. In this paper, we present a self-supervised learning approach that actively models the dynamics of nonlinear robotic systems. We combine offline learning from past experience and online learning from current robot interaction with the unknown environment. These two ingredients enable a highly sample-efficient and adaptive learning process, capable of accurately inferring model dynamics in real-time even in operating regimes that greatly differ from the training distribution. Moreover, we design an uncertainty-aware model predictive controller that is heuristically conditioned to the aleatoric (data) uncertainty of the learned dynamics. This controller actively chooses the optimal control actions that (i) optimize the control performance and (ii) improve the efficiency of online learning sample collection. We demonstrate the effectiveness of our method through a series of challenging real-world experiments using a quadrotor system. Our approach showcases high resilience and generalization capabilities by consistently adapting to unseen flight conditions, while it significantly outperforms classical and adaptive control baselines. △ Less

Submitted 7 December, 2023; v1 submitted 22 October, 2022; originally announced October 2022.

Journal ref: IEEE Transactions on Robotics

Versatile Skill Control via Self-supervised Adversarial Imitation of Unlabeled Mixed Motions

Authors: Chenhao Li, Sebastian Blaes, Pavel Kolev, Marin Vlastelica, Jonas Frey, Georg Martius

Abstract: Learning diverse skills is one of the main challenges in robotics. To this end, imitation learning approaches have achieved impressive results. These methods require explicitly labeled datasets or assume consistent skill execution to enable learning and active control of individual behaviors, which limits their applicability. In this work, we propose a cooperative adversarial method for obtaining… ▽ More Learning diverse skills is one of the main challenges in robotics. To this end, imitation learning approaches have achieved impressive results. These methods require explicitly labeled datasets or assume consistent skill execution to enable learning and active control of individual behaviors, which limits their applicability. In this work, we propose a cooperative adversarial method for obtaining single versatile policies with controllable skill sets from unlabeled datasets containing diverse state transition patterns by maximizing their discriminability. Moreover, we show that by utilizing unsupervised skill discovery in the generative adversarial imitation learning framework, novel and useful skills emerge with successful task fulfillment. Finally, the obtained versatile policies are tested on an agile quadruped robot called Solo 8 and present faithful replications of diverse skills encoded in the demonstrations. △ Less

Submitted 11 February, 2023; v1 submitted 16 September, 2022; originally announced September 2022.

Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrations

Authors: Chenhao Li, Marin Vlastelica, Sebastian Blaes, Jonas Frey, Felix Grimminger, Georg Martius

Abstract: Learning agile skills is one of the main challenges in robotics. To this end, reinforcement learning approaches have achieved impressive results. These methods require explicit task information in terms of a reward function or an expert that can be queried in simulation to provide a target control output, which limits their applicability. In this work, we propose a generative adversarial method fo… ▽ More Learning agile skills is one of the main challenges in robotics. To this end, reinforcement learning approaches have achieved impressive results. These methods require explicit task information in terms of a reward function or an expert that can be queried in simulation to provide a target control output, which limits their applicability. In this work, we propose a generative adversarial method for inferring reward functions from partial and potentially physically incompatible demonstrations for successful skill acquirement where reference or expert demonstrations are not easily accessible. Moreover, we show that by using a Wasserstein GAN formulation and transitions from demonstrations with rough and partial information as input, we are able to extract policies that are robust and capable of imitating demonstrated behaviors. Finally, the obtained skills such as a backflip are tested on an agile quadruped robot called Solo 8 and present faithful replication of hand-held human demonstrations. △ Less

Submitted 21 November, 2022; v1 submitted 23 June, 2022; originally announced June 2022.

Locomotion Policy Guided Traversability Learning using Volumetric Representations of Complex Environments

Authors: Jonas Frey, David Hoeller, Shehryar Khattak, Marco Hutter

Abstract: Despite the progress in legged robotic locomotion, autonomous navigation in unknown environments remains an open problem. Ideally, the navigation system utilizes the full potential of the robots' locomotion capabilities while operating within safety limits under uncertainty. The robot must sense and analyze the traversability of the surrounding terrain, which depends on the hardware, locomotion co… ▽ More Despite the progress in legged robotic locomotion, autonomous navigation in unknown environments remains an open problem. Ideally, the navigation system utilizes the full potential of the robots' locomotion capabilities while operating within safety limits under uncertainty. The robot must sense and analyze the traversability of the surrounding terrain, which depends on the hardware, locomotion control, and terrain properties. It may contain information about the risk, energy, or time consumption needed to traverse the terrain. To avoid hand-crafted traversability cost functions we propose to collect traversability information about the robot and locomotion policy by simulating the traversal over randomly generated terrains using a physics simulator. Thousand of robots are simulated in parallel controlled by the same locomotion policy used in reality to acquire 57 years of real-world locomotion experience equivalent. For deployment on the real robot, a sparse convolutional network is trained to predict the simulated traversability cost, which is tailored to the deployed locomotion policy, from an entirely geometric representation of the environment in the form of a 3D voxel-occupancy map. This representation avoids the need for commonly used elevation maps, which are error-prone in the presence of overhanging obstacles and multi-floor or low-ceiling scenarios. The effectiveness of the proposed traversability prediction network is demonstrated for path planning for the legged robot ANYmal in various indoor and natural environments. △ Less

Submitted 21 August, 2022; v1 submitted 29 March, 2022; originally announced March 2022.

Comments: accepted for 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2022)

Journal ref: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Towards identifying optimal biased feedback for various user states and traits in motor imagery BCI

Authors: Jelena Mladenović, Jeremy Frey, Smeety Pramij, Jeremie Mattout, Fabien Lotte

Abstract: Objective. Neural self-regulation is necessary for achieving control over brain-computer interfaces (BCIs). This can be an arduous learning process especially for motor imagery BCI. Various training methods were proposed to assist users in accomplishing BCI control and increase performance. Notably the use of biased feedback, i.e. non-realistic representation of performance. Benefits of biased fee… ▽ More Objective. Neural self-regulation is necessary for achieving control over brain-computer interfaces (BCIs). This can be an arduous learning process especially for motor imagery BCI. Various training methods were proposed to assist users in accomplishing BCI control and increase performance. Notably the use of biased feedback, i.e. non-realistic representation of performance. Benefits of biased feedback on performance and learning vary between users (e.g. depending on their initial level of BCI control) and remain speculative. To disentangle the speculations, we investigate what personality type, initial state and calibration performance (CP) could benefit from a biased feedback. Methods. We conduct an experiment (n=30 for 2 sessions). The feedback provided to each group (n=10) is either positively, negatively or not biased. Results. Statistical analyses suggest that interactions between bias and: 1) workload, 2) anxiety, and 3) self-control significantly affect online performance. For instance, low initial workload paired with negative bias is associated to higher peak performances (86%) than without any bias (69%). High anxiety relates negatively to performance no matter the bias (60%), while low anxiety matches best with negative bias (76%). For low CP, learning rate (LR) increases with negative bias only short term (LR=2%) as during the second session it severely drops (LR=-1%). Conclusion. We unveil many interactions between said human factors and bias. Additionally, we use prediction models to confirm and reveal even more interactions. Significance. This paper is a first step towards identifying optimal biased feedback for a personality type, state, and CP in order to maximize BCI performance and learning. △ Less

Submitted 23 December, 2021; originally announced December 2021.

Comments: IEEE Transactions on Biomedical Engineering, Institute of Electrical and Electronics Engineers, 2021

Introducing the quadratically-constrained quadratic programming framework in HPIPM

Authors: Gianluca Frison, Jonathan Frey, Florian Messerer, Andrea Zanelli, Moritz Diehl

Abstract: This paper introduces the quadratically-constrained quadratic programming (QCQP) framework recently added in HPIPM alongside the original quadratic-programming (QP) framework. The aim of the new framework is unchanged, namely providing the building blocks to efficiently and reliably solve (more general classes of) optimal control problems (OCP). The newly introduced QCQP framework provides full fe… ▽ More This paper introduces the quadratically-constrained quadratic programming (QCQP) framework recently added in HPIPM alongside the original quadratic-programming (QP) framework. The aim of the new framework is unchanged, namely providing the building blocks to efficiently and reliably solve (more general classes of) optimal control problems (OCP). The newly introduced QCQP framework provides full features parity with the original QP framework: three types of QCQPs (dense, optimal control and tree-structured optimal control QCQPs) and interior point method (IPM) solvers as well as (partial) condensing and other pre-processing routines. Leveraging the modular structure of HPIPM, the new QCQP framework builds on the QP building blocks and similarly provides fast and reliable IPM solvers. △ Less

Submitted 22 December, 2021; originally announced December 2021.

Comments: 7 pages, 1 figure, submitted to ECC 2022

Continual Adaptation of Semantic Segmentation using Complementary 2D-3D Data Representations

Authors: Jonas Frey, Hermann Blum, Francesco Milano, Roland Siegwart, Cesar Cadena

Abstract: Semantic segmentation networks are usually pre-trained once and not updated during deployment. As a consequence, misclassifications commonly occur if the distribution of the training data deviates from the one encountered during the robot's operation. We propose to mitigate this problem by adapting the neural network to the robot's environment during deployment, without any need for external super… ▽ More Semantic segmentation networks are usually pre-trained once and not updated during deployment. As a consequence, misclassifications commonly occur if the distribution of the training data deviates from the one encountered during the robot's operation. We propose to mitigate this problem by adapting the neural network to the robot's environment during deployment, without any need for external supervision. Leveraging complementary data representations, we generate a supervision signal, by probabilistically accumulating consecutive 2D semantic predictions in a volumetric 3D map. We then train the network on renderings of the accumulated semantic map, effectively resolving ambiguities and enforcing multi-view consistency through the 3D representation. In contrast to scene adaptation methods, we aim to retain the previously-learned knowledge, and therefore employ a continual learning experience replay strategy to adapt the network. Through extensive experimental evaluation, we show successful adaptation to real-world indoor scenes both on the ScanNet dataset and on in-house data recorded with an RGB-D sensor. Our method increases the segmentation accuracy on average by 9.9% compared to the fixed pre-trained neural network, while retaining knowledge from the pre-training dataset. △ Less

Submitted 20 August, 2022; v1 submitted 3 November, 2021; originally announced November 2021.

Comments: Accepted for IEEE Robotics and Automation Letters (R-AL 2022)

Report number: 9874976

Journal ref: IEEE Robotics and Automation Letters 2022

Constructing Coproducts in locally Cartesian closed $\infty$-Categories

Authors: Jonas Frey, Nima Rasekh

Abstract: We prove that every locally Cartesian closed $\infty$-category with subobject classifier has a strict initial object and disjoint and universal binary coproducts. We prove that every locally Cartesian closed $\infty$-category with subobject classifier has a strict initial object and disjoint and universal binary coproducts. △ Less

Submitted 13 February, 2022; v1 submitted 25 August, 2021; originally announced August 2021.

Comments: 14 Pages, to appear in Homology, Homotopy and Applications

MSC Class: 18N60; 03G30; 18B25; 03B38

Artificial coherent states of light by multi-photon interference in a single-photon stream

Authors: P. Steindl, H. Snijders, G. Westra, E. Hissink, K. Iakovlev, S. Polla, J. A. Frey, J. Norman, A. C. Gossard, J. E. Bowers, D. Bouwmeester, W. Löffler

Abstract: Coherent optical states consist of a quantum superposition of different photon number (Fock) states, but because they do not form an orthogonal basis, no photon number states can be obtained from it by linear optics. Here we demonstrate the reverse, by manipulating a random continuous single-photon stream using quantum interference in an optical Sagnac loop, we create engineered quantum states of… ▽ More Coherent optical states consist of a quantum superposition of different photon number (Fock) states, but because they do not form an orthogonal basis, no photon number states can be obtained from it by linear optics. Here we demonstrate the reverse, by manipulating a random continuous single-photon stream using quantum interference in an optical Sagnac loop, we create engineered quantum states of light with tunable photon statistics, including approximate weak coherent states. We demonstrate this experimentally using a true single-photon stream produced by a semiconductor quantum dot in an optical microcavity, and show that we can obtain light with $g^{(2)}(0)\rightarrow1$ in agreement with our theory, which can only be explained by quantum interference of at least 3 photons. The produced artificial light states are, however, much more complex than coherent states, containing quantum entanglement of photons, making them a resource for multi-photon entanglement. △ Less

Submitted 11 April, 2021; v1 submitted 29 October, 2020; originally announced October 2020.

Comments: 6 pages + supplemental material

Journal ref: Phys. Rev. Lett. 126, 143601 (2021)

Criteria for deterministic single-photon emission in two-dimensional atomic crystals

Authors: Joshua J. P. Thompson, Samuel Brem, Hanlin Fang, Joey Frey, Saroj P. Dash, Witlef Wieczorek, Ermin Malic

Abstract: The deterministic production of single-photons from two dimensional materials promises to usher in a new generation of photonic quantum devices. In this work, we outline criteria by which single-photon emission can be realised in two dimensional materials: spatial isolation, spectral filtering and low excitation of quantum emitters. We explore how these criteria can be fulfilled in atomically thin… ▽ More The deterministic production of single-photons from two dimensional materials promises to usher in a new generation of photonic quantum devices. In this work, we outline criteria by which single-photon emission can be realised in two dimensional materials: spatial isolation, spectral filtering and low excitation of quantum emitters. We explore how these criteria can be fulfilled in atomically thin transition metal dichalcogenides, where excitonic physics dictates the observed photoemission. In particular, we model the effect of defects and localised strain, in accordance with the most common experimental realisations, on the photon statistics of emitted light. Moreover, we demonstrate that an optical cavity has a negative impact on the photon statistics, suppressing the single-photon character of the emission by diminishing the effect of spectral filtering on the emitted light. Our work provides a theoretical framework revealing criteria necessary to facilitate single-photon emission in two-dimensional materials and thus can guide future experimental studies in this field. △ Less

Submitted 24 August, 2020; originally announced August 2020.

Light controls motility and phase separation of photosynthetic microbes

Authors: Alexandros A. Fragkopoulos, Jérémy Vachier, Johannes Frey, Flora-Maud Le Menn, Michael Wilczek, Marco G. Mazza, Oliver Bäumchen

Abstract: Large ensembles of interacting, out-of-equilibrium agents are a paradigm of active matter. Their constituents' intrinsic activity may entail the spontaneous separation into localized phases of high and low densities. Motile microbes, equipped with ATP-fueled engines, are prime examples of such phase-separating active matter, which is fundamental in myriad biological processes. The fact that sponta… ▽ More Large ensembles of interacting, out-of-equilibrium agents are a paradigm of active matter. Their constituents' intrinsic activity may entail the spontaneous separation into localized phases of high and low densities. Motile microbes, equipped with ATP-fueled engines, are prime examples of such phase-separating active matter, which is fundamental in myriad biological processes. The fact that spontaneous spatial aggregation is not widely recognized as a general feature of microbial communities challenges the generalisation of phase separation beyond artificial active systems. Here, we report on the phase separation of populations of Chlamydomonas reinhardtii that can be controlled by light in a fully reversible manner. We trace this phenomenon back to the light- and density-dependent motility, thus bridging the gap from light perception on the single-cell level to collective spatial self-organization into regions of high and low density. Its spectral sensitivity suggests that microbial motility and phase separation are regulated by the activity of the photosynthetic machinery. Characteristic fingerprints of the stability and dynamics of this active system paint a picture that cannot be reconciled with the current physical understanding of phase separation in artificial active matter, whereby collective behavior can emerge from inherent motility modulation in response to changing stimuli. Our results therefore point towards the existence of a broader class of self-organization phenomena in living systems. △ Less

Submitted 2 June, 2020; originally announced June 2020.

Comments: 9 pages, 3 figures

Triposes as a Generalization of Localic Geometric Morphisms

Authors: J. Frey, T. Streicher

Abstract: We discuss how triposes may be understood as generalizations of localic geometric morphisms. We discuss how triposes may be understood as generalizations of localic geometric morphisms. △ Less

Submitted 18 September, 2023; v1 submitted 12 May, 2020; originally announced May 2020.

Comments: We have added a remark on the correct notion of morphism between regular triposes and why they are closed under composition

Physiologically Driven Storytelling: Concept and Software Tool

Authors: Jérémy Frey, Gilad Ostrin, May Grabli, Jessica Cauchard

Abstract: We put forth Physiologically Driven Storytelling, a new approach to interactive storytelling where narratives adaptively unfold based on the reader's physiological state. We first describe a taxonomy framing how physiological signals can be used to drive interactive systems both as input and output. We then propose applications to interactive storytelling and describe the implementation of a softw… ▽ More We put forth Physiologically Driven Storytelling, a new approach to interactive storytelling where narratives adaptively unfold based on the reader's physiological state. We first describe a taxonomy framing how physiological signals can be used to drive interactive systems both as input and output. We then propose applications to interactive storytelling and describe the implementation of a software tool to create Physiological Interactive Fiction (PIF). The results of an online study (N=140) provided guidelines towards augmenting the reading experience. PIF was then evaluated in a lab study (N=14) to determine how physiological signals can be used to infer a reader's state. Our results show that breathing, electrodermal activity, and eye tracking can help differentiate positive from negative tones, and monotonous from exciting events. This work demonstrates how PIF can support storytelling in creating engaging content and experience tailored to the reader. Moreover, it opens the space to future physiologically driven systems within broader application areas. △ Less

Submitted 20 March, 2020; originally announced March 2020.

Comments: CHI '20 - SIGCHI Conference on Human Factors in Computing System, Apr 2020, Honolulu, United States

Defining mass transfer in a capillary wave micro-bioreactor

Authors: Lasse Jannis Frey, David Vorländer, Detlev Rasch, Sven Meinen, Bernhard Müller, Torsten Mayr, Andreas Dietzel, Jan-Hendrik Grosch, Rainer Krull

Abstract: For high-throughput cell culture and associated analytics, droplet-based cultivation systems open up the opportunities for parallelization and rapid data generation. In contrast to microfluidics with continuous flow, sessile droplet approaches enhance the flexibility for fluid manipulation with usually less operational effort. Generating biologically favorable conditions and promoting cell growth… ▽ More For high-throughput cell culture and associated analytics, droplet-based cultivation systems open up the opportunities for parallelization and rapid data generation. In contrast to microfluidics with continuous flow, sessile droplet approaches enhance the flexibility for fluid manipulation with usually less operational effort. Generating biologically favorable conditions and promoting cell growth in a droplet, however, is particularly challenging due to mass transfer limitations, which has to be solved by implementing an effective mixing technique. Here, capillary waves induced by vertical oscillation are used to mix inside a sessile droplet micro-bioreactor (MBR) system avoiding additional moving parts inside the fluid. Depending on the excitation frequency, different patterns are formed on the oscillating liquid surface, which are described by a model of a vibrated sessile droplet. Analyzing mixing times and oxygen transport into the liquid, a strong dependency of mass transfer on the oscillation parameters, especially the excitation frequency, is demonstrated. Oscillations at distinct capillary wave resonant frequencies lead to rapid homogenization with mixing times of 2 s and volumetric liquid-phase mass transfer coefficients of more than 340 h-1. This shows that the mass transfer in a droplet MBR can be specifically controlled via capillary waves, what is subsequently demonstrated for cultivations of Escherichia coli BL21 cells. Therefore, the presented MBR in combination with vertical oscillation mixing for intensified mass transfer is a promising tool for highly parallel cultivation and data generation. △ Less

Submitted 4 February, 2020; originally announced February 2020.

Categories of partial equivalence relations as localizations

Authors: Jonas Frey

Abstract: We construct a category of fibrant objects $\mathbb{C}\langle P\rangle$ in the sense of K. Brown from any indexed frame (a kind of indexed poset generalizing triposes) $P$, and show that its homotopy category is the Barr-exact category $\mathbb{C}[P]$ of partial equivalence relations and compatible functional relations. In particular this gives a presentation of realizability toposes as homotopy c… ▽ More We construct a category of fibrant objects $\mathbb{C}\langle P\rangle$ in the sense of K. Brown from any indexed frame (a kind of indexed poset generalizing triposes) $P$, and show that its homotopy category is the Barr-exact category $\mathbb{C}[P]$ of partial equivalence relations and compatible functional relations. In particular this gives a presentation of realizability toposes as homotopy categories. We give criteria for the existence of left and right derived functors to functors $\mathbb{C}\langleΦ\rangle : \mathbb{C}\langle P\rangle\to \mathbb{C}\langle Q\rangle$ induced by finite-meet-preserving transformations $Φ : P \to Q$ between indexed frames. △ Less

Submitted 18 April, 2022; v1 submitted 13 December, 2019; originally announced December 2019.

Comments: 26 pages

MSC Class: 03G30

acados: a modular open-source framework for fast embedded optimal control

Authors: Robin Verschueren, Gianluca Frison, Dimitris Kouzoupis, Jonathan Frey, Niels van Duijkeren, Andrea Zanelli, Branimir Novoselnik, Thivaharan Albin, Rien Quirynen, Moritz Diehl

Abstract: This paper presents the acados software package, a collection of solvers for fast embedded optimization intended for fast embedded applications. Its interfaces to higher-level languages make it useful for quickly designing an optimization-based control algorithm by putting together different algorithmic components that can be readily connected and interchanged. Since the core of acados is written… ▽ More This paper presents the acados software package, a collection of solvers for fast embedded optimization intended for fast embedded applications. Its interfaces to higher-level languages make it useful for quickly designing an optimization-based control algorithm by putting together different algorithmic components that can be readily connected and interchanged. Since the core of acados is written on top of a high-performance linear algebra library, we do not sacrifice computational performance. Thus, we aim to provide both flexibility and performance through modularity, without the need to rely on automatic code generation, which facilitates maintainability and extensibility. The main features of acados are: efficient optimal control algorithms targeting embedded devices implemented in C, linear algebra based on the high-performance BLASFEO library, user-friendly interfaces to Matlab and Python, and compatibility with the modeling language of CasADi. acados is free and open-source software released under the permissive BSD 2-Clause license. △ Less

Submitted 26 November, 2020; v1 submitted 30 October, 2019; originally announced October 2019.

Comments: Second revision

Topology and geometry of molecular conformational spaces and energy landscapes

Authors: Ingrid Membrillo-Solis, Mariam Pirashvili, Lee Steinberg, Jacek Brodzki, Jeremy G. Frey

Abstract: Understanding the geometry and topology of configuration or conformational spaces of molecules has relevant applications in chemistry and biology such as the proteins folding problem, drug design and the structure activity relationship problem. Despite their relevance, configuration spaces of molecules are only partially understood. In this paper we discuss both theoretical and computational appro… ▽ More Understanding the geometry and topology of configuration or conformational spaces of molecules has relevant applications in chemistry and biology such as the proteins folding problem, drug design and the structure activity relationship problem. Despite their relevance, configuration spaces of molecules are only partially understood. In this paper we discuss both theoretical and computational approaches to the configuration spaces of molecules and their associated energy landscapes. Our mathematical approach shows that when symmetries of the molecules are taken into account, configuration spaces of molecules give rise to certain principal bundles and orbifolds. We also make use of a variety of geometric and topological tools for data analysis to study the topology and geometry of these spaces. △ Less

Submitted 18 July, 2019; originally announced July 2019.

Comments: 32 pages

MSC Class: 92E10