Showing 1–14 of 14 results for author: Hohmann, S

An Analysis of Driver-Initiated Takeovers during Assisted Driving and their Effect on Driver Satisfaction

Authors: Robin Schwager, Michael Grimm, Xin Liu, Lukas Ewecker, Tim Bruehl, Tin Stribor Sohn, Soeren Hohmann

Abstract: During the use of Advanced Driver Assistance Systems (ADAS), drivers can intervene in the active function and take back control due to various reasons. However, the specific reasons for driver-initiated takeovers in naturalistic driving are still not well understood. In order to get more information on the reasons behind these takeovers, a test group study was conducted. There, 17 participants use… ▽ More During the use of Advanced Driver Assistance Systems (ADAS), drivers can intervene in the active function and take back control due to various reasons. However, the specific reasons for driver-initiated takeovers in naturalistic driving are still not well understood. In order to get more information on the reasons behind these takeovers, a test group study was conducted. There, 17 participants used a predictive longitudinal driving function for their daily commutes and annotated the reasons for their takeovers during active function use. In this paper, the recorded takeovers are analyzed and the different reasons for them are highlighted. The results show that the reasons can be divided into three main categories. The most common category consists of takeovers which aim to adjust the behavior of the ADAS within its Operational Design Domain (ODD) in order to better match the drivers' personal preferences. Other reasons include takeovers due to leaving the ADAS's ODD and corrections of incorrect sensing state information. Using the questionnaire results of the test group study, it was found that the number and frequency of takeovers especially within the ADAS's ODD have a significant negative impact on driver satisfaction. Therefore, the driver satisfaction with the ADAS could be increased by adapting its behavior to the drivers' wishes and thereby lowering the number of takeovers within the ODD. The information contained in the takeover behavior of the drivers could be used as feedback for the ADAS. Finally, it is shown that there are considerable differences in the takeover behavior of different drivers, which shows a need for ADAS individualization. △ Less

Submitted 10 June, 2024; v1 submitted 19 April, 2024; originally announced April 2024.

Socially Integrated Navigation: A Social Acting Robot with Deep Reinforcement Learning

Authors: Daniel Flögel, Lars Fischer, Thomas Rudolf, Tobias Schürmann, Sören Hohmann

Abstract: Mobile robots are being used on a large scale in various crowded situations and become part of our society. The socially acceptable navigation behavior of a mobile robot with individual human consideration is an essential requirement for scalable applications and human acceptance. Deep Reinforcement Learning (DRL) approaches are recently used to learn a robot's navigation policy and to model the c… ▽ More Mobile robots are being used on a large scale in various crowded situations and become part of our society. The socially acceptable navigation behavior of a mobile robot with individual human consideration is an essential requirement for scalable applications and human acceptance. Deep Reinforcement Learning (DRL) approaches are recently used to learn a robot's navigation policy and to model the complex interactions between robots and humans. We propose to divide existing DRL-based navigation approaches based on the robot's exhibited social behavior and distinguish between social collision avoidance with a lack of social behavior and socially aware approaches with explicit predefined social behavior. In addition, we propose a novel socially integrated navigation approach where the robot's social behavior is adaptive and emerges from the interaction with humans. The formulation of our approach is derived from a sociological definition, which states that social acting is oriented toward the acting of others. The DRL policy is trained in an environment where other agents interact socially integrated and reward the robot's behavior individually. The simulation results indicate that the proposed socially integrated navigation approach outperforms a socially aware approach in terms of distance traveled, time to completion, and negative impact on all agents within the environment. △ Less

Submitted 14 March, 2024; originally announced March 2024.

ReACT: Reinforcement Learning for Controller Parametrization using B-Spline Geometries

Authors: Thomas Rudolf, Daniel Flögel, Tobias Schürmann, Simon Süß, Stefan Schwab, Sören Hohmann

Abstract: Robust and performant controllers are essential for industrial applications. However, deriving controller parameters for complex and nonlinear systems is challenging and time-consuming. To facilitate automatic controller parametrization, this work presents a novel approach using deep reinforcement learning (DRL) with N-dimensional B-spline geometries (BSGs). We focus on the control of parameter-va… ▽ More Robust and performant controllers are essential for industrial applications. However, deriving controller parameters for complex and nonlinear systems is challenging and time-consuming. To facilitate automatic controller parametrization, this work presents a novel approach using deep reinforcement learning (DRL) with N-dimensional B-spline geometries (BSGs). We focus on the control of parameter-variant systems, a class of systems with complex behavior which depends on the operating conditions. For this system class, gain-scheduling control structures are widely used in applications across industries due to well-known design principles. Facilitating the expensive controller parametrization task regarding these control structures, we deploy an DRL agent. Based on control system observations, the agent autonomously decides how to adapt the controller parameters. We make the adaptation process more efficient by introducing BSGs to map the controller parameters which may depend on numerous operating conditions. To preprocess time-series data and extract a fixed-length feature vector, we use a long short-term memory (LSTM) neural networks. Furthermore, this work contributes actor regularizations that are relevant to real-world environments which differ from training. Accordingly, we apply dropout layer normalization to the actor and critic networks of the truncated quantile critic (TQC) algorithm. To show our approach's working principle and effectiveness, we train and evaluate the DRL agent on the parametrization task of an industrial control structure with parameter lookup tables. △ Less

Submitted 10 January, 2024; originally announced January 2024.

Comments: 7 pages, 7 figures, accepted at the 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Honolulu, HI, USA

Shared Telemanipulation with VR controllers in an anti slosh scenario

Authors: Max Grobbel, Balint Varga, Sören Hohmann

Abstract: Telemanipulation has become a promising technology that combines human intelligence with robotic capabilities to perform tasks remotely. However, it faces several challenges such as insufficient transparency, low immersion, and limited feedback to the human operator. Moreover, the high cost of haptic interfaces is a major limitation for the application of telemanipulation in various fields, includ… ▽ More Telemanipulation has become a promising technology that combines human intelligence with robotic capabilities to perform tasks remotely. However, it faces several challenges such as insufficient transparency, low immersion, and limited feedback to the human operator. Moreover, the high cost of haptic interfaces is a major limitation for the application of telemanipulation in various fields, including elder care, where our research is focused. To address these challenges, this paper proposes the usage of nonlinear model predictive control for telemanipulation using low-cost virtual reality controllers, including multiple control goals in the objective function. The framework utilizes models for human input prediction and taskrelated models of the robot and the environment. The proposed framework is validated on an UR5e robot arm in the scenario of handling liquid without spilling. Further extensions of the framework such as pouring assistance and collision avoidance can easily be included. △ Less

Submitted 14 September, 2023; originally announced September 2023.

Intention-Aware Decision-Making for Mixed Intersection Scenarios

Authors: Balint Varga, Dongxu Yang, Soeren Hohmann

Abstract: This paper presents a white-box intention-aware decision-making for the handling of interactions between a pedestrian and an automated vehicle (AV) in an unsignalized street crossing scenario. Moreover, a design framework has been developed, which enables automated parameterization of the decision-making. This decision-making is designed in such a manner that it can understand pedestrians in urban… ▽ More This paper presents a white-box intention-aware decision-making for the handling of interactions between a pedestrian and an automated vehicle (AV) in an unsignalized street crossing scenario. Moreover, a design framework has been developed, which enables automated parameterization of the decision-making. This decision-making is designed in such a manner that it can understand pedestrians in urban traffic and can react accordingly to their intentions. That way, a human-like response to the actions of the pedestrian is ensured, leading to a higher acceptance of AVs. The core notion of this paper is that the intention prediction of the pedestrian to cross the street and decision-making are divided into two subsystems. On the one hand, the intention detection is a data-driven, black-box model. Thus, it can model the complex behavior of the pedestrians. On the other hand, the decision-making is a white-box model to ensure traceability and to enable a rapid verification and validation of AVs. This white-box decision-making provides human-like behavior and a guaranteed prevention of deadlocks. An additional benefit is that the proposed decision-making requires low computational resources only enabling real world usage. The automated parameterization uses a particle swarm optimization and compares two different models of the pedestrian: The social force model and the Markov decision process model. Consequently, a rapid design of the decision-making is possible and different pedestrian behaviors can be taken into account. The results reinforce the applicability of the proposed intention-aware decision-making. △ Less

Submitted 29 March, 2023; originally announced March 2023.

Development of a Mobile Vehicle Manipulator Simulator for the Validation of Shared Control Concepts

Authors: Balint Varga, Selina Meier, Soeren Hohmann

Abstract: This paper presents the development of a real-time simulator for the validation of controlling a large vehicle manipulator. The need for this development can be justified by the lack of such a simulator: There are neither open source projects nor commercial products, which would be suitable for testing cooperative control concepts. First, we present the nonlinear simulation model of the vehicle an… ▽ More This paper presents the development of a real-time simulator for the validation of controlling a large vehicle manipulator. The need for this development can be justified by the lack of such a simulator: There are neither open source projects nor commercial products, which would be suitable for testing cooperative control concepts. First, we present the nonlinear simulation model of the vehicle and the manipulator. For the modeling MATLAB/Simulink is used, which also enables a code generation into standalone C++ ROS-Nodes (Robot Operating System Nodes). The emerging challenges of the code generation are also discussed. Then, the obtained standalone C++ ROS-Nodes integrated in the simulator framework which includes a graphical user interface, a steering wheel and a joystick. This simulator can provide the real-time calculation of the overall system's motion enabling the interaction of human and automation. Furthermore, a qualitative validation of the model is given. Finally, the functionalities of the simulator is demonstrated in tests with a human operators. △ Less

Submitted 23 November, 2022; originally announced November 2022.

Comments: in German language

Human-machine Symbiosis: A Multivariate Perspective for Physically Coupled Human-machine Systems

Authors: Jairo Inga, Miriam Ruess, Jan Heinrich Robens, Thomas Nelius, Sean Kille, Philipp Dahlinger, Roland Thomaschke, Gerhard Neumann, Sven Matthiesen, Sören Hohmann, Andrea Kiesel

Abstract: The notion of symbiosis has been increasingly mentioned in research on physically coupled human-machine systems. Yet, a uniform specification on which aspects constitute human-machine symbiosis is missing. By combining the expertise of different disciplines, we elaborate on a multivariate perspective of symbiosis as the highest form of physically coupled human-machine systems. Four dimensions are… ▽ More The notion of symbiosis has been increasingly mentioned in research on physically coupled human-machine systems. Yet, a uniform specification on which aspects constitute human-machine symbiosis is missing. By combining the expertise of different disciplines, we elaborate on a multivariate perspective of symbiosis as the highest form of physically coupled human-machine systems. Four dimensions are considered: Task, interaction, performance, and experience. First, human and machine work together to accomplish a common task conceptualized on both a decision and an action level (task dimension). Second, each partner possesses an internal representation of own as well as the other partner's intentions and influence on the environment. This alignment, which is the core of the interaction, constitutes the symbiotic understanding between both partners, being the basis of a joint, highly coordinated and effective action (interaction dimension). Third, the symbiotic interaction leads to synergetic effects regarding the intention recognition and complementary strengths of the partners, resulting in a higher overall performance (performance dimension). Fourth, symbiotic systems specifically change the user's experiences, like flow, acceptance, sense of agency, and embodiment (experience dimension). This multivariate perspective is flexible and generic and is also applicable in diverse human-machine scenarios, hel** to bridge barriers between different disciplines. △ Less

Submitted 29 November, 2021; originally announced November 2021.

Comments: 28 pages, 7 figures

Adaptive Optimal Trajectory Tracking Control Applied to a Large-Scale Ball-on-Plate System

Authors: Florian Köpf, Sean Kille, Jairo Inga, Sören Hohmann

Abstract: While many theoretical works concerning Adaptive Dynamic Programming (ADP) have been proposed, application results are scarce. Therefore, we design an ADP-based optimal trajectory tracking controller and apply it to a large-scale ball-on-plate system. Our proposed method incorporates an approximated reference trajectory instead of using setpoint tracking and allows to automatically compensate for… ▽ More While many theoretical works concerning Adaptive Dynamic Programming (ADP) have been proposed, application results are scarce. Therefore, we design an ADP-based optimal trajectory tracking controller and apply it to a large-scale ball-on-plate system. Our proposed method incorporates an approximated reference trajectory instead of using setpoint tracking and allows to automatically compensate for constant offset terms. Due to the off-policy characteristics of the algorithm, the method requires only a small amount of measured data to train the controller. Our experimental results show that this tracking mechanism significantly reduces the control cost compared to setpoint controllers. Furthermore, a comparison with a model-based optimal controller highlights the benefits of our model-free data-based ADP tracking controller, where no system model and manual tuning are required but the controller is tuned automatically using measured data. △ Less

Submitted 25 January, 2021; v1 submitted 26 October, 2020; originally announced October 2020.

Comments: F. Köpf and S. Kille contributed equally to this work. \c{opyright} 2021 IEEE

Multi-Robot Task Allocation and Scheduling Considering Cooperative Tasks and Precedence Constraints

Authors: Esther Bischoff, Fabian Meyer, Jairo Inga, Sören Hohmann

Abstract: In order to fully exploit the advantages inherent to cooperating heterogeneous multi-robot teams, sophisticated coordination algorithms are essential. Time-extended multi-robot task allocation approaches assign and schedule a set of tasks to a group of robots such that certain objectives are optimized and operational constraints are met. This is particularly challenging if cooperative tasks, i.e.… ▽ More In order to fully exploit the advantages inherent to cooperating heterogeneous multi-robot teams, sophisticated coordination algorithms are essential. Time-extended multi-robot task allocation approaches assign and schedule a set of tasks to a group of robots such that certain objectives are optimized and operational constraints are met. This is particularly challenging if cooperative tasks, i.e. tasks that require two or more robots to work directly together, are considered. In this paper, we present an easy-to-implement criterion to validate the feasibility, i.e. executability, of solutions to time-extended multi-robot task allocation problems with cross schedule dependencies arising from the consideration of cooperative tasks and precedence constraints. Using the introduced feasibility criterion, we propose a local improvement heuristic based on a neighborhood operator for the problem class under consideration. The initial solution is obtained by a greedy constructive heuristic. Both methods use a generalized cost structure and are therefore able to handle various objective function instances. We evaluate the proposed approach using test scenarios of different problem sizes, all comprising the complexity aspects of the regarded problem. The simulation results illustrate the improvement potential arising from the application of the local improvement heuristic. △ Less

Submitted 8 May, 2020; originally announced May 2020.

Inverse Dynamic Games Based on Maximum Entropy Inverse Reinforcement Learning

Authors: Jairo Inga, Esther Bischoff, Florian Köpf, Sören Hohmann

Abstract: We consider the inverse problem of dynamic games, where cost function parameters are sought which explain observed behavior of interacting players. Maximum entropy inverse reinforcement learning is extended to the N-player case in order to solve inverse dynamic games with continuous-valued state and control spaces. We present methods for identification of cost function parameters from observed dat… ▽ More We consider the inverse problem of dynamic games, where cost function parameters are sought which explain observed behavior of interacting players. Maximum entropy inverse reinforcement learning is extended to the N-player case in order to solve inverse dynamic games with continuous-valued state and control spaces. We present methods for identification of cost function parameters from observed data which correspond to (i) a Pareto efficient solution, (ii) an open-loop Nash equilibrium or (iii) a feedback Nash equilibrium. Furthermore, we give results on the unbiasedness of the estimation of cost function parameters for each arising class of inverse dynamic game. The applicability of the methods is demonstrated with simulation examples of a nonlinear and a linear-quadratic dynamic game. △ Less

Submitted 24 July, 2020; v1 submitted 18 November, 2019; originally announced November 2019.

Deep Decentralized Reinforcement Learning for Cooperative Control

Authors: Florian Köpf, Samuel Tesfazgi, Michael Flad, Sören Hohmann

Abstract: In order to collaborate efficiently with unknown partners in cooperative control settings, adaptation of the partners based on online experience is required. The rather general and widely applicable control setting, where each cooperation partner might strive for individual goals while the control laws and objectives of the partners are unknown, entails various challenges such as the non-stationar… ▽ More In order to collaborate efficiently with unknown partners in cooperative control settings, adaptation of the partners based on online experience is required. The rather general and widely applicable control setting, where each cooperation partner might strive for individual goals while the control laws and objectives of the partners are unknown, entails various challenges such as the non-stationarity of the environment, the multi-agent credit assignment problem, the alter-exploration problem and the coordination problem. We propose new, modular deep decentralized Multi-Agent Reinforcement Learning mechanisms to account for these challenges. Therefore, our method uses a time-dependent prioritization of samples, incorporates a model of the system dynamics and utilizes variable, accountability-driven learning rates and simulated, artificial experiences in order to guide the learning process. The effectiveness of our method is demonstrated by means of a simulated, nonlinear cooperative control task. △ Less

Submitted 29 October, 2019; originally announced October 2019.

Adaptive Dynamic Programming for Model-free Tracking of Trajectories with Time-varying Parameters

Authors: Florian Köpf, Simon Ramsteiner, Michael Flad, Sören Hohmann

Abstract: In order to autonomously learn to control unknown systems optimally w.r.t. an objective function, Adaptive Dynamic Programming (ADP) is well-suited to adapt controllers based on experience from interaction with the system. In recent years, many researchers focused on the tracking case, where the aim is to follow a desired trajectory. So far, ADP tracking controllers assume that the reference traje… ▽ More In order to autonomously learn to control unknown systems optimally w.r.t. an objective function, Adaptive Dynamic Programming (ADP) is well-suited to adapt controllers based on experience from interaction with the system. In recent years, many researchers focused on the tracking case, where the aim is to follow a desired trajectory. So far, ADP tracking controllers assume that the reference trajectory follows time-invariant exo-system dynamics-an assumption that does not hold for many applications. In order to overcome this limitation, we propose a new Q-function which explicitly incorporates a parametrized approximation of the reference trajectory. This allows to learn to track a general class of trajectories by means of ADP. Once our Q-function has been learned, the associated controller copes with time-varying reference trajectories without need of further training and independent of exo-system dynamics. After proposing our general model-free off-policy tracking method, we provide analysis of the important special case of linear quadratic tracking. We conclude our paper with an example which demonstrates that our new method successfully learns the optimal tracking controller and outperforms existing approaches in terms of tracking error and cost. △ Less

Submitted 17 February, 2020; v1 submitted 16 September, 2019; originally announced September 2019.

Comments: This is a preprint submitted to the Int J Adapt Control Signal Process. The substantially revised version will be published in the Int J Adapt Control Signal Process (DOI: 10.1002/ACS.3106)

Partner Approximating Learners (PAL): Simulation-Accelerated Learning with Explicit Partner Modeling in Multi-Agent Domains

Authors: Florian Köpf, Alexander Nitsch, Michael Flad, Sören Hohmann

Abstract: Mixed cooperative-competitive control scenarios such as human-machine interaction with individual goals of the interacting partners are very challenging for reinforcement learning agents. In order to contribute towards intuitive human-machine collaboration, we focus on problems in the continuous state and control domain where no explicit communication is considered and the agents do not know the o… ▽ More Mixed cooperative-competitive control scenarios such as human-machine interaction with individual goals of the interacting partners are very challenging for reinforcement learning agents. In order to contribute towards intuitive human-machine collaboration, we focus on problems in the continuous state and control domain where no explicit communication is considered and the agents do not know the others' goals or control laws but only sense their control inputs retrospectively. Our proposed framework combines a learned partner model based on online data with a reinforcement learning agent that is trained in a simulated environment including the partner model. Thus, we overcome drawbacks of independent learners and, in addition, benefit from a reduced amount of real world data required for reinforcement learning which is vital in the human-machine context. We finally analyze an example that demonstrates the merits of our proposed framework which learns fast due to the simulated environment and adapts to the continuously changing partner due to the partner approximation. △ Less

Submitted 2 March, 2020; v1 submitted 9 September, 2019; originally announced September 2019.

Adaptive Optimal Control for Reference Tracking Independent of Exo-System Dynamics

Authors: Florian Köpf, Johannes Westermann, Michael Flad, Sören Hohmann

Abstract: Model-free control based on the idea of Reinforcement Learning is a promising approach that has recently gained extensive attention. However, Reinforcement-Learning-based control methods solely focus on the regulation problem or learn to track a reference that is generated by a time-invariant exo-system. In the latter case, controllers are only able to track the time-invariant reference dynamics w… ▽ More Model-free control based on the idea of Reinforcement Learning is a promising approach that has recently gained extensive attention. However, Reinforcement-Learning-based control methods solely focus on the regulation problem or learn to track a reference that is generated by a time-invariant exo-system. In the latter case, controllers are only able to track the time-invariant reference dynamics which they have been trained on and need to be re-trained each time the reference dynamics change. Consequently, these methods fail in a number of applications which obviously rely on a trajectory not being generated by an exo-system. One prominent example is autonomous driving. This paper provides for the first time an adaptive optimal control method capable to track reference trajectories not being generated by a time-invariant exo-system. The main innovation is a novel Q-function that directly incorporates a given reference trajectory on a moving horizon. This new Q-function exhibits a particular structure which allows the design of an efficient, iterative, provably convergent Reinforcement Learning algorithm that enables optimal tracking. Two real-world examples demonstrate the effectiveness of our new method. △ Less

Submitted 28 November, 2019; v1 submitted 12 June, 2019; originally announced June 2019.