Showing 1–10 of 10 results for author: Hohmann, S

Trustworthiness of Optimality Condition Violation in Inverse Dynamic Game Methods Based on the Minimum Principle

Authors: Philipp Karg, Adrian Kienzle, Jonas Kaub, Balint Varga, Sören Hohmann

Abstract: In this work, we analyze the applicability of Inverse Dynamic Game (IDG) methods based on the Minimum Principle (MP). The IDG method determines unknown cost functions in a single- or multi-agent setting from observed system trajectories by minimizing the so-called residual error, i.e. the extent to which the optimality conditions of the MP are violated with a current guess of cost functions. The m… ▽ More In this work, we analyze the applicability of Inverse Dynamic Game (IDG) methods based on the Minimum Principle (MP). The IDG method determines unknown cost functions in a single- or multi-agent setting from observed system trajectories by minimizing the so-called residual error, i.e. the extent to which the optimality conditions of the MP are violated with a current guess of cost functions. The main assumption of the IDG method to recover cost functions such that the resulting trajectories match the observed ones is that the given trajectories are the result of a Dynamic Game (DG) problem with known parameterized cost function structures. However, in practice, when the IDG method is used to identify the behavior of unknown agents, e.g. humans, this assumption cannot be guaranteed. Hence, we introduce the notion of the trustworthiness of the residual error and provide necessary conditions for it to define when the IDG method based on the MP is applicable to such problems. From the necessary conditions, we conclude that the MP-based IDG method cannot be used to validate DG models for unknown agents but can yield under certain conditions robust parameter identifications, e.g. to measurement noise. Finally, we illustrate these conclusions by validating a DG model for the collision avoidance behavior between two mobile robots with human operators. △ Less

Submitted 17 June, 2024; v1 submitted 5 February, 2024; originally announced February 2024.

Bi-Level-Based Inverse Stochastic Optimal Control

Authors: Philipp Karg, Manuel Hess, Balint Varga, Sören Hohmann

Abstract: In this paper, we propose a new algorithm to solve the Inverse Stochastic Optimal Control (ISOC) problem of the linear-quadratic sensorimotor (LQS) control model. The LQS model represents the current state-of-the-art in describing goal-directed human movements. The ISOC problem aims at determining the cost function and noise scaling matrices of the LQS model from measurement data since both parame… ▽ More In this paper, we propose a new algorithm to solve the Inverse Stochastic Optimal Control (ISOC) problem of the linear-quadratic sensorimotor (LQS) control model. The LQS model represents the current state-of-the-art in describing goal-directed human movements. The ISOC problem aims at determining the cost function and noise scaling matrices of the LQS model from measurement data since both parameter types influence the statistical moments predicted by the model and are unknown in practice. We prove global convergence for our new algorithm and at a numerical example, validate the theoretical assumptions of our method. By comprehensive simulations, the influence of the tuning parameters of our algorithm on convergence behavior and computation time is analyzed. The new algorithm computes ISOC solutions nearly 33 times faster than the single previously existing ISOC algorithm. △ Less

Submitted 19 March, 2024; v1 submitted 3 November, 2023; originally announced November 2023.

Identification Methods for Ordinal Potential Differential Games

Authors: Balint Varga, Da Huang, Sören Hohmann

Abstract: This paper introduces two new identification methods for the class of linear quadratic (LQ) ordinal potential differential games (OPDGs). Potential games are notable for their benefits, including the computability and guaranteed existence of Nash Equilibria. Previous literature has explored the analysis of static ordinal potential games, yet their applicability to various engineering applications… ▽ More This paper introduces two new identification methods for the class of linear quadratic (LQ) ordinal potential differential games (OPDGs). Potential games are notable for their benefits, including the computability and guaranteed existence of Nash Equilibria. Previous literature has explored the analysis of static ordinal potential games, yet their applicability to various engineering applications remains limited. Despite the previous introduction of the core idea of OPDGs, a systematic method for identifying a potential game for a given LQ differential game has not been developed yet. To address this research gap, this paper proposes two identification methods that provide the quadratic potential cost function for the given LQ differential game. Both identification methods are based on linear matrix inequalities. The first identification method aims to minimize the condition number of the potential cost function's parameters, providing a faster and more precise technique compared to earlier solutions. In addition, an evaluation regarding the feasibility of system structure requirements is presented. With a less rigid formulation, the second identification technique can successfully identify LQ OPDGs in instances where the first method fails. These two novel identification methods are verified through simulations. The results demonstrate their advantages and potential in designing and analyzing cooperative control systems. △ Less

Submitted 13 February, 2024; v1 submitted 22 June, 2023; originally announced June 2023.

Comments: This work has been submitted to a possible Springer publication

Validation of Stochastic Optimal Control Models for Goal-Directed Human Movements on the Example of Human Driving Behavior

Authors: Philipp Karg, Simon Stoll, Simon Rothfuß, Sören Hohmann

Abstract: Stochastic Optimal Control models represent the state-of-the-art in modeling goal-directed human movements. The linear-quadratic sensorimotor (LQS) model based on signal-dependent noise processes in state and output equation is the current main representative. With our newly introduced Inverse Stochastic Optimal Control algorithm building upon two bi-level optimizations, we can identify its unknow… ▽ More Stochastic Optimal Control models represent the state-of-the-art in modeling goal-directed human movements. The linear-quadratic sensorimotor (LQS) model based on signal-dependent noise processes in state and output equation is the current main representative. With our newly introduced Inverse Stochastic Optimal Control algorithm building upon two bi-level optimizations, we can identify its unknown model parameters, namely cost function matrices and scaling parameters of the noise processes, for the first time. In this paper, we use this algorithm to identify the parameters of a deterministic linear-quadratic, a linear-quadratic Gaussian and a LQS model from human measurement data to compare the models' capability in describing goal-directed human movements. Human steering behavior in a simplified driving task shown to posses similar features as point-ot-point human hand reaching movements serves as our example movement. The results show that the identified LQS model outperforms the others with statistical significance. Particularly, the average human steering behavior is modeled significantly better by the LQS model. This validates the positive impact of signal-dependent noise processes on modeling human average behavior. △ Less

Submitted 27 March, 2023; v1 submitted 31 October, 2022; originally announced October 2022.

Inverse Stochastic Optimal Control for Linear-Quadratic Gaussian and Linear-Quadratic Sensorimotor Control Models

Authors: Philipp Karg, Simon Stoll, Simon Rothfuß, Sören Hohmann

Abstract: In this paper, we define and solve the Inverse Stochastic Optimal Control (ISOC) problem of the linear-quadratic Gaussian (LQG) and the linear-quadratic sensorimotor (LQS) control model. These Stochastic Optimal Control (SOC) models are state-of-the-art approaches describing human movements. The LQG ISOC problem consists of finding the unknown weighting matrices of the quadratic cost function and… ▽ More In this paper, we define and solve the Inverse Stochastic Optimal Control (ISOC) problem of the linear-quadratic Gaussian (LQG) and the linear-quadratic sensorimotor (LQS) control model. These Stochastic Optimal Control (SOC) models are state-of-the-art approaches describing human movements. The LQG ISOC problem consists of finding the unknown weighting matrices of the quadratic cost function and the covariance matrices of the additive Gaussian noise processes based on ground truth trajectories observed from the human in practice. The LQS ISOC problem aims at additionally finding the covariance matrices of the signal-dependent noise processes characteristic for the LQS model. We propose a solution to both ISOC problems which iteratively estimates cost function and covariance matrices via two bi-level optimizations. Simulation examples show the effectiveness of our developed algorithm. It finds parameters that yield trajectories matching mean and variance of the ground truth data. △ Less

Submitted 31 October, 2022; originally announced October 2022.

Limited Information Shared Control: A Potential Game Approach

Authors: Balint Varga, Jairo Inga, Soeren Hohmann

Abstract: This paper presents a systematic method for the design of a limited information shared control (LISC). LISC is used in applications where not all system states or reference trajectories are measurable by the automation. Typical examples are partially human-controlled systems, in which some subsystems are fully controlled by automation while others are controlled by a human. The proposed systematic… ▽ More This paper presents a systematic method for the design of a limited information shared control (LISC). LISC is used in applications where not all system states or reference trajectories are measurable by the automation. Typical examples are partially human-controlled systems, in which some subsystems are fully controlled by automation while others are controlled by a human. The proposed systematic design method uses a novel class of games to model human-machine interaction: the near potential differential games (NPDG). We provide a necessary and sufficient condition for the existence of an NPDG and derive an algorithm for finding a NPDG that completely describes a given differential game. The proposed design method is applied to the control of a large vehicle-manipulator system, in which the manipulator is controlled by a human operator and the vehicle is fully automated. The suitability of the NPDG to model differential games is verified in simulations, leading to a faster and more accurate controller design compared to manual tuning. Furthermore, the overall design process is validated in a study with sixteen test subjects, indicating the applicability of the proposed concept in real applications. △ Less

Submitted 1 June, 2022; v1 submitted 17 January, 2022; originally announced January 2022.

Comments: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Excitation for Adaptive Optimal Control of Nonlinear Systems in Differential Games

Authors: Philipp Karg, Florian Köpf, Christian A. Braun, Sören Hohmann

Abstract: This work focuses on the fulfillment of the Persistent Excitation (PE) condition for signals which result from transformations by means of polynomials. This is essential e.g. for the convergence of Adaptive Dynamic Programming algorithms due to commonly used polynomial function approximators. As theoretical statements are scarce regarding the nonlinear transformation of PE signals, we propose cond… ▽ More This work focuses on the fulfillment of the Persistent Excitation (PE) condition for signals which result from transformations by means of polynomials. This is essential e.g. for the convergence of Adaptive Dynamic Programming algorithms due to commonly used polynomial function approximators. As theoretical statements are scarce regarding the nonlinear transformation of PE signals, we propose conditions on the system state such that its transformation by polynomials is PE. To validate our theoretical statements, we develop an exemplary excitation procedure based on our conditions using a feedforward control approach and demonstrate the effectiveness of our method in a nonzero-sum differential game. In this setting, our approach outperforms commonly used probing noise in terms of convergence time and the degree of PE, shown by a numerical example. △ Less

Submitted 20 January, 2022; v1 submitted 5 May, 2021; originally announced May 2021.

Optimal Distributed Frequency and Voltage Control for Zonal Electricity Markets

Authors: Lukas Kölsch, Lena Zellmann, Rishabh Vyas, Martin Pfeifer, Sören Hohmann

Abstract: Zonal pricing is a well-suited mechanism to incentivize grid-supporting behavior of profit-maximizing producers and consumers operating on a large-scale power system. In zonal electricity markets, local system operators create individual price zones, which provide appropriate price signals depending on local grid conditions such as an excess or shortage of electrical energy in certain regions. In… ▽ More Zonal pricing is a well-suited mechanism to incentivize grid-supporting behavior of profit-maximizing producers and consumers operating on a large-scale power system. In zonal electricity markets, local system operators create individual price zones, which provide appropriate price signals depending on local grid conditions such as an excess or shortage of electrical energy in certain regions. In this paper, a real-time zonal pricing controller for AC power networks is presented that ensures frequency and voltage stability as well as Pareto efficiency of the resulting closed-loop equilibria. Based on a dynamic network model that takes line losses and power exchange with adjacent price zones into account, distributed continuous-time control laws are derived which require only neighbor-to-neighbor communication. Application to a real-time congestion management strategy illustrates how spatially and temporally differentiated prices enable grid-supportive operation of the participants without interventions by a superordinate control authority. Effectiveness of different zonal pricing concepts compared to an isolated grid operation is demonstrated through simulations on the IEEE-57 bus system. △ Less

Submitted 19 June, 2021; v1 submitted 15 February, 2021; originally announced February 2021.

Optimal Control of Port-Hamiltonian Systems: A Time-Continuous Learning Approach

Authors: Lukas Kölsch, Pol Jané Soneira, Felix Strehle, Sören Hohmann

Abstract: Feedback controllers for port-Hamiltonian systems reveal an intrinsic inverse optimality property since each passivating state feedback controller is optimal with respect to some specific performance index. Due to the nonlinear port-Hamiltonian system structure, however, explicit (forward) methods for optimal control of port-Hamiltonian systems require the generally intractable analytical solution… ▽ More Feedback controllers for port-Hamiltonian systems reveal an intrinsic inverse optimality property since each passivating state feedback controller is optimal with respect to some specific performance index. Due to the nonlinear port-Hamiltonian system structure, however, explicit (forward) methods for optimal control of port-Hamiltonian systems require the generally intractable analytical solution of the Hamilton-Jacobi-Bellman equation. Adaptive dynamic programming methods provide a means to circumvent this issue. However, the few existing approaches for port-Hamiltonian systems hinge on very specific sub-classes of either performance indices or system dynamics or require the intransparent guessing of stabilizing initial weights. In this paper, we contribute towards closing this largely unexplored research area by proposing a time-continuous adaptive feedback controller for the optimal control of general time-continuous input-state-output port-Hamiltonian systems with respect to general Lagrangian performance indices. Its control law implements an online learning procedure which uses the Hamiltonian of the system as an initial value function candidate. The time-continuous learning of the value function is achieved by means of a certain Lagrange multiplier that allows to evaluate the optimality of the current solution. In particular, constructive conditions for stabilizing initial weights are stated and asymptotic stability of the closed-loop equilibrium is proven. Our work is concluded by simulations for exemplary linear and nonlinear optimization problems which demonstrate asymptotic convergence of the controllers resulting from the proposed online adaptation procedure. △ Less

Submitted 16 July, 2020; originally announced July 2020.

Distributed Frequency and Voltage Control for AC Microgrids based on Primal-Dual Gradient Dynamics

Authors: Lukas Kölsch, Katharina Wieninger, Stefan Krebs, Sören Hohmann

Abstract: With the gradual transformation of power generation towards renewables, distributed energy resources are becoming more and more relevant for grid stabilization. In order to involve all participants in the joint solution of this challenging task, we propose a distributed, model-based and unifying controller for frequency and voltage regulation in AC microgrids, based on steady-state optimal control… ▽ More With the gradual transformation of power generation towards renewables, distributed energy resources are becoming more and more relevant for grid stabilization. In order to involve all participants in the joint solution of this challenging task, we propose a distributed, model-based and unifying controller for frequency and voltage regulation in AC microgrids, based on steady-state optimal control. It not only unifies frequency and voltage control, but also incorporates the classic hierarchy of primary, secondary and tertiary control layers with each closed-loop equilibrium being a minimizer of a user-defined cost function. By considering the individual voltage limits as additional constraints in the corresponding optimization problem, no superordinate specification of voltage setpoints is required. Since the dynamic model of the microgrid has a port-Hamiltonian structure, stability of the overall system can be assessed using shifted passivity properties. Furthermore, we demonstrate the effectiveness of the controller and its robustness against fluctuations in active and reactive power demand by means of numerical examples. △ Less

Submitted 17 December, 2019; originally announced December 2019.