Showing 1–50 of 197 results for author: Dorfler, F

Fairness in Social Influence Maximization via Optimal Transport

Authors: Shubham Chowdhary, Giulia De Pasquale, Nicolas Lanzetti, Ana-Andreea Stoica, Florian Dorfler

Abstract: We study fairness in social influence maximization, whereby one seeks to select seeds that spread a given information throughout a network, ensuring balanced outreach among different communities (e.g. demographic groups). In the literature, fairness is often quantified in terms of the expected outreach within individual communities. In this paper, we demonstrate that such fairness metrics can be m… ▽ More We study fairness in social influence maximization, whereby one seeks to select seeds that spread a given information throughout a network, ensuring balanced outreach among different communities (e.g. demographic groups). In the literature, fairness is often quantified in terms of the expected outreach within individual communities. In this paper, we demonstrate that such fairness metrics can be misleading since they ignore the stochastic nature of information diffusion processes. When information diffusion occurs in a probabilistic manner, multiple outreach scenarios can occur. As such, outcomes such as "in 50% of the cases, no one of group 1 receives the information and everyone in group 2 receives it and in other 50%, the opposite happens", which always results in largely unfair outcomes, are classified as fair by a variety of fairness metrics in the literature. We tackle this problem by designing a new fairness metric, mutual fairness, that captures variability in outreach through optimal transport theory. We propose a new seed selection algorithm that optimizes both outreach and mutual fairness, and we show its efficacy on several real datasets. We find that our algorithm increases fairness with only a minor decrease (and at times, even an increase) in efficiency. △ Less

Submitted 25 June, 2024; originally announced June 2024.

Learning Diffusion at Lightspeed

Authors: Antonio Terpin, Nicolas Lanzetti, Florian Dörfler

Abstract: Diffusion regulates a phenomenal number of natural processes and the dynamics of many successful generative models. Existing models to learn the diffusion terms from observational data rely on complex bilevel optimization problems and properly model only the drift of the system. We propose a new simple model, JKOnet*, which bypasses altogether the complexity of existing architectures while present… ▽ More Diffusion regulates a phenomenal number of natural processes and the dynamics of many successful generative models. Existing models to learn the diffusion terms from observational data rely on complex bilevel optimization problems and properly model only the drift of the system. We propose a new simple model, JKOnet*, which bypasses altogether the complexity of existing architectures while presenting significantly enhanced representational capacity: JKOnet* recovers the potential, interaction, and internal energy components of the underlying diffusion process. JKOnet* minimizes a simple quadratic loss, runs at lightspeed, and drastically outperforms other baselines in practice. Additionally, JKOnet* provides a closed-form optimal solution for linearly parametrized functionals. Our methodology is based on the interpretation of diffusion processes as energy-minimizing trajectories in the probability space via the so-called JKO scheme, which we study via its first-order optimality conditions, in light of few-weeks-old advancements in optimization in the probability space. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Variational Analysis in the Wasserstein Space

Authors: Nicolas Lanzetti, Antonio Terpin, Florian Dörfler

Abstract: We study optimization problems whereby the optimization variable is a probability measure. Since the probability space is not a vector space, many classical and powerful methods for optimization (e.g., gradients) are of little help. Thus, one typically resorts to the abstract machinery of infinite-dimensional analysis or other ad-hoc methodologies, not tailored to the probability space, which howe… ▽ More We study optimization problems whereby the optimization variable is a probability measure. Since the probability space is not a vector space, many classical and powerful methods for optimization (e.g., gradients) are of little help. Thus, one typically resorts to the abstract machinery of infinite-dimensional analysis or other ad-hoc methodologies, not tailored to the probability space, which however involve projections or rely on convexity-type assumptions. We believe instead that these problems call for a comprehensive methodological framework for calculus in probability spaces. In this work, we combine ideas from optimal transport, variational analysis, and Wasserstein gradient flows to equip the Wasserstein space (i.e., the space of probability measures endowed with the Wasserstein distance) with a variational structure, both by combining and extending existing results and introducing novel tools. Our theoretical analysis culminates in very general necessary optimality conditions for optimality. Notably, our conditions (i) resemble the rationales of Euclidean spaces, such as the Karush-Kuhn-Tucker and Lagrange conditions, (ii) are intuitive, informative, and easy to study, and (iii) yield closed-form solutions or can be used to design computationally attractive algorithms. We believe this framework lays the foundation for new algorithmic and theoretical advancements in the study of optimization problems in probability spaces, which we exemplify with numerous case studies and applications to machine learning, drug discovery, and distributionally robust optimization. △ Less

Submitted 15 June, 2024; originally announced June 2024.

NeoRL: Efficient Exploration for Nonepisodic RL

Authors: Bhavya Sukhija, Lenart Treven, Florian Dörfler, Stelian Coros, Andreas Krause

Abstract: We study the problem of nonepisodic reinforcement learning (RL) for nonlinear dynamical systems, where the system dynamics are unknown and the RL agent has to learn from a single trajectory, i.e., without resets. We propose Nonepisodic Optimistic RL (NeoRL), an approach based on the principle of optimism in the face of uncertainty. NeoRL uses well-calibrated probabilistic models and plans optimist… ▽ More We study the problem of nonepisodic reinforcement learning (RL) for nonlinear dynamical systems, where the system dynamics are unknown and the RL agent has to learn from a single trajectory, i.e., without resets. We propose Nonepisodic Optimistic RL (NeoRL), an approach based on the principle of optimism in the face of uncertainty. NeoRL uses well-calibrated probabilistic models and plans optimistically w.r.t. the epistemic uncertainty about the unknown dynamics. Under continuity and bounded energy assumptions on the system, we provide a first-of-its-kind regret bound of $\setO(β_T \sqrt{T Γ_T})$ for general nonlinear systems with Gaussian process dynamics. We compare NeoRL to other baselines on several deep RL environments and empirically demonstrate that NeoRL achieves the optimal average cost while incurring the least regret. △ Less

Submitted 4 June, 2024; v1 submitted 3 June, 2024; originally announced June 2024.

When to Sense and Control? A Time-adaptive Approach for Continuous-Time RL

Authors: Lenart Treven, Bhavya Sukhija, Yarden As, Florian Dörfler, Andreas Krause

Abstract: Reinforcement learning (RL) excels in optimizing policies for discrete-time Markov decision processes (MDP). However, various systems are inherently continuous in time, making discrete-time MDPs an inexact modeling choice. In many applications, such as greenhouse control or medical treatments, each interaction (measurement or switching of action) involves manual intervention and thus is inherently… ▽ More Reinforcement learning (RL) excels in optimizing policies for discrete-time Markov decision processes (MDP). However, various systems are inherently continuous in time, making discrete-time MDPs an inexact modeling choice. In many applications, such as greenhouse control or medical treatments, each interaction (measurement or switching of action) involves manual intervention and thus is inherently costly. Therefore, we generally prefer a time-adaptive approach with fewer interactions with the system. In this work, we formalize an RL framework, Time-adaptive Control & Sensing (TaCoS), that tackles this challenge by optimizing over policies that besides control predict the duration of its application. Our formulation results in an extended MDP that any standard RL algorithm can solve. We demonstrate that state-of-the-art RL algorithms trained on TaCoS drastically reduce the interaction amount over their discrete-time counterpart while retaining the same or improved performance, and exhibiting robustness over discretization frequency. Finally, we propose OTaCoS, an efficient model-based algorithm for our setting. We show that OTaCoS enjoys sublinear regret for systems with sufficiently smooth dynamics and empirically results in further sample-efficiency gains. △ Less

Submitted 4 June, 2024; v1 submitted 3 June, 2024; originally announced June 2024.

Carbon-Aware Computing in a Network of Data Centers: A Hierarchical Game-Theoretic Approach

Authors: Enno Breukelman, Sophie Hall, Giuseppe Belgioioso, Florian Dörfler

Abstract: Over the past decade, the continuous surge in cloud computing demand has intensified data center workloads, leading to significant carbon emissions and driving the need for improving their efficiency and sustainability. This paper focuses on the optimal allocation problem of batch compute loads with temporal and spatial flexibility across a global network of data centers. We propose a bilevel game… ▽ More Over the past decade, the continuous surge in cloud computing demand has intensified data center workloads, leading to significant carbon emissions and driving the need for improving their efficiency and sustainability. This paper focuses on the optimal allocation problem of batch compute loads with temporal and spatial flexibility across a global network of data centers. We propose a bilevel game-theoretic solution approach that captures the inherent hierarchical relationship between supervisory control objectives, such as carbon reduction and peak shaving, and operational objectives, such as priority-aware scheduling. Numerical simulations with real carbon intensity data demonstrate that the proposed approach successfully reduces carbon emissions while simultaneously ensuring operational reliability and priority-aware scheduling. △ Less

Submitted 28 May, 2024; originally announced May 2024.

Advanced Safety Filter for Smooth Transient Operation of a Battery Energy Storage System

Authors: Michael Schneeberger, Florian Dörfler, Silvia Mastellone

Abstract: In this paper, we implement an advanced safety filter to smoothly limit the current of an inverter-based Battery Energy Storage System. The task involves finding suitable Control Barrier Function and Control Lyapunov Function via Sum-of-Squares optimization to certify the system's safety during grid transients. In contrast to the conventional safety filter, the advanced safety filter not only prov… ▽ More In this paper, we implement an advanced safety filter to smoothly limit the current of an inverter-based Battery Energy Storage System. The task involves finding suitable Control Barrier Function and Control Lyapunov Function via Sum-of-Squares optimization to certify the system's safety during grid transients. In contrast to the conventional safety filter, the advanced safety filter not only provides a safety certificate but also achieves finite-time convergence to a nominal region. Within this region, the action of the nominal control, i.e. the Enhanced Direct Power Control, remains unaltered by the safety filter. The advanced safety filter is implemented using a Quadratically Constrained Quadratic Program, providing the capability to also encode quadratic input constraints. Finally, we showcase the effectiveness of the implementation through simulations involving a load step at the Point of Common Coupling, and we compare the outcomes with those obtained using a standard vector current controller. △ Less

Submitted 23 May, 2024; originally announced May 2024.

Distributed Traffic Signal Control via Coordinated Maximum Pressure-plus-Penalty

Authors: Vinzenz Tütsch, Zhiyu He, Florian Dörfler, Kenan Zhang

Abstract: This paper develops an adaptive traffic control policy inspired by Maximum Pressure (MP) while imposing coordination across intersections. The proposed Coordinated Maximum Pressure-plus-Penalty (CMPP) control policy features a local objective for each intersection that consists of the total pressure within the neighborhood and a penalty accounting for the queue capacities and continuous green time… ▽ More This paper develops an adaptive traffic control policy inspired by Maximum Pressure (MP) while imposing coordination across intersections. The proposed Coordinated Maximum Pressure-plus-Penalty (CMPP) control policy features a local objective for each intersection that consists of the total pressure within the neighborhood and a penalty accounting for the queue capacities and continuous green time for certain movements. The corresponding control task is reformulated as a distributed optimization problem and solved via two customized algorithms: one based on the alternating direction method of multipliers (ADMM) and the other follows a greedy heuristic augmented with a majority vote. CMPP not only provides a theoretical guarantee of queuing network stability but also outperforms several benchmark controllers in simulations on a large-scale real traffic network with lower average travel and waiting time per vehicle, as well as less network congestion. Furthermore, CPMM with the greedy algorithm enjoys comparable computational efficiency as fully decentralized controllers without significantly compromising the control performance, which highlights its great potential for real-world deployment. △ Less

Submitted 30 April, 2024; originally announced April 2024.

The Continuous-Time Weighted-Median Opinion Dynamics

Authors: Yi Han, Ge Chen, Florian Dörfler, Wenjun Mei

Abstract: Opinion dynamics models are important in understanding and predicting opinion formation processes within social groups. Although the weighted-averaging opinion-update mechanism is widely adopted as the micro-foundation of opinion dynamics, it bears a non-negligibly unrealistic implication: opinion attractiveness increases with opinion distance. Recently, the weighted-median mechanism has been prop… ▽ More Opinion dynamics models are important in understanding and predicting opinion formation processes within social groups. Although the weighted-averaging opinion-update mechanism is widely adopted as the micro-foundation of opinion dynamics, it bears a non-negligibly unrealistic implication: opinion attractiveness increases with opinion distance. Recently, the weighted-median mechanism has been proposed as a new microscopic mechanism of opinion exchange. Numerous advancements have been achieved regarding this new micro-foundation, from theoretical analysis to empirical validation, in a discrete-time asynchronous setup. However, the original discrete-time weighted-median model does not allow for "compromise behavior" in opinion exchanges, i.e., no intermediate opinions are created between disagreeing agents. To resolve this problem, this paper propose a novel continuous-time weighted-median opinion dynamics model, in which agents' opinions move towards the weighted-medians of their out-neighbors' opinions. It turns out that the proof methods for the original discrete-time asynchronous model are no longer applicable to the analysis of the continuous-time model. In this paper, we first establish the existence and uniqueness of the solution to the continuous-time weighted-median opinion dynamics by showing that the weighted-median map** is contractive on any graph. We also characterize the set of all the equilibria. Then, by leveraging a new LaSalle invariance principle argument, we prove the convergence of the continuous-time weighted-median model for any initial condition and derive a necessary and sufficient condition for the convergence to consensus. △ Less

Submitted 28 April, 2024; v1 submitted 24 April, 2024; originally announced April 2024.

Comments: 13 pages, 1 figure

MSC Class: 91D30(Primary) 93A16(Secondary)

Constrained multi-cluster game: Distributed Nash equilibrium seeking over directed graphs

Authors: Duong Thuy Anh Nguyen, Mattia Bianchi, Florian Dörfler, Duong Tung Nguyen, Angelia Nedić

Abstract: Motivated by the complex dynamics of cooperative and competitive interactions within networked agent systems, multi-cluster games provide a framework for modeling the interconnected goals of self-interested clusters of agents. For this setup, the existing literature lacks comprehensive gradient-based solutions that simultaneously consider constraint sets and directed communication networks, both o… ▽ More Motivated by the complex dynamics of cooperative and competitive interactions within networked agent systems, multi-cluster games provide a framework for modeling the interconnected goals of self-interested clusters of agents. For this setup, the existing literature lacks comprehensive gradient-based solutions that simultaneously consider constraint sets and directed communication networks, both of which are crucial for many practical applications. To address this gap, this paper proposes a distributed Nash equilibrium seeking algorithm that integrates consensus-based methods and gradient-tracking techniques, where inter-cluster and intra-cluster communications only use row- and column-stochastic weight matrices, respectively. To handle constraints, we introduce an averaging procedure, which can effectively address the complications associated with projections. In turn, we can show linear convergence of our algorithm, focusing on the contraction property of the optimality gap. We demonstrate the efficacy of the proposed algorithm through a microgrid energy management application. △ Less

Submitted 22 April, 2024; originally announced April 2024.

Cross-Forming Control and Fault Current Limiting for Grid-Forming Inverters

Authors: Xiuqiang He, Maitraya Avadhut Desai, Linbin Huang, Florian Dörfler

Abstract: We propose a novel "cross-forming" control concept for grid-forming inverters operating against grid faults. Cross-forming refers to voltage angle forming and current magnitude forming, differing from classical grid-forming and grid-following concepts, i.e., voltage magnitude-and-angle forming and current magnitude-and-angle forming, respectively. Unlike purely grid-forming or grid-following parad… ▽ More We propose a novel "cross-forming" control concept for grid-forming inverters operating against grid faults. Cross-forming refers to voltage angle forming and current magnitude forming, differing from classical grid-forming and grid-following concepts, i.e., voltage magnitude-and-angle forming and current magnitude-and-angle forming, respectively. Unlike purely grid-forming or grid-following paradigms, the cross-forming concept is motivated by device security requirements for fault current limitation and meanwhile grid code requirements for voltage angle forming preserving. We propose two feasible cross-forming control implementations, enabling inverters to quickly limit fault currents at a prescribed level and preserve voltage angle forming for grid-forming synchronization and dynamic ancillary services provision, during symmetrical or asymmetrical fault ride-through. Moreover, the cross-forming control yields an equivalent system featuring a constant virtual impedance and a "normal form" of representation, allowing us to extend previously established transient stability results to encompass scenarios of current saturation. Simulations and hardware experiments validate the efficacy of the proposed cross-forming control. △ Less

Submitted 6 May, 2024; v1 submitted 20 April, 2024; originally announced April 2024.

Stability Certificates for Receding Horizon Games

Authors: Sophie Hall, Dominic Liao-McPherson, Giuseppe Belgioioso, Florian Dörfler

Abstract: Game-theoretic MPC (or Receding Horizon Games) is an emerging control methodology for multi-agent systems that generates control actions by solving a dynamic game with coupling constraints in a receding-horizon fashion. This control paradigm has recently received an increasing attention in various application fields, including robotics, autonomous driving, traffic networks, and energy grids, due t… ▽ More Game-theoretic MPC (or Receding Horizon Games) is an emerging control methodology for multi-agent systems that generates control actions by solving a dynamic game with coupling constraints in a receding-horizon fashion. This control paradigm has recently received an increasing attention in various application fields, including robotics, autonomous driving, traffic networks, and energy grids, due to its ability to model the competitive nature of self-interested agents with shared resources while incorporating future predictions, dynamic models, and constraints into the decision-making process. In this work, we present the first formal stability analysis based on dissipativity and monotone operator theory that is valid also for non-potential games. Specifically, we derive LMI-based certificates that ensure asymptotic stability and are numerically verifiable. Moreover, we show that, if the agents have decoupled dynamics, the numerical verification can be performed in a scalable manner. Finally, we present tuning guidelines for the agents' cost function weights to fulfill the certificates and, thus, ensure stability. △ Less

Submitted 18 April, 2024; originally announced April 2024.

Dynamic Complex-Frequency Control of Grid-Forming Converters

Authors: Roger Domingo-Enrich, Xiuqiang He, Verena Häberle, Florian Dörfler

Abstract: Complex droop control, alternatively known as dispatchable virtual oscillator control (dVOC), stands out for its unique capabilities in synchronization and voltage stabilization among existing control strategies for grid-forming converters. Complex droop control leverages the novel concept of ``complex frequency'', thereby establishing a coupled connection between active and reactive power inputs… ▽ More Complex droop control, alternatively known as dispatchable virtual oscillator control (dVOC), stands out for its unique capabilities in synchronization and voltage stabilization among existing control strategies for grid-forming converters. Complex droop control leverages the novel concept of ``complex frequency'', thereby establishing a coupled connection between active and reactive power inputs and frequency and rate-of-change-of voltage outputs. However, its reliance on static droop gains limits its ability to exhibit crucial dynamic response behaviors required in future power systems. To address this limitation, this paper introduces \textit{dynamic complex-frequency control}, upgrading static droop gains with dynamic transfer functions to enhance the richness and flexibility in dynamic responses for frequency and voltage control. Unlike existing approaches, the complex-frequency control framework treats frequency and voltage dynamics collectively, ensuring small-signal stability for frequency synchronization and voltage stabilization simultaneously. The control framework is validated through detailed numerical case studies on the IEEE nine-bus system, also showcasing its applicability in multi-converter setups. △ Less

Submitted 15 April, 2024; originally announced April 2024.

Comments: 6 Pages, 7 Figures

Saturation-Informed Current-Limiting Control for Grid-Forming Converters

Authors: Maitraya Avadhut Desai, Xiuqiang He, Linbin Huang, Florian Dörfler

Abstract: In this paper, we investigate the transient stability of a state-of-the-art grid-forming complex-droop control (i.e., dispatchable virtual oscillator control, dVOC) under current saturation. We quantify the saturation level of a converter by introducing the concept of degree of saturation (DoS), and we propose a provably stable current-limiting control with saturation-informed feedback, which feed… ▽ More In this paper, we investigate the transient stability of a state-of-the-art grid-forming complex-droop control (i.e., dispatchable virtual oscillator control, dVOC) under current saturation. We quantify the saturation level of a converter by introducing the concept of degree of saturation (DoS), and we propose a provably stable current-limiting control with saturation-informed feedback, which feeds the degree of saturation back to the inner voltage-control loop and the outer grid-forming loop. As a result, although the output current is saturated, the voltage phase angle can still be generated from an internal virtual voltage-source node that is governed by an equivalent complex-droop control. We prove that the proposed control achieves transient stability during current saturation under grid faults. We also provide parametric stability conditions for multi-converter systems under grid-connected and islanded scenarios. The stability performance of the current-limiting control is validated with various case studies. △ Less

Submitted 1 July, 2024; v1 submitted 11 April, 2024; originally announced April 2024.

Journal ref: Electric Power Systems Research, 2024

Gray-Box Nonlinear Feedback Optimization

Authors: Zhiyu He, Saverio Bolognani, Michael Muehlebach, Florian Dörfler

Abstract: Feedback optimization enables autonomous optimality seeking of a dynamical system through its closed-loop interconnection with iterative optimization algorithms. Among various iteration structures, model-based approaches require the input-output sensitivity of the system to construct gradients, whereas model-free approaches bypass this need by estimating gradients from real-time evaluations of the… ▽ More Feedback optimization enables autonomous optimality seeking of a dynamical system through its closed-loop interconnection with iterative optimization algorithms. Among various iteration structures, model-based approaches require the input-output sensitivity of the system to construct gradients, whereas model-free approaches bypass this need by estimating gradients from real-time evaluations of the objective. These approaches own complementary benefits in sample efficiency and accuracy against model mismatch, i.e., errors of sensitivities. To achieve the best of both worlds, we propose gray-box feedback optimization controllers, featuring systematic incorporation of approximate sensitivities into model-free updates via adaptive convex combination. We quantify conditions on the accuracy of the sensitivities that render the gray-box approach preferable. We elucidate how the closed-loop performance is determined by the number of iterations, the problem dimension, and the cumulative effect of inaccurate sensitivities. The proposed controller contributes to a balanced closed-loop behavior, which retains provable sample efficiency and optimality guarantees for nonconvex problems. We further develop a running gray-box controller to handle constrained time-varying problems with changing objectives and steady-state maps. △ Less

Submitted 5 April, 2024; originally announced April 2024.

Karma: An Experimental Study

Authors: Ezzat Elokda, Heinrich Nax, Saverio Bolognani, Florian Dörfler

Abstract: A system of non-tradable credits that flow between individuals like karma, hence proposed under that name, is a mechanism for repeated resource allocation that comes with attractive efficiency and fairness properties, in theory. In this study, we test karma in an online experiment in which human subjects repeatedly compete for a resource with time-varying and stochastic individual preferences or u… ▽ More A system of non-tradable credits that flow between individuals like karma, hence proposed under that name, is a mechanism for repeated resource allocation that comes with attractive efficiency and fairness properties, in theory. In this study, we test karma in an online experiment in which human subjects repeatedly compete for a resource with time-varying and stochastic individual preferences or urgency to acquire the resource. We confirm that karma has significant and sustained welfare benefits even in a population with no prior training. We identify mechanism usage in contexts with sporadic high urgency, more so than with frequent moderate urgency, and implemented as an easy (binary) karma bidding scheme as particularly effective for welfare improvements: relatively larger aggregate efficiency gains are realized that are (almost) Pareto superior. These findings provide guidance for further testing and for future implementation plans of such mechanisms in the real world. △ Less

Submitted 3 April, 2024; originally announced April 2024.

Mitigating Transient Bullwhip Effects Under Imperfect Demand Forecasts

Authors: Sarah H. Q. Li, Florian Dörfler

Abstract: Motivated by how forecast errors exacerbate order fluctuations in supply chains, we use tools from robust control theory to characterize and compute the worst-case order fluctuation experienced by an individual supply chain vendor under bounded forecast errors and demand fluctuations. Building on existing discrete time, linear time-invariant (LTI) models of supply chains, we separately model forec… ▽ More Motivated by how forecast errors exacerbate order fluctuations in supply chains, we use tools from robust control theory to characterize and compute the worst-case order fluctuation experienced by an individual supply chain vendor under bounded forecast errors and demand fluctuations. Building on existing discrete time, linear time-invariant (LTI) models of supply chains, we separately model forecast error and demand fluctuations as inputs to the inventory dynamics. We then define a transient Bullwhip measure to evaluate the vendor's worst-case order fluctuation and show that for bounded forecast errors and demand fluctuations, this measure is equivalent to the disturbance to control peak gain. To compute the controller that minimizes the worst-case peak gain, we formulate an optimization problem with bilinear matrix inequalities and show that solving this problem is equivalent to minimizing a quasi-convex function on a bounded domain. In contrast to the existing Bullwhip measure in literature, the transient Bullwhip measure has an explicit dependency on the forecast error and does not need the forecast to be a deterministic function of the demand history. This explicit dependency enables us to separately quantify the transient Bullwhip measure's sensitivity to forecast error and demand fluctuations. We empirically verify our model for vendors with non-zero perishable rates and order backlogging rates. △ Less

Submitted 1 April, 2024; originally announced April 2024.

Comments: 8 pages, 5 figures

Online Feedback Optimization over Networks: A Distributed Model-free Approach

Authors: Wenbin Wang, Zhiyu He, Giuseppe Belgioioso, Saverio Bolognani, Florian Dörfler

Abstract: Online feedback optimization (OFO) enables optimal steady-state operations of a physical system by employing an iterative optimization algorithm as a dynamic feedback controller. When the plant consists of several interconnected sub-systems, centralized implementations become impractical due to the heavy computational burden and the need to pre-compute system-wide sensitivities, which may not be e… ▽ More Online feedback optimization (OFO) enables optimal steady-state operations of a physical system by employing an iterative optimization algorithm as a dynamic feedback controller. When the plant consists of several interconnected sub-systems, centralized implementations become impractical due to the heavy computational burden and the need to pre-compute system-wide sensitivities, which may not be easily accessible in practice. Motivated by these challenges, we develop a fully distributed model-free OFO controller, featuring consensus-based tracking of the global objective value and local iterative (projected) updates that use stochastic gradient estimates. We characterize how the closed-loop performance depends on the size of the network, the number of iterations, and the level of accuracy of consensus. Numerical simulations on a voltage control problem in a direct current power grid corroborate the theoretical findings. △ Less

Submitted 28 March, 2024; originally announced March 2024.

Bridging the Sim-to-Real Gap with Bayesian Inference

Authors: Jonas Rothfuss, Bhavya Sukhija, Lenart Treven, Florian Dörfler, Stelian Coros, Andreas Krause

Abstract: We present SIM-FSVGD for learning robot dynamics from data. As opposed to traditional methods, SIM-FSVGD leverages low-fidelity physical priors, e.g., in the form of simulators, to regularize the training of neural network models. While learning accurate dynamics already in the low data regime, SIM-FSVGD scales and excels also when more data is available. We empirically show that learning with imp… ▽ More We present SIM-FSVGD for learning robot dynamics from data. As opposed to traditional methods, SIM-FSVGD leverages low-fidelity physical priors, e.g., in the form of simulators, to regularize the training of neural network models. While learning accurate dynamics already in the low data regime, SIM-FSVGD scales and excels also when more data is available. We empirically show that learning with implicit physical priors results in accurate mean model estimation as well as precise uncertainty quantification. We demonstrate the effectiveness of SIM-FSVGD in bridging the sim-to-real gap on a high-performance RC racecar system. Using model-based RL, we demonstrate a highly dynamic parking maneuver with drifting, using less than half the data compared to the state of the art. △ Less

Submitted 25 March, 2024; originally announced March 2024.

Decoupling parameter variation from noise: Biquadratic Lyapunov forms in data-driven LPV control

Authors: Chris Verhoek, Jaap Eising, Florian Dörfler, Roland Tóth

Abstract: A promising step from linear towards nonlinear data-driven control is via the design of controllers for linear parameter-varying (LPV) systems, which are linear systems whose parameters are varying along a measurable scheduling signal. However, the interplay between uncertainty arising from corrupted data and the parameter-varying nature of these systems impacts the stability analysis, and limits… ▽ More A promising step from linear towards nonlinear data-driven control is via the design of controllers for linear parameter-varying (LPV) systems, which are linear systems whose parameters are varying along a measurable scheduling signal. However, the interplay between uncertainty arising from corrupted data and the parameter-varying nature of these systems impacts the stability analysis, and limits the generalization of well-understood data-driven methods for linear time-invariant systems. In this work, we decouple this interplay using a recently developed variant of the Fundamental Lemma for LPV systems and the viewpoint of data-informativity, in combination with biquadratic Lyapunov forms. Together, these allow us to develop novel linear matrix inequality conditions for the existence of scheduling-dependent Lyapunov functions, incorporating the intrinsic nonlinearity. Appealingly, these results are stated purely in terms of the collected data and bounds on the noise, and they are computationally favorable to check. △ Less

Submitted 25 March, 2024; originally announced March 2024.

Comments: Submitted for CDC 2024

A Coupled Optimization Framework for Correlated Equilibria in Normal-Form Game

Authors: Sarah H. Q. Li, Yue Yu, Florian Dörfler, John Lygeros

Abstract: In competitive multi-player interactions, simultaneous optimality is a key requirement for establishing strategic equilibria. This property is explicit when the game-theoretic equilibrium is the simultaneously optimal solution of coupled optimization problems. However, no such optimization problems exist for the correlated equilibrium, a strategic equilibrium where the players can correlate their… ▽ More In competitive multi-player interactions, simultaneous optimality is a key requirement for establishing strategic equilibria. This property is explicit when the game-theoretic equilibrium is the simultaneously optimal solution of coupled optimization problems. However, no such optimization problems exist for the correlated equilibrium, a strategic equilibrium where the players can correlate their actions. We address the lack of a coupled optimization framework for the correlated equilibrium by introducing an {unnormalized game} -- an extension of normal-form games in which the player strategies are lifted to unnormalized measures over the joint actions. We show that the set of fully mixed generalized Nash equilibria of this unnormalized game is a subset of the correlated equilibrium of the normal-form game. Furthermore, we introduce an entropy regularization to the unnormalized game and prove that the entropy-regularized generalized Nash equilibrium is a sub-optimal correlated equilibrium of the normal form game where the degree of sub-optimality depends on the magnitude of regularization. We prove that the entropy-regularized unnormalized game has a closed-form solution, and empirically verify its computational efficacy at approximating the correlated equilibrium of normal-form games. △ Less

Submitted 3 April, 2024; v1 submitted 24 March, 2024; originally announced March 2024.

Comments: 8 pages, 2 figures

Optimal control of continuous-time symmetric systems with unknown dynamics and noisy measurements

Authors: Hamed Taghavian, Florian Dorfler, Mikael Johansson

Abstract: An iterative learning algorithm is presented for continuous-time linear-quadratic optimal control problems where the system is externally symmetric with unknown dynamics. Both finite-horizon and infinite-horizon problems are considered. It is shown that the proposed algorithm is globally convergent to the optimal solution and has some advantages over adaptive dynamic programming, including being u… ▽ More An iterative learning algorithm is presented for continuous-time linear-quadratic optimal control problems where the system is externally symmetric with unknown dynamics. Both finite-horizon and infinite-horizon problems are considered. It is shown that the proposed algorithm is globally convergent to the optimal solution and has some advantages over adaptive dynamic programming, including being unbiased under noisy measurements and having a relatively low computational burden. Numerical experiments show the effectiveness of the results. △ Less

Submitted 20 March, 2024; originally announced March 2024.

Control Strategies for Recommendation Systems in Social Networks

Authors: Ben Sprenger, Giulia De Pasquale, Raffaele Soloperto, John Lygeros, Florian Dörfler

Abstract: A closed-loop control model to analyze the impact of recommendation systems on opinion dynamics within social networks is introduced. The core contribution is the development and formalization of model-free and model-based approaches to recommendation system design, integrating the dynamics of social interactions within networks via an extension of the Friedkin-Johnsen (FJ) model. Comparative anal… ▽ More A closed-loop control model to analyze the impact of recommendation systems on opinion dynamics within social networks is introduced. The core contribution is the development and formalization of model-free and model-based approaches to recommendation system design, integrating the dynamics of social interactions within networks via an extension of the Friedkin-Johnsen (FJ) model. Comparative analysis and numerical simulations demonstrate the effectiveness of the proposed control strategies in maximizing user engagement and their potential for influencing opinion formation processes. △ Less

Submitted 10 March, 2024; originally announced March 2024.

To Spend or to Gain: Online Learning in Repeated Karma Auctions

Authors: Damien Berriaud, Ezzat Elokda, Devansh Jalota, Emilio Frazzoli, Marco Pavone, Florian Dörfler

Abstract: Recent years have seen a surge of artificial currency-based mechanisms in contexts where monetary instruments are deemed unfair or inappropriate, e.g., in allocating food donations to food banks, course seats to students, and, more recently, even for traffic congestion management. Yet the applicability of these mechanisms remains limited in repeated auction settings, as it is challenging for users… ▽ More Recent years have seen a surge of artificial currency-based mechanisms in contexts where monetary instruments are deemed unfair or inappropriate, e.g., in allocating food donations to food banks, course seats to students, and, more recently, even for traffic congestion management. Yet the applicability of these mechanisms remains limited in repeated auction settings, as it is challenging for users to learn how to bid an artificial currency that has no value outside the auctions. Indeed, users must jointly learn the value of the currency in addition to how to spend it optimally. In this work, we study the problem of learning to bid in two prominent classes of artificial currency auctions: those in which currency, which users spend to obtain public resources, is only issued at the beginning of a finite period; and those where, in addition to the initial currency endowment, currency payments are redistributed to users at each time step. In the latter class, the currency has been referred to as karma, since users do not only spend karma to obtain public resources but also gain karma for yielding them. In both classes, we propose a simple learning strategy, called adaptive karma pacing, and show that this strategy a) is asymptotically optimal for a single user bidding against competing bids drawn from a stationary distribution; b) leads to convergent learning dynamics when all users adopt it; and c) constitutes an approximate Nash equilibrium as the number of users grows. Our results require a novel analysis in comparison to adaptive pacing strategies in monetary auctions, since we depart from the classical assumption that the currency has known value outside the auctions, and moreover consider that the currency is both spent and gained in the class of auctions with redistribution. △ Less

Submitted 6 March, 2024; originally announced March 2024.

Comments: Manuscript submitted for review to the 25th ACM Conference on Economics & Computation (EC'24)

Tuning and Testing an Online Feedback Optimization Controller to Provide Curative Distribution Grid Flexibility

Authors: Lukas Ortmann, Fabian Böhm, Florian Klein-Helmkamp, Andreas Ulbig, Saverio Bolognani, Florian Dörfler

Abstract: Due to more volatile generation, flexibility will become more important in transmission grids. One potential source of this flexibility can be distribution grids. A flexibility request from the transmission grid to a distribution grid then needs to be split up onto the different flexibility providing units (FPU) in the distribution grid. One potential way to do this is Online Feedback Optimization… ▽ More Due to more volatile generation, flexibility will become more important in transmission grids. One potential source of this flexibility can be distribution grids. A flexibility request from the transmission grid to a distribution grid then needs to be split up onto the different flexibility providing units (FPU) in the distribution grid. One potential way to do this is Online Feedback Optimization (OFO). OFO is a new control method that steers power systems to the optimal solution of an optimization problem using minimal model information and computation power. This paper will show how to choose the optimization problem and how to tune the OFO controller. Afterward, we test the resulting controller on a real distribution grid laboratory and show its performance, its interaction with other controllers in the grid, and how it copes with disturbances. Overall, the paper makes a clear recommendation on how to phrase the optimization problem and tune the OFO controller. Furthermore, it experimentally verifies that an OFO controller is a powerful tool to disaggregate flexibility requests onto FPUs while satisfying operational constraints inside the flexibility providing distribution grid. △ Less

Submitted 4 March, 2024; originally announced March 2024.

Hypergraph reconstruction from dynamics

Authors: Robin Delabays, Giulia De Pasquale, Florian Dörfler, Yuanzhao Zhang

Abstract: A plethora of methods have been developed in the past two decades to infer the underlying network structure of an interconnected system from its collective dynamics. However, methods capable of inferring nonpairwise interactions are only starting to appear. Here, we develop an inference algorithm based on sparse identification of nonlinear dynamics (SINDy) to reconstruct hypergraphs and simplicial… ▽ More A plethora of methods have been developed in the past two decades to infer the underlying network structure of an interconnected system from its collective dynamics. However, methods capable of inferring nonpairwise interactions are only starting to appear. Here, we develop an inference algorithm based on sparse identification of nonlinear dynamics (SINDy) to reconstruct hypergraphs and simplicial complexes from time-series data. Our model-free method does not require information about node dynamics or coupling functions, making it applicable to complex systems that do not have reliable mathematical descriptions. We first benchmark the new method on synthetic data generated from Kuramoto and Lorenz dynamics. We then use it to infer the effective connectivity among seven brain regions from resting-state EEG data, which reveals significant contributions from non-pairwise interactions in sha** the macroscopic brain dynamics. △ Less

Submitted 30 January, 2024; originally announced February 2024.

Comments: Main text: 9 pages, 4 figures. Supp. Mat.: 2 pages, 3 figures

Optimal Dynamic Ancillary Services Provision Based on Local Power Grid Perception

Authors: Verena Häberle, Xiuqiang He, Linbin Huang, Eduardo Prieto-Araujo, Florian Dörfler

Abstract: In this paper, we propose a systematic closed-loop approach to provide optimal dynamic ancillary services with converter-interfaced generation systems based on local power grid perception. In particular, we structurally encode dynamic ancillary services such as fast frequency and voltage regulation in the form of a parametric transfer function matrix, which includes several parameters to define a… ▽ More In this paper, we propose a systematic closed-loop approach to provide optimal dynamic ancillary services with converter-interfaced generation systems based on local power grid perception. In particular, we structurally encode dynamic ancillary services such as fast frequency and voltage regulation in the form of a parametric transfer function matrix, which includes several parameters to define a set of different feasible response behaviors, among which we aim to find the optimal one to be realized by the converter system. Our approach is based on a so-called "perceive-and-optimize" (P&O) strategy: First, we identify a grid dynamic equivalent at the interconnection terminals of the converter system. Second, we consider the closed-loop interconnection of the identified grid equivalent and the parametric transfer function matrix, which we optimize for the set of transfer function parameters, resulting in a stable and optimal closed-loop performance for ancillary services provision. In the process, we ensure that grid-code and device-level requirements are satisfied. Finally, we demonstrate the effectiveness of our approach in different numerical case studies based on a modified Kundur two-area test system. △ Less

Submitted 29 April, 2024; v1 submitted 31 January, 2024; originally announced January 2024.

Comments: 10 pages, 18 Figures

Data-Enabled Policy Optimization for Direct Adaptive Learning of the LQR

Authors: Feiran Zhao, Florian Dörfler, Alessandro Chiuso, Keyou You

Abstract: Direct data-driven design methods for the linear quadratic regulator (LQR) mainly use offline or episodic data batches, and their online adaptation has been acknowledged as an open problem. In this paper, we propose a direct adaptive method to learn the LQR from online closed-loop data. First, we propose a new policy parameterization based on the sample covariance to formulate a direct data-driven… ▽ More Direct data-driven design methods for the linear quadratic regulator (LQR) mainly use offline or episodic data batches, and their online adaptation has been acknowledged as an open problem. In this paper, we propose a direct adaptive method to learn the LQR from online closed-loop data. First, we propose a new policy parameterization based on the sample covariance to formulate a direct data-driven LQR problem, which is shown to be equivalent to the certainty-equivalence LQR with optimal non-asymptotic guarantees. Second, we design a novel data-enabled policy optimization (DeePO) method to directly update the policy, where the gradient is explicitly computed using only a batch of persistently exciting (PE) data. Third, we establish its global convergence via a projected gradient dominance property. Importantly, we efficiently use DeePO to adaptively learn the LQR by performing only one-step projected gradient descent per sample of the closed-loop system, which also leads to an explicit recursive update of the policy. Under PE inputs and for bounded noise, we show that the average regret of the LQR cost is upper-bounded by two terms signifying a sublinear decrease in time $\mathcal{O}(1/\sqrt{T})$ plus a bias scaling inversely with signal-to-noise ratio (SNR), which are independent of the noise statistics. Finally, we perform simulations to validate the theoretical results and demonstrate the computational and sample efficiency of our method. △ Less

Submitted 19 April, 2024; v1 submitted 26 January, 2024; originally announced January 2024.

Towards a Systems Theory of Algorithms

Authors: Florian Dörfler, Zhiyu He, Giuseppe Belgioioso, Saverio Bolognani, John Lygeros, Michael Muehlebach

Abstract: Traditionally, numerical algorithms are seen as isolated pieces of code confined to an {\em in silico} existence. However, this perspective is not appropriate for many modern computational approaches in control, learning, or optimization, wherein {\em in vivo} algorithms interact with their environment. Examples of such {\em open algorithms} include various real-time optimization-based control str… ▽ More Traditionally, numerical algorithms are seen as isolated pieces of code confined to an {\em in silico} existence. However, this perspective is not appropriate for many modern computational approaches in control, learning, or optimization, wherein {\em in vivo} algorithms interact with their environment. Examples of such {\em open algorithms} include various real-time optimization-based control strategies, reinforcement learning, decision-making architectures, online optimization, and many more. Further, even {\em closed} algorithms in learning or optimization are increasingly abstracted in block diagrams with interacting dynamic modules and pipelines. In this opinion paper, we state our vision on a to-be-cultivated {\em systems theory of algorithms} and argue in favor of viewing algorithms as open dynamical systems interacting with other algorithms, physical systems, humans, or databases. Remarkably, the manifold tools developed under the umbrella of systems theory are well suited for addressing a range of challenges in the algorithmic domain. We survey various instances where the principles of algorithmic systems theory are being developed and outline pertinent modeling, analysis, and design challenges. △ Less

Submitted 30 April, 2024; v1 submitted 25 January, 2024; originally announced January 2024.

Power Grid Parameter Estimation Without Phase Measurements: Theory and Empirical Validation

Authors: Jean-Sébastien Brouillon, Keith Moffat, Florian Dörfler, Giancarlo Ferrari-trecate

Abstract: Reliable integration and operation of renewable distributed energy resources requires accurate distribution grid models. However, obtaining precise models is often prohibitively expensive, given their large scale and the ongoing nature of grid operations. To address this challenge, considerable efforts have been devoted to harnessing abundant consumption data for automatic model inference. The pri… ▽ More Reliable integration and operation of renewable distributed energy resources requires accurate distribution grid models. However, obtaining precise models is often prohibitively expensive, given their large scale and the ongoing nature of grid operations. To address this challenge, considerable efforts have been devoted to harnessing abundant consumption data for automatic model inference. The primary result of the paper is that, while the impedance of a line or a network can be estimated without synchronized phase angle measurements in a consistent way, the admittance cannot. Furthermore, a detailed statistical analysis is presented, quantifying the expected estimation errors of four prevalent admittance estimation methods. Such errors constitute fundamental model inference limitations that cannot be resolved with more data. These findings are empirically validated using synthetic data and real measurements from the town of Walenstadt, Switzerland, confirming the theory. The results contribute to our understanding of grid estimation limitations and uncertainties, offering guidance for both practitioners and researchers in the pursuit of more reliable and cost-effective solutions. △ Less

Submitted 18 January, 2024; originally announced January 2024.

Game-theoretic Model Predictive Control for Modelling Competitive Supply Chains

Authors: Sophie Hall, Laura Guerrini, Florian Dörfler, Dominic Liao-McPherson

Abstract: Supply chains transform raw materials into finished goods and distribute them to end consumers. The vast majority of products we use daily are supplied to us through complex global supply chains. This paper proposes a modelling methodology for dynamic competitive supply chains based on game theory and model predictive control. We model each manufacturer in the supply chain as a rational utility ma… ▽ More Supply chains transform raw materials into finished goods and distribute them to end consumers. The vast majority of products we use daily are supplied to us through complex global supply chains. This paper proposes a modelling methodology for dynamic competitive supply chains based on game theory and model predictive control. We model each manufacturer in the supply chain as a rational utility maximizing agent that selects their actions by finding an open-loop generalized Nash equilibrium of a multi-stage game. To react to competitors and the state of the market, every agent re-plans their actions in a receding horizon manner based on estimates of market and supplier parameters thereby creating an approximate closed-loop equilibrium policy. We demonstrate through numerical simulations that this modelling approach is computationally tractable and generates economically interpretable behaviors in a variety of settings such as demand spikes, supply shocks, and information asymmetry. △ Less

Submitted 18 January, 2024; originally announced January 2024.

Advanced safety filter based on SOS Control Barrier and Lyapunov Functions

Authors: Michael Schneeberger, Silvia Mastellone, Florian Dörfler

Abstract: This paper presents a novel safety filter framework based on Control Barrier Functions (CBFs) and Control Lyapunov-like Functions (CLFs). The CBF guarantees forward invariance of the safe set, constraining system trajectories within state constraints, while the CLF guides the system away from unsafe states towards a nominal region, preserving the performance of a nominal controller. The first part… ▽ More This paper presents a novel safety filter framework based on Control Barrier Functions (CBFs) and Control Lyapunov-like Functions (CLFs). The CBF guarantees forward invariance of the safe set, constraining system trajectories within state constraints, while the CLF guides the system away from unsafe states towards a nominal region, preserving the performance of a nominal controller. The first part of this work focuses on determining compatible CBF and CLF in the presence of linear or quadratic input constraints. This is achieved by formulating the CBF and CLF conditions, along with the input constraints, as Sum of Squares (SOS) constraints using Putinar's Positivstellensatz. For solving the resulting SOS optimization problem, we employ an alternating algorithm that simultaneously searches for a feasible controller in the class of rational functions of the state. The second part of this work details the implementation of the safety filter as a Quadratically Constrained Quadratic Program (QCQP), whose constraints encode the CBF and CLF conditions as well as the input constraints. To avoid the chattering effect and guarantee the uniqueness and Lipschitz continuity of solutions, the state-dependent inequality constraints of the QCQP are selected to be sufficiently regular. Finally, we demonstrate the method on a detailed case study involving the control of a three-phase ac/dc power converter connected to an infinite bus. △ Less

Submitted 12 January, 2024; originally announced January 2024.

Comments: 15 pages, 11 figures, submitted to IEEE Transactions on Control Systems Technology

Distributionally Robust Infinite-horizon Control: from a pool of samples to the design of dependable controllers

Authors: Jean-Sébastien Brouillon, Andrea Martin, John Lygeros, Florian Dörfler, Giancarlo Ferrari Trecate

Abstract: We study control of constrained linear systems with only partial statistical information about the uncertainty affecting the system dynamics and the sensor measurements. Specifically, given a finite collection of disturbance realizations drawn from a generic distribution, we consider the problem of designing a stabilizing control policy with provable safety and performance guarantees despite the m… ▽ More We study control of constrained linear systems with only partial statistical information about the uncertainty affecting the system dynamics and the sensor measurements. Specifically, given a finite collection of disturbance realizations drawn from a generic distribution, we consider the problem of designing a stabilizing control policy with provable safety and performance guarantees despite the mismatch between the empirical and true distributions. We capture this discrepancy using Wasserstein ambiguity sets, and we formulate a distributionally robust (DR) optimal control problem, which provides guarantees on the expected cost, safety, and stability of the system. To solve this problem, we first present new results for DR optimization of quadratic objectives using convex programming, showing that strong duality holds under mild conditions. Then, by combining our results with the system-level parametrization of linear feedback policies, we show that the design problem can be reduced to a semidefinite program. We present numerical simulations to validate the effectiveness of our approach and to highlight the value of empirical distributions for control design. △ Less

Submitted 11 July, 2024; v1 submitted 12 December, 2023; originally announced December 2023.

Urban traffic congestion control: a DeePC change

Authors: Alessio Rimoldi, Carlo Cenedese, Alberto Padoan, Florian Dörfler, John Lygeros

Abstract: Urban traffic congestion remains a pressing challenge in our rapidly expanding cities, despite the abundance of available data and the efforts of policymakers. By leveraging behavioral system theory and data-driven control, this paper exploits the DeePC algorithm in the context of urban traffic control performed via dynamic traffic lights. To validate our approach, we consider a high-fidelity case… ▽ More Urban traffic congestion remains a pressing challenge in our rapidly expanding cities, despite the abundance of available data and the efforts of policymakers. By leveraging behavioral system theory and data-driven control, this paper exploits the DeePC algorithm in the context of urban traffic control performed via dynamic traffic lights. To validate our approach, we consider a high-fidelity case study using the state-of-the-art simulation software package Simulation of Urban MObility (SUMO). Preliminary results indicate that DeePC outperforms existing approaches across various key metrics, including travel time and CO$_2$ emissions, demonstrating its potential for effective traffic management △ Less

Submitted 16 November, 2023; originally announced November 2023.

Comments: This paper has been submitted to IEEE ECC24

Decentralized Feedback Optimization via Sensitivity Decoupling: Stability and Sub-optimality

Authors: Wenbin Wang, Zhiyu He, Giuseppe Belgioioso, Saverio Bolognani, Florian Dörfler

Abstract: Online feedback optimization is a controller design paradigm for optimizing the steady-state behavior of a dynamical system. It employs an optimization algorithm as a dynamic feedback controller and utilizes real-time measurements to bypass knowing exact plant dynamics and disturbances. Different from existing centralized settings, we present a fully decentralized feedback optimization controller… ▽ More Online feedback optimization is a controller design paradigm for optimizing the steady-state behavior of a dynamical system. It employs an optimization algorithm as a dynamic feedback controller and utilizes real-time measurements to bypass knowing exact plant dynamics and disturbances. Different from existing centralized settings, we present a fully decentralized feedback optimization controller for networked systems to lift the communication burden and improve scalability. We approximate the overall input-output sensitivity matrix through its diagonal elements, which capture local model information. For the closed-loop behavior, we characterize the stability and bound the sub-optimality due to decentralization. We prove that the proposed decentralized controller yields solutions that correspond to the Nash equilibria of a non-cooperative game. △ Less

Submitted 28 March, 2024; v1 submitted 15 November, 2023; originally announced November 2023.

Efficient Exploration in Continuous-time Model-based Reinforcement Learning

Authors: Lenart Treven, Jonas Hübotter, Bhavya Sukhija, Florian Dörfler, Andreas Krause

Abstract: Reinforcement learning algorithms typically consider discrete-time dynamics, even though the underlying systems are often continuous in time. In this paper, we introduce a model-based reinforcement learning algorithm that represents continuous-time dynamics using nonlinear ordinary differential equations (ODEs). We capture epistemic uncertainty using well-calibrated probabilistic models, and use t… ▽ More Reinforcement learning algorithms typically consider discrete-time dynamics, even though the underlying systems are often continuous in time. In this paper, we introduce a model-based reinforcement learning algorithm that represents continuous-time dynamics using nonlinear ordinary differential equations (ODEs). We capture epistemic uncertainty using well-calibrated probabilistic models, and use the optimistic principle for exploration. Our regret bounds surface the importance of the measurement selection strategy(MSS), since in continuous time we not only must decide how to explore, but also when to observe the underlying system. Our analysis demonstrates that the regret is sublinear when modeling ODEs with Gaussian Processes (GP) for common choices of MSS, such as equidistant sampling. Additionally, we propose an adaptive, data-dependent, practical MSS that, when combined with GP dynamics, also achieves sublinear regret with significantly fewer samples. We showcase the benefits of continuous-time modeling over its discrete-time counterpart, as well as our proposed adaptive MSS over standard baselines, on several applications. △ Less

Submitted 30 October, 2023; originally announced October 2023.

Tutorial on Congestion Control in Multi-Area Transmission Grids via Online Feedback Equilibrium Seeking

Authors: Giuseppe Belgioioso, Saverio Bolognani, Giulia Pejrani, Florian Dörfler

Abstract: Online feedback optimization (OFO) is an emerging control methodology for real-time optimal steady-state control of complex dynamical systems. This tutorial focuses on the application of OFO for the autonomous operation of large-scale transmission grids, with a specific goal of minimizing renewable generation curtailment and losses while satisfying voltage and current limits. When this control met… ▽ More Online feedback optimization (OFO) is an emerging control methodology for real-time optimal steady-state control of complex dynamical systems. This tutorial focuses on the application of OFO for the autonomous operation of large-scale transmission grids, with a specific goal of minimizing renewable generation curtailment and losses while satisfying voltage and current limits. When this control methodology is applied to multi-area transmission grids, where each area independently manages its congestion while being dynamically interconnected with the rest of the grid, a non-cooperative game arises. In this context, OFO must be interpreted as an online feedback equilibrium seeking (FES) scheme. Our analysis incorporates technical tools from game theory and monotone operator theory to evaluate the stability and performance of multi-area grid operation. Through numerical simulations, we illustrate the key challenge of this non-cooperative setting: on the one hand, independent multi-area decisions are suboptimal compared to a centralized control scheme; on the other hand, some areas are heavily penalized by the centralized decision, which may discourage participation in the coordination mechanism. △ Less

Submitted 24 October, 2023; originally announced October 2023.

Data-driven representations of conical, convex, and affine behaviors

Authors: Alberto Padoan, Florian Dörfler, John Lygeros

Abstract: The paper studies conical, convex, and affine models in the framework of behavioral systems theory. We investigate basic properties of such behaviors and address the problem of constructing models from measured data. We prove that closed, shift-invariant, conical, convex, and affine models have the intersection property, thereby enabling the definition of most powerful unfalsified models based on… ▽ More The paper studies conical, convex, and affine models in the framework of behavioral systems theory. We investigate basic properties of such behaviors and address the problem of constructing models from measured data. We prove that closed, shift-invariant, conical, convex, and affine models have the intersection property, thereby enabling the definition of most powerful unfalsified models based on infinite-horizon measurements. We then provide necessary and sufficient conditions for representing conical, convex, and affine finite-horizon behaviors using raw data matrices, expressing persistence of excitation requirements in terms of non-negative rank conditions. The applicability of our results is demonstrated by a numerical example arising in population ecology. △ Less

Submitted 23 October, 2023; originally announced October 2023.

Controller implementability: a data-driven approach

Authors: Alberto Padoan, Jeremy Coulson, Florian Dörfler

Abstract: We study the controller implementability problem, which seeks to determine if a controller can make the closed-loop behavior of a given plant match that of a desired reference behavior. We establish necessary and sufficient conditions for controller implementability which only rely on raw data. Subsequently, we consider the problem of constructing controllers directly from data. By leveraging the… ▽ More We study the controller implementability problem, which seeks to determine if a controller can make the closed-loop behavior of a given plant match that of a desired reference behavior. We establish necessary and sufficient conditions for controller implementability which only rely on raw data. Subsequently, we consider the problem of constructing controllers directly from data. By leveraging the concept of canonical controller, we provide a formula to directly construct controllers that implement plant-compatible reference behaviors using measurements of both reference and plant behaviors. △ Less

Submitted 23 October, 2023; originally announced October 2023.

Passivity and Decentralized Stability Conditions for Grid-Forming Converters

Authors: Xiuqiang He, Florian Dörfler

Abstract: We prove that the popular grid-forming control, i.e., dispatchable virtual oscillator control (dVOC), also termed complex droop control, exhibits output-feedback passivity in its large-signal model, featuring an explicit and physically meaningful passivity index. Using this passivity property, we derive decentralized stability conditions for the transient stability of dVOC in multi-converter grid-… ▽ More We prove that the popular grid-forming control, i.e., dispatchable virtual oscillator control (dVOC), also termed complex droop control, exhibits output-feedback passivity in its large-signal model, featuring an explicit and physically meaningful passivity index. Using this passivity property, we derive decentralized stability conditions for the transient stability of dVOC in multi-converter grid-connected systems, beyond prior small-signal stability results. The decentralized conditions are of practical significance, particularly for ensuring the transient stability of renewable power plants under grid disturbances. △ Less

Submitted 2 February, 2024; v1 submitted 15 October, 2023; originally announced October 2023.

Journal ref: IEEE Transactions on Power Systems, 2024

Quantitative Stability Conditions for Grid-Forming Converters With Complex Droop Control

Authors: Xiuqiang He, Linbin Huang, Irina Subotić, Verena Häberle, Florian Dörfler

Abstract: In this paper, we analytically study the transient stability of grid-connected converters with grid-forming complex droop control, also known as dispatchable virtual oscillator control. We prove theoretically that complex droop control, as a state-of-the-art grid-forming control, always possesses steady-state equilibria whereas classical droop control does not. We provide quantitative conditions f… ▽ More In this paper, we analytically study the transient stability of grid-connected converters with grid-forming complex droop control, also known as dispatchable virtual oscillator control. We prove theoretically that complex droop control, as a state-of-the-art grid-forming control, always possesses steady-state equilibria whereas classical droop control does not. We provide quantitative conditions for complex droop control maintaining transient stability (global asymptotic stability) under grid disturbances, which is beyond the well-established local (non-global) stability for classical droop control. For the transient instability of complex droop control, we reveal that the unstable trajectories are bounded, manifesting as limit cycle oscillations. Moreover, we extend our stability results from second-order grid-forming control dynamics to full-order system dynamics that additionally encompass both circuit electromagnetic transients and inner-loop dynamics. Our theoretical results contribute an insightful understanding of the transient stability and instability of complex droop control and offer practical guidelines for parameter tuning and stability guarantees. △ Less

Submitted 23 May, 2024; v1 submitted 15 October, 2023; originally announced October 2023.

Journal ref: IEEE Transactions on Power Electronics, 2024

Dynamic Ancillary Services: From Grid Codes to Transfer Function-Based Converter Control

Authors: Verena Häberle, Linbin Huang, Xiuqiang He, Eduardo Prieto-Araujo, Florian Dörfler

Abstract: Conventional grid-code specifications for dynamic ancillary services provision such as fast frequency and voltage regulation are typically defined by means of piece-wise linear step-response capability curves in the time domain. However, although the specification of such time-domain curves is straightforward, their practical implementation in a converter-based generation system is not immediate,… ▽ More Conventional grid-code specifications for dynamic ancillary services provision such as fast frequency and voltage regulation are typically defined by means of piece-wise linear step-response capability curves in the time domain. However, although the specification of such time-domain curves is straightforward, their practical implementation in a converter-based generation system is not immediate, and no customary methods have been developed yet. In this paper, we thus propose a systematic approach for the practical implementation of piece-wise linear time-domain curves to provide dynamic ancillary services by converter-based generation systems, while ensuring grid-code and device-level requirements to be reliably satisfied. Namely, we translate the piece-wise linear time-domain curves for active and reactive power provision in response to a frequency and voltage step change into a desired rational parametric transfer function in the frequency domain, which defines a dynamic response behavior to be realized by the converter. The obtained transfer function can be easily implemented e.g. via a PI-based matching control in the power loop of standard converter control architectures. We demonstrate the performance of our method in numerical grid-code compliance tests, and reveal its superiority over classical droop and virtual inertia schemes which may not satisfy the grid codes due to their structural limitations. △ Less

Submitted 22 February, 2024; v1 submitted 2 October, 2023; originally announced October 2023.

Comments: 8 pages, 11 figures

Physics-Informed Graph Neural Network for Dynamic Reconfiguration of Power Systems

Authors: Jules Authier, Rabab Haider, Anuradha Annaswamy, Florian Dorfler

Abstract: To maintain a reliable grid we need fast decision-making algorithms for complex problems like Dynamic Reconfiguration (DyR). DyR optimizes distribution grid switch settings in real-time to minimize grid losses and dispatches resources to supply loads with available generation. DyR is a mixed-integer problem and can be computationally intractable to solve for large grids and at fast timescales. We… ▽ More To maintain a reliable grid we need fast decision-making algorithms for complex problems like Dynamic Reconfiguration (DyR). DyR optimizes distribution grid switch settings in real-time to minimize grid losses and dispatches resources to supply loads with available generation. DyR is a mixed-integer problem and can be computationally intractable to solve for large grids and at fast timescales. We propose GraPhyR, a Physics-Informed Graph Neural Network (GNNs) framework tailored for DyR. We incorporate essential operational and connectivity constraints directly within the GNN framework and train it end-to-end. Our results show that GraPhyR is able to learn to optimize the DyR task. △ Less

Submitted 2 April, 2024; v1 submitted 1 October, 2023; originally announced October 2023.

Comments: 8 pages, 5 figures, 2 tables. To appear at PSCC 2024

The Impact of Recommendation Systems on Opinion Dynamics: Microscopic versus Macroscopic Effects

Authors: Nicolas Lanzetti, Florian Dörfler, Nicolò Pagan

Abstract: Recommendation systems are widely used in web services, such as social networks and e-commerce platforms, to serve personalized content to the users and, thus, enhance their experience. While personalization assists users in navigating through the available options, there have been growing concerns regarding its repercussions on the users and their opinions. Examples of negative impacts include th… ▽ More Recommendation systems are widely used in web services, such as social networks and e-commerce platforms, to serve personalized content to the users and, thus, enhance their experience. While personalization assists users in navigating through the available options, there have been growing concerns regarding its repercussions on the users and their opinions. Examples of negative impacts include the emergence of filter bubbles and the amplification of users' confirmation bias, which can cause opinion polarization and radicalization. In this paper, we study the impact of recommendation systems on users, both from a microscopic (i.e., at the level of individual users) and a macroscopic (i.e., at the level of a homogenous population) perspective. Specifically, we build on recent work on the interactions between opinion dynamics and recommendation systems to propose a model for this closed loop, which we then study both analytically and numerically. Among others, our analysis reveals that shifts in the opinions of individual users do not always align with shifts in the opinion distribution of the population. In particular, even in settings where the opinion distribution appears unaltered (e.g., measured via surveys across the population), the opinion of individual users might be significantly distorted by the recommendation system. △ Less

Submitted 7 December, 2023; v1 submitted 16 September, 2023; originally announced September 2023.

Comments: Accepted for presentation at, and publication in the proceedings of, the 62nd IEEE Conference on Decision and Control

Gain and Phase: Decentralized Stability Conditions for Power Electronics-Dominated Power Systems

Authors: Linbin Huang, Dan Wang, Xiongfei Wang, Huanhai Xin, ** Ju, Karl H. Johansson, Florian Dörfler

Abstract: This paper proposes decentralized stability conditions for multi-converter systems based on the combination of the small gain theorem and the small phase theorem. Instead of directly computing the closed-loop dynamics, e.g., eigenvalues of the state-space matrix, or using the generalized Nyquist stability criterion, the proposed stability conditions are more scalable and computationally lighter, w… ▽ More This paper proposes decentralized stability conditions for multi-converter systems based on the combination of the small gain theorem and the small phase theorem. Instead of directly computing the closed-loop dynamics, e.g., eigenvalues of the state-space matrix, or using the generalized Nyquist stability criterion, the proposed stability conditions are more scalable and computationally lighter, which aim at evaluating the closed-loop system stability by comparing the individual converter dynamics with the network dynamics in a decentralized and open-loop manner. Moreover, our approach can handle heterogeneous converters' dynamics and is suitable to analyze large-scale multi-converter power systems that contain grid-following (GFL), grid-forming (GFM) converters, and synchronous generators. Compared with other decentralized stability conditions, e.g., passivity-based stability conditions, the proposed conditions are significantly less conservative and can be generally satisfied in practice across the whole frequency range. △ Less

Submitted 10 January, 2024; v1 submitted 14 September, 2023; originally announced September 2023.

Nash equilibrium seeking over digraphs with row-stochastic matrices and network-independent step-sizes

Authors: Duong Thuy Anh Nguyen, Mattia Bianchi, Florian Dörfler, Duong Tung Nguyen, Angelia Nedić

Abstract: In this paper, we address the challenge of Nash equilibrium (NE) seeking in non-cooperative convex games with partial-decision information. We propose a distributed algorithm, where each agent refines its strategy through projected-gradient steps and an averaging procedure. Each agent uses estimates of competitors' actions obtained solely from local neighbor interactions, in a directed communicati… ▽ More In this paper, we address the challenge of Nash equilibrium (NE) seeking in non-cooperative convex games with partial-decision information. We propose a distributed algorithm, where each agent refines its strategy through projected-gradient steps and an averaging procedure. Each agent uses estimates of competitors' actions obtained solely from local neighbor interactions, in a directed communication network. Unlike previous approaches that rely on (strong) monotonicity assumptions, this work establishes the convergence towards a NE under a diagonal dominance property of the pseudo-gradient map**, that can be checked locally by the agents. Further, this condition is physically interpretable and of relevance for many applications, as it suggests that an agent's objective function is primarily influenced by its individual strategic decisions, rather than by the actions of its competitors. In virtue of a novel block-infinity norm convergence argument, we provide explicit bounds for constant step-size that are independent of the communication structure, and can be computed in a totally decentralized way. Numerical simulations on an optical network's power control problem validate the algorithm's effectiveness. △ Less

Submitted 14 September, 2023; originally announced September 2023.

Joint Oscillation Dam** and Inertia Provision Service for Converter-Interfaced Generation

Authors: Cheng Feng, Linbin Huang, Xiuqiang He, Yi Wang, Florian Dörfler, Qixin Chen

Abstract: As renewable generation becomes more prevalent, traditional power systems dominated by synchronous generators are transitioning to systems dominated by converter-interfaced generation. These devices, with their weaker dam** capabilities and lower inertia, compromise the system's ability to withstand disturbances, pose a threat to system stability, and lead to oscillations and poor frequency resp… ▽ More As renewable generation becomes more prevalent, traditional power systems dominated by synchronous generators are transitioning to systems dominated by converter-interfaced generation. These devices, with their weaker dam** capabilities and lower inertia, compromise the system's ability to withstand disturbances, pose a threat to system stability, and lead to oscillations and poor frequency response performance. While some new converter-interfaced generations are capable of providing superior dam** and fast frequency control, there is a lack of effective measures to incentivize manufacturers to adopt them. To address this gap, this paper defines the joint oscillation dam** and inertia provision services at the system level, seeking to encourage converter-interfaced generation to provide enhanced dam** and fast frequency response capabilities. Our approach is anchored in a novel convex parametric formulation that combines oscillation mode and frequency stability constraints. These constraints ensure a sufficient dam** ratio for all oscillation modes and maintain transient frequency trajectories within acceptable limits. They are designed to integrate smoothly into various operational and planning optimization frameworks. Using this formulation, we introduce a joint service for oscillation dam** and inertia provision based on a cost-minimization problem. This facilitates the optimal allocation of dam** and virtual inertia to converters, achieving both small-signal stability and frequency stability. Furthermore, we investigate the economic effects of introducing this service into a new ancillary service market, assessing its impact on system operations and cost-efficiency. Numerical tests highlight the service's efficacy in ensuring both small-signal stability and frequency stability, and offer insights into potential economic benefits. △ Less

Submitted 3 September, 2023; originally announced September 2023.

Comments: Submitted for IEEE PES journal for possible publications

Experimental Validation of a Dynamic Virtual Power Plant Concept Based on Multiple-Converter Power Hardware-In-the-Loop Test Bench

Authors: Moritz Andrejewski, Verena Häberle, Nico Goldschmidt, Florian Dörfler, Horst Schulte

Abstract: Recently, the concept of dynamic virtual power plants (DVPP) has been proposed to collectively provide desired dynamic ancillary services such as fast frequency and voltage control by a heterogeneous ensemble of distributed energy resources (DER). This paper presents an experimental validation of a recent DVPP control design approach on a multi-converter power hardware-in-the-loop (PHIL) test bed… ▽ More Recently, the concept of dynamic virtual power plants (DVPP) has been proposed to collectively provide desired dynamic ancillary services such as fast frequency and voltage control by a heterogeneous ensemble of distributed energy resources (DER). This paper presents an experimental validation of a recent DVPP control design approach on a multi-converter power hardware-in-the-loop (PHIL) test bed system. More specifically, we consider a DVPP composed of a wind generation system, a photovoltaic (PV) system, and a STATCOM with small storage capacity to collectively provide grid-following fast frequency regulation in the presence of grid-frequency and load variations. The performance of the aggregated DVPP response is evaluated with respect to its ability to match a desired dynamic behavior while taking practical limitations of the individual DVPP units into account. △ Less

Submitted 2 September, 2023; originally announced September 2023.

Comments: 8 Pages, 11 Figures, 22nd Wind & Solar Integration Workshop 2023

Strategic Interactions in Multi-modal Mobility Systems: A Game-Theoretic Perspective

Authors: Gioele Zardini, Nicolas Lanzetti, Giuseppe Belgioioso, Christian Hartnik, Saverio Bolognani, Florian Dörfler, Emilio Frazzoli

Abstract: The evolution of existing transportation systems,mainly driven by urbanization and increased availability of mobility options, such as private, profit-maximizing ride-hailing companies, calls for tools to reason about their design and regulation. To study this complex socio-technical problem, one needs to account for the strategic interactions of the heterogeneous stakeholders involved in the mobi… ▽ More The evolution of existing transportation systems,mainly driven by urbanization and increased availability of mobility options, such as private, profit-maximizing ride-hailing companies, calls for tools to reason about their design and regulation. To study this complex socio-technical problem, one needs to account for the strategic interactions of the heterogeneous stakeholders involved in the mobility ecosystem and analyze how they influence the system. In this paper, we focus on the interactions between citizens who compete for the limited resources of a mobility system to complete their desired trip. Specifically, we present a game-theoretic framework for multi-modal mobility systems, where citizens, characterized by heterogeneous preferences, have access to various mobility options and seek individually-optimal decisions. We study the arising game and prove the existence of an equilibrium, which can be efficiently computed via a convex optimization problem. Through both an analytical and a numerical case study for the classic scenario of Sioux Falls, USA, we illustrate the capabilities of our model and perform sensitivity analyses. Importantly, we show how to embed our framework into a "larger" game among stakeholders of the mobility ecosystem (e.g., municipality, Mobility Service Providers, and citizens), effectively giving rise to tools to inform strategic interventions and policy-making in the mobility ecosystem. △ Less

Submitted 9 August, 2023; originally announced August 2023.

Comments: 8 pages, 5 figures, to appear in the proceedings of the 2023 IEEE 26th International Conference on Intelligent Transportation Systems

Grid-Forming Hybrid Angle Control: Behavior, Stability, Variants and Verification

Authors: Ali Tayyebi, Denis Vettoretti, Adolfo Anta, Florian Dörfler

Abstract: This work explores the stability, behavior, variants, and a controller-hardware-in-the-loop (C-HiL) verification of the recently proposed grid-forming (GFM) hybrid angle control (HAC). We revisit the foundation of GFM HAC, and highlight its behavioral properties in relation to the conventional synchronous machine (SM). Next, we introduce the required complementary controls to be combined with the… ▽ More This work explores the stability, behavior, variants, and a controller-hardware-in-the-loop (C-HiL) verification of the recently proposed grid-forming (GFM) hybrid angle control (HAC). We revisit the foundation of GFM HAC, and highlight its behavioral properties in relation to the conventional synchronous machine (SM). Next, we introduce the required complementary controls to be combined with the HAC to realize a GFM behavior. The characterization of the analytical operating point and nonlinear energy-based stability analysis of a grid-connected converter under the HAC is presented. Further, we consider various output filter configurations and derive an approximation for the original control proposal. Moreover, we provide details on the integration of GFM HAC into a complex converter control architecture and introduce several variants of the standard HAC. Finally, the performance of GFM HAC is verified by several test scenarios in a C-HiL setup to test its behavior against real-world effect such as noise and delays. △ Less

Submitted 18 July, 2023; originally announced July 2023.