Showing 1–6 of 6 results for author: Ochoa, D E

Decentralized Concurrent Learning with Coordinated Momentum and Restart

Authors: Daniel E. Ochoa, Muhammad U. Javed, Xudong Chen, Jorge I. Poveda

Abstract: This paper studies the stability and convergence properties of a class of multi-agent concurrent learning (CL) algorithms with momentum and restart. Such algorithms can be integrated as part of the estimation pipelines of data-enabled multi-agent control systems to enhance transient performance while maintaining stability guarantees. However, characterizing restarting policies that yield stable be… ▽ More This paper studies the stability and convergence properties of a class of multi-agent concurrent learning (CL) algorithms with momentum and restart. Such algorithms can be integrated as part of the estimation pipelines of data-enabled multi-agent control systems to enhance transient performance while maintaining stability guarantees. However, characterizing restarting policies that yield stable behaviors in decentralized CL systems, especially when the network topology of the communication graph is directed, has remained an open problem. In this paper, we provide an answer to this problem by synergistically leveraging tools from graph theory and hybrid dynamical systems theory. Specifically, we show that under a cooperative richness condition on the overall multi-agent system's data, and by employing coordinated periodic restart with a frequency that is tempered by the level of asymmetry of the communication graph, the resulting decentralized dynamics exhibit robust asymptotic stability properties, characterized in terms of input-to-state stability bounds, and also achieve a desirable transient performance. To demonstrate the practical implications of the theoretical findings, three applications are also presented: cooperative parameter estimation over networks with private data sets, cooperative model-reference adaptive control, and cooperative data-enabled feedback optimization of nonlinear plants. △ Less

Submitted 20 June, 2024; originally announced June 2024.

Prescribed-Time Control in Switching Systems with Resets: A Hybrid Dynamical Systems Approach

Authors: Daniel E. Ochoa, Nicolas Espitia, Jorge I. Poveda

Abstract: We consider the problem of achieving prescribed-time stability (PT-S) in a class of hybrid dynamical systems that incorporate switching nonlinear dynamics, exogenous inputs, and resets. By "prescribed-time stability", we refer to the property of having the states converge to a particular compact set of interest before a given time defined a priori by the user. We focus on dynamical systems that ac… ▽ More We consider the problem of achieving prescribed-time stability (PT-S) in a class of hybrid dynamical systems that incorporate switching nonlinear dynamics, exogenous inputs, and resets. By "prescribed-time stability", we refer to the property of having the states converge to a particular compact set of interest before a given time defined a priori by the user. We focus on dynamical systems that achieve this property via time-varying gains. For continuous-time systems, this approach has received significant attention in recent years, with various applications in control, optimization, and estimation problems. However, its extensions beyond continuous-time systems have been limited. This gap motivates this paper, which introduces a novel class of switching conditions for switching systems with resets that incorporate time-varying gains, ensuring the PT-S property even in the presence of unstable modes. The analysis leverages tools from hybrid dynamical system's theory, and a contraction-dilation property that is established for the hybrid time domains of the solutions of the system. We present the model and main results in a general framework and subsequently apply them to three novel applications: (a) PT regulation of switching plants with no common Lyapunov functions; (b) PT control of dynamic plants with uncertainty and intermittent feedback; and (c) PT decision-making in non-cooperative switching games via hybrid Nash seeking dynamics. △ Less

Submitted 30 August, 2023; originally announced August 2023.

Model-Free Optimization on Smooth Compact Manifolds: Overcoming Topological Obstructions using Zeroth-Order Hybrid Dynamics

Authors: Daniel E. Ochoa, Jorge I. Poveda

Abstract: Smooth autonomous dynamical systems modeled by ordinary differential equations (ODEs) cannot robustly and globally stabilize a point in compact, boundaryless manifolds. This obstruction, which is topological in nature, implies that traditional smooth optimization dynamics are not able to robustly solve global optimization problems in such spaces. In turn, model-free optimization algorithms, which… ▽ More Smooth autonomous dynamical systems modeled by ordinary differential equations (ODEs) cannot robustly and globally stabilize a point in compact, boundaryless manifolds. This obstruction, which is topological in nature, implies that traditional smooth optimization dynamics are not able to robustly solve global optimization problems in such spaces. In turn, model-free optimization algorithms, which usually inherit their stability and convergence properties from their model-based counterparts, might also suffer from similar topological obstructions. For example, this is the case in zeroth-order methods and perturbation-based techniques, where gradients and Hessians are usually estimated in real time via measurements or evaluations of the cost function. To address this issue, we introduce a class of hybrid model-free optimization dynamics that combine continuous-time and discrete-time feedback to overcome the obstructions that emerge in traditional ODE-based optimization algorithms evolving on smooth compact manifolds. In particular, we introduce a hybrid controller that switches between different model-free feedback-laws obtained by applying suitable exploratory geodesic dithers to a family of synergistic diffeomorphisms adapted to the cost function that defines the optimization problem. The geodesic dithers enable exploration of the manifold while preserving its forward invariance, a property that is important for many practical applications where physics-based constraints limit the feasible trajectories of the system. The hybrid controller exploits the information obtained from the geodesic dithers to achieve robust global practical stability of the set of minimizers of the cost. The proposed method is of model-free nature since it only requires measurements or evaluations of the cost function. Numerical results are presented to illustrate the main ideas and advantages of the method. △ Less

Submitted 6 December, 2022; originally announced December 2022.

Accelerated Continuous-Time Approximate Dynamic Programming via Data-Assisted Hybrid Control

Authors: Daniel E. Ochoa, Jorge I. Poveda

Abstract: We introduce a new closed-loop architecture for the online solution of approximate optimal control problems in the context of continuous-time systems. Specifically, we introduce the first algorithm that incorporates dynamic momentum in actor-critic structures to control continuous-time dynamic plants with an affine structure in the input. By incorporating dynamic momentum in our algorithm, we are… ▽ More We introduce a new closed-loop architecture for the online solution of approximate optimal control problems in the context of continuous-time systems. Specifically, we introduce the first algorithm that incorporates dynamic momentum in actor-critic structures to control continuous-time dynamic plants with an affine structure in the input. By incorporating dynamic momentum in our algorithm, we are able to accelerate the convergence properties of the closed-loop system, achieving superior transient performance compared to traditional gradient-descent based techniques. In addition, by leveraging the existence of past recorded data with sufficiently rich information properties, we dispense with the persistence of excitation condition traditionally imposed on the regressors of the critic and the actor. Given that our continuous-time momentum-based dynamics also incorporate periodic discrete-time resets that emulate restarting techniques used in the machine learning literature, we leverage tools from hybrid dynamical systems theory to establish asymptotic stability properties for the closed-loop system. We illustrate our results with a numerical example. △ Less

Submitted 27 April, 2022; originally announced April 2022.

High-Performance Optimal Incentive-Seeking in Transactive Control for Traffic Congestion

Authors: Daniel E. Ochoa, Jorge I. Poveda

Abstract: Traffic congestion has dire economic and social impacts in modern metropolitan areas. To address this problem, in this paper we introduce a novel type of model-free transactive controllers to manage vehicle traffic in highway networks for which precise mathematical models are not available. Specifically, we consider a highway system with managed lanes on which dynamic tolling mechanisms can be imp… ▽ More Traffic congestion has dire economic and social impacts in modern metropolitan areas. To address this problem, in this paper we introduce a novel type of model-free transactive controllers to manage vehicle traffic in highway networks for which precise mathematical models are not available. Specifically, we consider a highway system with managed lanes on which dynamic tolling mechanisms can be implemented in real-time using measurements from the roads. We present three incentive-seeking feedback controllers able to find in real-time the optimal economic incentives (e.g., tolls) that persuade highway users to follow a suitable driving behavior that minimizes a predefined performance index. The controllers are agnostic with respect to the exact model of the highway, and they are also able to guarantee fast convergence to the optimal tolls by leveraging non-smooth and hybrid dynamic mechanisms that combine continuous-time dynamics and discrete-time dynamics. We provide numerical examples to illustrate the advantages of the different presented techniques. △ Less

Submitted 20 April, 2022; originally announced April 2022.

Momentum-Based Nash Set Seeking over Networks via Multi-Time Scale Hybrid Dynamic Inclusions

Authors: Daniel E. Ochoa, Jorge I. Poveda

Abstract: Multi-time scale techniques, such as singular perturbations and averaging theory, have played an essential role in the development of distributed Nash equilibrium-seeking algorithms for network systems. Such techniques intrinsically rely on the uniform asymptotic stability properties of the dynamics that evolve in each time scale of the closed-loop system. When such properties are absent, synthesi… ▽ More Multi-time scale techniques, such as singular perturbations and averaging theory, have played an essential role in the development of distributed Nash equilibrium-seeking algorithms for network systems. Such techniques intrinsically rely on the uniform asymptotic stability properties of the dynamics that evolve in each time scale of the closed-loop system. When such properties are absent, synthesizing multi-time scale Nash equilibrium-seeking algorithms is more challenging and requires additional regularization mechanisms. In this paper, we investigate the synthesis and analysis of these mechanisms in the context of accelerated pseudogradient flows with time-varying dam** in non-cooperative games. Specifically, we introduce a new class of distributed and hybrid Nash set-seeking (NSS) algorithms that synergistically combine dynamic momentum-based flows with coordinated discrete-time resets. The reset mechanisms can be seen as restarting techniques that allow individual players to choose their own momentum restarting policy to achieve better transient performance. The resulting closed-loop system is modeled as a hybrid dynamic inclusion, which is analyzed using tools from hybrid dynamical system's theory. Our algorithms are developed for potential games, as well as for monotone games for which a potential function does not exist. They can be implemented in games where players have access to gradient Oracles with full or partial information, as well as in games where players have access only to measurements of their costs. In the latter case, we use tools from hybrid extremum seeking control to achieve accelerated model-free Nash set seeking. △ Less

Submitted 6 December, 2022; v1 submitted 14 October, 2021; originally announced October 2021.