Showing 1–50 of 70 results for author: Dall'Anese, E

Online convex optimization for robust control of constrained dynamical systems

Authors: Marko Nonhoff, Emiliano Dall'Anese, Matthias A. Müller

Abstract: This article investigates the problem of controlling linear time-invariant systems subject to time-varying and a priori unknown cost functions, state and input constraints, and exogenous disturbances. We combine the online convex optimization framework with tools from robust model predictive control to propose an algorithm that is able to guarantee robust constraint satisfaction. The performance o… ▽ More This article investigates the problem of controlling linear time-invariant systems subject to time-varying and a priori unknown cost functions, state and input constraints, and exogenous disturbances. We combine the online convex optimization framework with tools from robust model predictive control to propose an algorithm that is able to guarantee robust constraint satisfaction. The performance of the closed loop emerging from application of our framework is studied in terms of its dynamic regret, which is proven to be bounded linearly by the variation of the cost functions and the magnitude of the disturbances. We corroborate our theoretical findings and illustrate implementational aspects of the proposed algorithm by a numerical case study of a tracking control problem of an autonomous vehicle. △ Less

Submitted 9 January, 2024; originally announced January 2024.

Comments: 16 pages

Solving Decision-Dependent Games by Learning from Feedback

Authors: Killian Wood, Ahmed Zamzam, Emiliano Dall'Anese

Abstract: This paper tackles the problem of solving stochastic optimization problems with a decision-dependent distribution in the setting of stochastic strongly-monotone games and when the distributional dependence is unknown. A two-stage approach is proposed, which initially involves estimating the distributional dependence on decision variables, and subsequently optimizing over the estimated distribution… ▽ More This paper tackles the problem of solving stochastic optimization problems with a decision-dependent distribution in the setting of stochastic strongly-monotone games and when the distributional dependence is unknown. A two-stage approach is proposed, which initially involves estimating the distributional dependence on decision variables, and subsequently optimizing over the estimated distributional map. The paper presents guarantees for the approximation of the cost of each agent. Furthermore, a stochastic gradient-based algorithm is developed and analyzed for finding the Nash equilibrium in a distributed fashion. Numerical simulations are provided for a novel electric vehicle charging market formulation using real-world data. △ Less

Submitted 19 April, 2024; v1 submitted 28 December, 2023; originally announced December 2023.

Comments: Paper submitted to the IEEE Open Journal of Control Systems

Optimal Power Flow Pursuit via Feedback-based Safe Gradient Flow

Authors: Antonin Colot, Yiting Chen, Bertrand Cornelusse, Jorge Cortes, Emiliano Dall'Anese

Abstract: This paper considers the problem of controlling the operation of inverter-interfaced distributed energy resources (DERs) in a distribution grid, to achieve operational and performance goals with limited system-level information. We develop an online feedback optimization method to drive the DERs' power setpoints to solutions of an AC optimal power flow (OPF) problem based only on voltage measureme… ▽ More This paper considers the problem of controlling the operation of inverter-interfaced distributed energy resources (DERs) in a distribution grid, to achieve operational and performance goals with limited system-level information. We develop an online feedback optimization method to drive the DERs' power setpoints to solutions of an AC optimal power flow (OPF) problem based only on voltage measurements (and without requiring measurements of the power consumption of non-controllable assets). The proposed method - grounded on the theory of control barrier functions - is based on a continuous approximation of the projected gradient flow, appropriately modified to accommodate measurements from the power network. We provide results in terms of local exponential stability, and assess the robustness to errors in the measurements and in the system Jacobian matrix. We show that the proposed method ensures anytime satisfaction of the voltage constraints when no model and measurement errors are present; if these errors are present and are small, the voltage violation is practically negligible. We also discuss extensions of the framework to virtual power plant setups. Numerical experiments on a 93-bus distribution system and with realistic load and production profiles show a superior performance in terms of voltage regulation relative to existing methods. △ Less

Submitted 2 May, 2024; v1 submitted 19 December, 2023; originally announced December 2023.

Online Regulation of Dynamical Systems to Solutions of Constrained Optimization Problems

Authors: Yiting Chen, Liliaokeawawa Cothren, Jorge Cortes, Emiliano Dall'Anese

Abstract: This paper considers the problem of regulating a dynamical system to equilibria that are defined as solutions of an input- and state-constrained optimization problem. To solve this regulation task, we design a state feedback controller based on a continuous approximation of the projected gradient flow. We first show that the equilibria of the interconnection between the plant and the proposed cont… ▽ More This paper considers the problem of regulating a dynamical system to equilibria that are defined as solutions of an input- and state-constrained optimization problem. To solve this regulation task, we design a state feedback controller based on a continuous approximation of the projected gradient flow. We first show that the equilibria of the interconnection between the plant and the proposed controller correspond to critical points of the constrained optimization problem. We then derive sufficient conditions to ensure that, for the closed-loop system, isolated locally optimal solutions of the optimization problem are locally exponentially stable and show that input constraints are satisfied at all times by identifying an appropriate forward-invariant set. △ Less

Submitted 12 December, 2023; v1 submitted 29 November, 2023; originally announced November 2023.

Comments: This paper has been accepted for publication on IEEE Control Systems Letters

Bounds for the Tracking Error and Dynamic Regret of Inexact Online Optimization Methods: A General Analysis via Sequential Semidefinite Programs

Authors: Usman Syed, Emiliano Dall'Anese, Bin Hu

Abstract: In this paper, we develop a unified framework for analyzing the tracking error and dynamic regret of inexact online optimization methods under a variety of settings. Specifically, we leverage the quadratic constraint approach from control theory to formulate sequential semidefinite programs (SDPs) whose feasible points naturally correspond to tracking error bounds of various inexact online optimiz… ▽ More In this paper, we develop a unified framework for analyzing the tracking error and dynamic regret of inexact online optimization methods under a variety of settings. Specifically, we leverage the quadratic constraint approach from control theory to formulate sequential semidefinite programs (SDPs) whose feasible points naturally correspond to tracking error bounds of various inexact online optimization methods including the inexact online gradient descent (OGD) method, the online gradient descent-ascent method, the online stochastic gradient method, and the inexact proximal online gradient method. We provide exact analytical solutions for our proposed sequential SDPs, and obtain fine-grained tracking error bounds for the online algorithms studied in this paper. We also provide a simple routine to convert the obtained tracking error bounds into dynamic regret bounds. The main novelty of our analysis is that we derive exact analytical solutions for our proposed sequential SDPs under various inexact oracle assumptions in a unified manner. △ Less

Submitted 1 March, 2023; originally announced March 2023.

Multi-Task System Identification of Similar Linear Time-Invariant Dynamical Systems

Authors: Yiting Chen, Ana M. Ospina, Fabio Pasqualetti, Emiliano Dall'Anese

Abstract: This paper presents a system identification framework -- inspired by multi-task learning -- to estimate the dynamics of a given number of linear time-invariant (LTI) systems jointly by leveraging structural similarities across the systems. In particular, we consider LTI systems that model networked systems with similar connectivity, or LTI systems with small differences in their matrices. The syst… ▽ More This paper presents a system identification framework -- inspired by multi-task learning -- to estimate the dynamics of a given number of linear time-invariant (LTI) systems jointly by leveraging structural similarities across the systems. In particular, we consider LTI systems that model networked systems with similar connectivity, or LTI systems with small differences in their matrices. The system identification task involves the minimization of the least-squares (LS) fit for individual systems, augmented with a regularization function that enforces structural similarities. The proposed method is particularly suitable for cases when the recorded trajectories for one or more LTI systems are not sufficiently rich, leading to ill-conditioning of LS methods. We analyze the performance of the proposed method when the matrices of the LTI systems feature a common sparsity pattern (i.e., similar connectivity), and provide simulations based on real data for the estimation of the brain dynamics. We show that the proposed method requires a significantly smaller number of fMRI scans to achieve similar error levels of the LS. △ Less

Submitted 9 September, 2023; v1 submitted 3 January, 2023; originally announced January 2023.

Comments: Paper to appear in the proceedings of 62nd IEEE Conference on Decision and Control (CDC 2023)

Online Saddle Point Tracking with Decision-Dependent Data

Authors: Killian Wood, Emiliano Dall'Anese

Abstract: In this work, we consider a time-varying stochastic saddle point problem in which the objective is revealed sequentially, and the data distribution depends on the decision variables. Problems of this type express the distributional dependence via a distributional map, and are known to have two distinct types of solutions--saddle points and equilibrium points. We demonstrate that, under suitable co… ▽ More In this work, we consider a time-varying stochastic saddle point problem in which the objective is revealed sequentially, and the data distribution depends on the decision variables. Problems of this type express the distributional dependence via a distributional map, and are known to have two distinct types of solutions--saddle points and equilibrium points. We demonstrate that, under suitable conditions, online primal-dual type algorithms are capable of tracking equilibrium points. In contrast, since computing closed-form gradient of the objective requires knowledge of the distributional map, we offer an online stochastic primal-dual algorithm for tracking equilibrium trajectories. We provide bounds in expectation and in high probability, with the latter leveraging a sub-Weibull model for the gradient error. We illustrate our results on an electric vehicle charging problem where responsiveness to prices follows a location-scale family based distributional map. △ Less

Submitted 5 December, 2022; originally announced December 2022.

Comments: Submitted to the Learning for Dynamics and Control Conference (L4DC)

Perception-Based Sampled-Data Optimization of Dynamical Systems

Authors: Liliaokeawawa Cothren, Gianluca Bianchin, Sarah Dean, Emiliano Dall'Anese

Abstract: Motivated by perception-based control problems in autonomous systems, this paper addresses the problem of develo** feedback controllers to regulate the inputs and the states of a dynamical system to optimal solutions of an optimization problem when one has no access to exact measurements of the system states. In particular, we consider the case where the states need to be estimated from high-dim… ▽ More Motivated by perception-based control problems in autonomous systems, this paper addresses the problem of develo** feedback controllers to regulate the inputs and the states of a dynamical system to optimal solutions of an optimization problem when one has no access to exact measurements of the system states. In particular, we consider the case where the states need to be estimated from high-dimensional sensory data received only at discrete time intervals. We develop a sampled-data feedback controller that is based on adaptations of a projected gradient descent method, and that includes neural networks as integral components to estimate the state of the system from perceptual information. We derive sufficient conditions to guarantee (local) input-to-state stability of the control loop. Moreover, we show that the interconnected system tracks the solution trajectory of the underlying optimization problem up to an error that depends on the approximation errors of the neural network and on the time-variability of the optimization problem; the latter originates from time-varying safety and performance objectives, input constraints, and unknown disturbances. As a representative application, we illustrate our results with numerical simulations for vision-based autonomous driving. △ Less

Submitted 13 October, 2023; v1 submitted 17 November, 2022; originally announced November 2022.

Comments: This is an extended version of the paper accepted to IFAC World Congress 2023 for publication, containing proofs, and recently updated to address a typo in Assumption 3

k-Dimensional Agreement in Multiagent Systems for Distributed Coordination

Authors: Gianluca Bianchin, Miguel Vaquero, Jorge Cortes, Emiliano Dall'Anese

Abstract: Given a network of agents, each characterized by an initial scalar value, we study the problem of designing distributed linear algorithms such that the agents agree in a generalized sense on a vector quantity that belongs to a k-dimensional subspace. This problem is motivated by applications in distributed computing and sensing, where agents seek to simultaneously evaluate k independent functions… ▽ More Given a network of agents, each characterized by an initial scalar value, we study the problem of designing distributed linear algorithms such that the agents agree in a generalized sense on a vector quantity that belongs to a k-dimensional subspace. This problem is motivated by applications in distributed computing and sensing, where agents seek to simultaneously evaluate k independent functions at a common vector point by running a single distributed algorithm. We show that linear protocols can agree only on quantities that are oblique projections of the vector of initial conditions, and we provide an algebraic characterization of all agreement protocols that are consistent with a certain communication graph. By leveraging this characterization, we propose a design procedure for constructing agreement protocols, and we investigate what are the structural properties of communication networks that can reach an agreement on arbitrary weights. More broadly, our results show that agreement algorithms are capable of simultaneously solving k consensus problems at a fraction of the communication volume and space complexity of classical algorithms but, in general, require higher network connectivity. The applicability of the framework is illustrated via simulations on two problems in robotic formation and distributed regression. △ Less

Submitted 3 July, 2023; v1 submitted 18 August, 2022; originally announced August 2022.

Online Optimization of Dynamical Systems with Deep Learning Perception

Authors: Liliaokeawawa Cothren, Gianluca Bianchin, Emiliano Dall'Anese

Abstract: This paper considers the problem of controlling a dynamical system when the state cannot be directly measured and the control performance metrics are unknown or partially known. In particular, we focus on the design of data-driven controllers to regulate a dynamical system to the solution of a constrained convex optimization problem where: i) the state must be estimated from nonlinear and possibly… ▽ More This paper considers the problem of controlling a dynamical system when the state cannot be directly measured and the control performance metrics are unknown or partially known. In particular, we focus on the design of data-driven controllers to regulate a dynamical system to the solution of a constrained convex optimization problem where: i) the state must be estimated from nonlinear and possibly high-dimensional data; and, ii) the cost of the optimization problem -- which models control objectives associated with inputs and states of the system -- is not available and must be learned from data. We propose a data-driven feedback controller that is based on adaptations of a projected gradient-flow method; the controller includes neural networks as integral components for the estimation of the unknown functions. Leveraging stability theory for perturbed systems, we derive sufficient conditions to guarantee exponential input-to-state stability (ISS) of the control loop. In particular, we show that the interconnected system is ISS with respect to the approximation errors of the neural network and unknown disturbances affecting the system. The transient bounds combine the universal approximation property of deep neural networks with the ISS characterization. Illustrative numerical results are presented in the context of control of robotics and epidemics. △ Less

Submitted 30 August, 2023; v1 submitted 19 May, 2022; originally announced May 2022.

Comments: This is an extended version of the paper published in the IEEE Open Journal of Control Systems - Special Section on Machine Learning with Control, containing proofs with a correction in Section II.A

Online Weak-form Sparse Identification of Partial Differential Equations

Authors: Daniel A. Messenger, Emiliano Dall'Anese, David M. Bortz

Abstract: This paper presents an online algorithm for identification of partial differential equations (PDEs) based on the weak-form sparse identification of nonlinear dynamics algorithm (WSINDy). The algorithm is online in a sense that if performs the identification task by processing solution snapshots that arrive sequentially. The core of the method combines a weak-form discretization of candidate PDEs w… ▽ More This paper presents an online algorithm for identification of partial differential equations (PDEs) based on the weak-form sparse identification of nonlinear dynamics algorithm (WSINDy). The algorithm is online in a sense that if performs the identification task by processing solution snapshots that arrive sequentially. The core of the method combines a weak-form discretization of candidate PDEs with an online proximal gradient descent approach to the sparse regression problem. In particular, we do not regularize the $\ell_0$-pseudo-norm, instead finding that directly applying its proximal operator (which corresponds to a hard thresholding) leads to efficient online system identification from noisy data. We demonstrate the success of the method on the Kuramoto-Sivashinsky equation, the nonlinear wave equation with time-varying wavespeed, and the linear wave equation, in one, two, and three spatial dimensions, respectively. In particular, our examples show that the method is capable of identifying and tracking systems with coefficients that vary abruptly in time, and offers a streaming alternative to problems in higher dimensions. △ Less

Submitted 8 March, 2022; originally announced March 2022.

Comments: 21 pages, 4 figures

MSC Class: 62J07; 68T05; 93-10

Stochastic Saddle Point Problems with Decision-Dependent Distributions

Authors: Killian Wood, Emiliano Dall'Anese

Abstract: This paper focuses on stochastic saddle point problems with decision-dependent distributions. These are problems whose objective is the expected value of a stochastic payoff function and whose data distribution drifts in response to decision variables--a phenomenon represented by a distributional map. A common approach to accommodating distributional shift is to retrain optimal decisions once a ne… ▽ More This paper focuses on stochastic saddle point problems with decision-dependent distributions. These are problems whose objective is the expected value of a stochastic payoff function and whose data distribution drifts in response to decision variables--a phenomenon represented by a distributional map. A common approach to accommodating distributional shift is to retrain optimal decisions once a new distribution is revealed, or repeated retraining. We introduce the notion of equilibrium points, which are the fixed points of this repeated retraining procedure, and provide sufficient conditions for their existence and uniqueness. To find equilibrium points, we develop deterministic and stochastic primal-dual algorithms and demonstrate their convergence with constant step-size in the former and polynomial decay step-size schedule in the latter. By modeling errors emerging from a stochastic gradient estimator as sub-Weibull random variables, we provide error bounds in expectation and in high probability that hold for each iteration. Without additional knowledge of the distributional map, computing saddle points is intractable. Thus we propose a condition on the distributional map--which we call opposing mixture dominance--that ensures that the objective is strongly-convex-strongly-concave. Finally, we demonstrate that derivative-free algorithms with a single function evaluation are capable of approximating saddle points △ Less

Submitted 11 November, 2022; v1 submitted 6 January, 2022; originally announced January 2022.

Comments: Completed first revision with SIOPT. A link to the codebase is provided in the document

Data-Enabled Gradient Flow as Feedback Controller: Regulation of Linear Dynamical Systems to Minimizers of Unknown Functions

Authors: Liliaokeawawa Cothren, Gianluca Bianchin, Emiliano Dall'Anese

Abstract: This paper considers the problem of regulating a linear dynamical system to the solution of a convex optimization problem with an unknown or partially-known cost. We design a data-driven feedback controller - based on gradient flow dynamics - that (i) is augmented with learning methods to estimate the cost function based on infrequent (and possibly noisy) functional evaluations; and, concurrently,… ▽ More This paper considers the problem of regulating a linear dynamical system to the solution of a convex optimization problem with an unknown or partially-known cost. We design a data-driven feedback controller - based on gradient flow dynamics - that (i) is augmented with learning methods to estimate the cost function based on infrequent (and possibly noisy) functional evaluations; and, concurrently, (ii) is designed to drive the inputs and outputs of the dynamical system to the optimizer of the problem. We derive sufficient conditions on the learning error and the controller gain to ensure that the error between the optimizer of the problem and the state of the closed-loop system is ultimately bounded; the error bound accounts for the functional estimation errors and the temporal variability of the unknown disturbance affecting the linear dynamical system. Our results directly lead to exponential input-to-state stability of the closed-loop system. The proposed method and the theoretical bounds are validated numerically. △ Less

Submitted 2 April, 2022; v1 submitted 2 December, 2021; originally announced December 2021.

Comments: This is an extended version of the paper accepted to the 4th Annual Learning for Dynamics & Control Conference (L4DC 2022), containing proofs

Feedback-Based Optimization with Sub-Weibull Gradient Errors and Intermittent Updates

Authors: Ana M. Ospina, Nicola Bastianello, Emiliano Dall'Anese

Abstract: This paper considers a feedback-based projected gradient method for optimizing systems modeled as algebraic maps. The focus is on a setup where the gradient is corrupted by random errors that follow a sub-Weibull distribution, and where the measurements of the output -- which replace the input-output map of the system in the algorithmic updates -- may not be available at each iteration. The sub-We… ▽ More This paper considers a feedback-based projected gradient method for optimizing systems modeled as algebraic maps. The focus is on a setup where the gradient is corrupted by random errors that follow a sub-Weibull distribution, and where the measurements of the output -- which replace the input-output map of the system in the algorithmic updates -- may not be available at each iteration. The sub-Weibull error model is particularly well-suited in frameworks where the cost of the problem is learned via Gaussian Process (GP) regression (from functional evaluations) concurrently with the execution of the algorithm; however, it also naturally models setups where nonparametric methods and neural networks are utilized to estimate the cost. Using the sub-Weibull model, and with Bernoulli random variables modeling missing measurements of the system output, we show that the online algorithm generates points that are within a bounded error from the optimal solutions. In particular, we provide error bounds in expectation and in high probability. Numerical results are presented in the context of a demand response problem in smart power grids. △ Less

Submitted 30 June, 2023; v1 submitted 13 September, 2021; originally announced September 2021.

Comments: Version of the IEEE Control Systems Letters (L-CSS) paper with one typo corrected

Planning a Return to Normal after the COVID-19 Pandemic: Identifying Safe Contact Levels via Online Optimization

Authors: Gianluca Bianchin, Emiliano Dall'Anese, Jorge I. Poveda, David Jacobson, Elizabeth J. Carlton, Andrea G. Buchwald

Abstract: Since the early months of 2020, non-pharmaceutical interventions (NPIs) -- implemented at varying levels of severity and based on widely-divergent perspectives of risk tolerance -- have been the primary means to control SARS-CoV-2 transmission. We seek to identify how risk tolerance and vaccination rates impact the rate at which a population can return to pre-pandemic contact behavior. To this end… ▽ More Since the early months of 2020, non-pharmaceutical interventions (NPIs) -- implemented at varying levels of severity and based on widely-divergent perspectives of risk tolerance -- have been the primary means to control SARS-CoV-2 transmission. We seek to identify how risk tolerance and vaccination rates impact the rate at which a population can return to pre-pandemic contact behavior. To this end, we develop a novel feedback control method for data-driven decision-making to identify optimal levels of NPIs across geographical regions in order to guarantee that hospitalizations will not exceed a given risk tolerance. Results are shown for the state of Colorado, and they suggest that: coordination in decision-making across regions is essential to maintain the daily number of hospitalizations below the desired limits; increasing risk tolerance can decrease the number of days required to discontinue NPIs, at the cost of an increased number of deaths; and if vaccination uptake is less than 70\%, at most levels of risk tolerance, return to pre-pandemic contact behaviors before the early months of 2022 may newly jeopardize the healthcare system. △ Less

Submitted 13 September, 2021; originally announced September 2021.

Comments: A preliminary version of this work appeared on medrXiv on April 1, 2021. Available at: https://www.medrxiv.org/content/10.1101/2021.04.20.21255350v1

Online Stochastic Optimization for Unknown Linear Systems: Data-Driven Synthesis and Controller Analysis

Authors: Gianluca Bianchin, Miguel Vaquero, Jorge Cortes, Emiliano Dall'Anese

Abstract: This paper proposes a data-driven control framework to regulate an unknown, stochastic linear dynamical system to the solution of a (stochastic) convex optimization problem. Despite the centrality of this problem, most of the available methods critically rely on a precise knowledge of the system dynamics (thus requiring off-line system identification and model refinement). To this aim, in this pap… ▽ More This paper proposes a data-driven control framework to regulate an unknown, stochastic linear dynamical system to the solution of a (stochastic) convex optimization problem. Despite the centrality of this problem, most of the available methods critically rely on a precise knowledge of the system dynamics (thus requiring off-line system identification and model refinement). To this aim, in this paper we first show that the steady-state transfer function of a linear system can be computed directly from control experiments, bypassing explicit model identification. Then, we leverage the estimated transfer function to design a controller -- which is inspired by stochastic gradient descent methods -- that regulates the system to the solution of the prescribed optimization problem. A distinguishing feature of our methods is that they do not require any knowledge of the system dynamics, disturbance terms, or their distributions. Our technical analysis combines concepts and tools from behavioral system theory, stochastic optimization with decision-dependent distributions, and stability analysis. We illustrate the applicability of the framework on a case study for mobility-on-demand ride service scheduling in Manhattan, NY. △ Less

Submitted 30 August, 2021; originally announced August 2021.

Online Stochastic Gradient Methods Under Sub-Weibull Noise and the Polyak-Łojasiewicz Condition

Authors: Seunghyun Kim, Liam Madden, Emiliano Dall'Anese

Abstract: This paper focuses on the online gradient and proximal-gradient methods with stochastic gradient errors. In particular, we examine the performance of the online gradient descent method when the cost satisfies the Polyak-Łojasiewicz (PL) inequality. We provide bounds in expectation and in high probability (that hold iteration-wise), with the latter derived by leveraging a sub-Weibull model for the… ▽ More This paper focuses on the online gradient and proximal-gradient methods with stochastic gradient errors. In particular, we examine the performance of the online gradient descent method when the cost satisfies the Polyak-Łojasiewicz (PL) inequality. We provide bounds in expectation and in high probability (that hold iteration-wise), with the latter derived by leveraging a sub-Weibull model for the errors affecting the gradient. The convergence results show that the instantaneous regret converges linearly up to an error that depends on the variability of the problem and the statistics of the sub-Weibull gradient error. Similar convergence results are then provided for the online proximal-gradient method, under the assumption that the composite cost satisfies the proximal-PL condition. In the case of static costs, we provide new bounds for the regret incurred by these methods when the gradient errors are modeled as sub-Weibull random variables. Illustrative simulations are provided to corroborate the technical findings. △ Less

Submitted 31 March, 2022; v1 submitted 6 August, 2021; originally announced August 2021.

Online Projected Gradient Descent for Stochastic Optimization with Decision-Dependent Distributions

Authors: Killian Wood, Gianluca Bianchin, Emiliano Dall'Anese

Abstract: This paper investigates the problem of tracking solutions of stochastic optimization problems with time-varying costs that depend on random variables with decision-dependent distributions. In this context, we propose the use of an online stochastic gradient descent method to solve the optimization, and we provide explicit bounds in expectation and in high probability for the distance between the o… ▽ More This paper investigates the problem of tracking solutions of stochastic optimization problems with time-varying costs that depend on random variables with decision-dependent distributions. In this context, we propose the use of an online stochastic gradient descent method to solve the optimization, and we provide explicit bounds in expectation and in high probability for the distance between the optimizers and the points generated by the algorithm. In particular, we show that when the gradient error due to sampling is modeled as a sub-Weibull random variable, then the tracking error is ultimately bounded in expectation and in high probability. The theoretical findings are validated via numerical simulations in the context of charging optimization of a fleet of electric vehicles. △ Less

Submitted 27 October, 2021; v1 submitted 20 July, 2021; originally announced July 2021.

Comments: To appear in IEEE L-CSS

OpReg-Boost: Learning to Accelerate Online Algorithms with Operator Regression

Authors: Nicola Bastianello, Andrea Simonetto, Emiliano Dall'Anese

Abstract: This paper presents a new regularization approach -- termed OpReg-Boost -- to boost the convergence and lessen the asymptotic error of online optimization and learning algorithms. In particular, the paper considers online algorithms for optimization problems with a time-varying (weakly) convex composite cost. For a given online algorithm, OpReg-Boost learns the closest algorithmic map that yields… ▽ More This paper presents a new regularization approach -- termed OpReg-Boost -- to boost the convergence and lessen the asymptotic error of online optimization and learning algorithms. In particular, the paper considers online algorithms for optimization problems with a time-varying (weakly) convex composite cost. For a given online algorithm, OpReg-Boost learns the closest algorithmic map that yields linear convergence; to this end, the learning procedure hinges on the concept of operator regression. We show how to formalize the operator regression problem and propose a computationally-efficient Peaceman-Rachford solver that exploits a closed-form solution of simple quadratically-constrained quadratic programs (QCQPs). Simulation results showcase the superior properties of OpReg-Boost w.r.t. the more classical forward-backward algorithm, FISTA, and Anderson acceleration. △ Less

Submitted 4 April, 2022; v1 submitted 27 May, 2021; originally announced May 2021.

Comments: Code available here https://github.com/nicola-bastianello/reg4opt To be presented at the 4th Annual Learning for Dynamics & Control Conference (L4DC'22)

A Stochastic Operator Framework for Optimization and Learning with Sub-Weibull Errors

Authors: Nicola Bastianello, Liam Madden, Ruggero Carli, Emiliano Dall'Anese

Abstract: This paper proposes a framework to study the convergence of stochastic optimization and learning algorithms. The framework is modeled over the different challenges that these algorithms pose, such as (i) the presence of random additive errors (e.g. due to stochastic gradients), and (ii) random coordinate updates (e.g. due to asynchrony in distributed set-ups). The paper covers both convex and stro… ▽ More This paper proposes a framework to study the convergence of stochastic optimization and learning algorithms. The framework is modeled over the different challenges that these algorithms pose, such as (i) the presence of random additive errors (e.g. due to stochastic gradients), and (ii) random coordinate updates (e.g. due to asynchrony in distributed set-ups). The paper covers both convex and strongly convex problems, and it also analyzes online scenarios, involving changes in the data and costs. The paper relies on interpreting stochastic algorithms as the iterated application of stochastic operators, thus allowing us to use the powerful tools of operator theory. In particular, we consider operators characterized by additive errors with sub-Weibull distribution (which parameterize a broad class of errors by their tail probability), and random updates. In this framework we derive convergence results in mean and in high probability, by providing bounds to the distance of the current iteration from a solution of the optimization or learning problem. The contributions are discussed in light of federated learning applications. △ Less

Submitted 24 June, 2024; v1 submitted 20 May, 2021; originally announced May 2021.

Comments: To appear in IEEE Transactions on Automatic Control

Data-Driven Synthesis of Optimization-Based Controllers for Regulation of Unknown Linear Systems

Authors: Gianluca Bianchin, Miguel Vaquero, Jorge Cortes, Emiliano Dall'Anese

Abstract: This paper proposes a data-driven framework to solve time-varying optimization problems associated with unknown linear dynamical systems. Making online control decisions to regulate a dynamical system to the solution of an optimization problem is a central goal in many modern engineering applications. Yet, the available methods critically rely on a precise knowledge of the system dynamics, thus ma… ▽ More This paper proposes a data-driven framework to solve time-varying optimization problems associated with unknown linear dynamical systems. Making online control decisions to regulate a dynamical system to the solution of an optimization problem is a central goal in many modern engineering applications. Yet, the available methods critically rely on a precise knowledge of the system dynamics, thus mandating a preliminary system identification phase before a controller can be designed. In this work, we leverage results from behavioral theory to show that the steady-state transfer function of a linear system can be computed from data samples without any knowledge or estimation of the system model. We then use this data-driven representation to design a controller, inspired by a gradient-descent optimization method, that regulates the system to the solution of a convex optimization problem, without requiring any knowledge of the time-varying disturbances affecting the model equation. Results are tailored to cost functions satisfy the Polyak-Łojasiewicz inequality. △ Less

Submitted 7 September, 2021; v1 submitted 30 March, 2021; originally announced March 2021.

Time-Varying Optimization of Networked Systems with Human Preferences

Authors: Ana M. Ospina, Andrea Simonetto, Emiliano Dall'Anese

Abstract: This paper considers a time-varying optimization problem associated with a network of systems, with each of the systems shared by (and affecting) a number of individuals. The objective is to minimize cost functions associated with the individuals' preferences, which are unknown, subject to time-varying constraints that capture physical or operational limits of the network. To this end, the paper d… ▽ More This paper considers a time-varying optimization problem associated with a network of systems, with each of the systems shared by (and affecting) a number of individuals. The objective is to minimize cost functions associated with the individuals' preferences, which are unknown, subject to time-varying constraints that capture physical or operational limits of the network. To this end, the paper develops a distributed online optimization algorithm with concurrent learning of the cost functions. The cost functions are learned on-the-fly based on the users' feedback (provided at irregular intervals) by leveraging tools from shape-constrained Gaussian Process. The online algorithm is based on a primal-dual method, and acts effectively in a closed-loop fashion where: i) users' feedback is utilized to estimate the cost, and ii) measurements from the network are utilized in the algorithmic steps to bypass the need for sensing of (unknown) exogenous inputs of the network. The performance of the algorithm is analyzed in terms of dynamic network regret and constraint violation. Numerical examples are presented in the context of real-time optimization of distributed energy resources. △ Less

Submitted 11 March, 2022; v1 submitted 24 March, 2021; originally announced March 2021.

Online Optimization of LTI Systems Under Persistent Attacks: Stability, Tracking, and Robustness

Authors: Felipe Galarza-Jimenez, Gianluca Bianchin, Jorge I. Poveda, Emiliano Dall'Anese

Abstract: We study the stability properties of a control system composed of a dynamical plant and a feedback controller, the latter generating control signals that can be compromised by a malicious attacker. We consider two classes of feedback controllers: a static output-feedback controller, and a dynamical gradient-flow controller that seeks to steer the output of the plant towards the solution of a conve… ▽ More We study the stability properties of a control system composed of a dynamical plant and a feedback controller, the latter generating control signals that can be compromised by a malicious attacker. We consider two classes of feedback controllers: a static output-feedback controller, and a dynamical gradient-flow controller that seeks to steer the output of the plant towards the solution of a convex optimization problem. In both cases, we analyze the stability properties of the closed-loop system under a class of switching attacks that persistently modify the control inputs generated by the controllers. Our stability analysis leverages the framework of hybrid dynamical systems, Lyapunov-based arguments for switching systems with unstable modes, and singular perturbation theory. Our results reveal that, under a suitable time-scale separation between plant and controllers, the stability of the interconnected system can be preserved when the attack occurs with "sufficiently low frequency" in any bounded time interval. We present simulation results in a power-grid example that corroborate the technical findings. △ Less

Submitted 18 December, 2023; v1 submitted 18 February, 2021; originally announced February 2021.

Comments: Appeared in Nonlinear Analysis: Hybrid Systems, pp. 101152, vol. 44, may. 2022

Time-Varying Optimization of LTI Systems via Projected Primal-Dual Gradient Flows

Authors: Gianluca Bianchin, Jorge Cortes, Jorge I. Poveda, Emiliano Dall'Anese

Abstract: This paper investigates the problem of regulating in real time a linear dynamical system to the solution trajectory of a time-varying constrained convex optimization problem. The proposed feedback controller is based on an adaptation of the saddle-flow dynamics, modified to take into account projections on constraint sets and output-feedback from the plant. We derive sufficient conditions on the t… ▽ More This paper investigates the problem of regulating in real time a linear dynamical system to the solution trajectory of a time-varying constrained convex optimization problem. The proposed feedback controller is based on an adaptation of the saddle-flow dynamics, modified to take into account projections on constraint sets and output-feedback from the plant. We derive sufficient conditions on the tunable parameters of the controller (inherently related to the time-scale separation between plant and controller dynamics) to guarantee exponential and input-to-state stability of the closed-loop system. The analysis is tailored to the case of time-varying strongly convex cost functions and polytopic output constraints. The theoretical results are further validated in a ramp metering control problem in a network of traffic highways. △ Less

Submitted 10 September, 2021; v1 submitted 5 January, 2021; originally announced January 2021.

Online Optimization of Switched LTI Systems Using Continuous-Time and Hybrid Accelerated Gradient Flows

Authors: Gianluca Bianchin, Jorge I. Poveda, Emiliano Dall'Anese

Abstract: This paper studies the design of feedback controllers to steer a switching linear time-invariant dynamical system towards the solution trajectory of a time-varying convex optimization problem. We propose two types of controllers: (i) a continuous controller inspired by the online gradient descent method, and (ii) a hybrid controller that can be interpreted as an online version of Nesterov's accele… ▽ More This paper studies the design of feedback controllers to steer a switching linear time-invariant dynamical system towards the solution trajectory of a time-varying convex optimization problem. We propose two types of controllers: (i) a continuous controller inspired by the online gradient descent method, and (ii) a hybrid controller that can be interpreted as an online version of Nesterov's accelerated gradient method with restarts of the state variables. By design, the controllers continuously steer the system towards the time-varying optimizer without requiring knowledge of exogenous disturbances affecting the system. For cost functions that are smooth and satisfy the Polyak-Łojasiewicz inequality, we demonstrate that the online gradient-flow controller ensures uniform global exponential stability when the time scales of the system and controller are sufficiently separated and the switching signal of the system varies slowly on average. For cost functions that are strongly convex, we show that the hybrid accelerated controller outperforms the continuous gradient descent method. When the cost function is not strongly convex, we show that the the hybrid accelerated method guarantees global practical asymptotic stability. △ Less

Submitted 19 February, 2022; v1 submitted 10 August, 2020; originally announced August 2020.

Data-Driven and Online Estimation of Linear Sensitivity Distribution Factors: A Low-rank Approach

Authors: Ana M. Ospina, Emiliano Dall'Anese

Abstract: Estimation of sensitivity matrices in electrical transmission systems allows grid operators to evaluate in real-time how changes in power injections reflect into changes in power flows. In this paper, we propose a robust low-rank minimization approach to estimate sensitivity matrices based on measurements of power injections and power flows. An online proximal-gradient method is proposed to estima… ▽ More Estimation of sensitivity matrices in electrical transmission systems allows grid operators to evaluate in real-time how changes in power injections reflect into changes in power flows. In this paper, we propose a robust low-rank minimization approach to estimate sensitivity matrices based on measurements of power injections and power flows. An online proximal-gradient method is proposed to estimate sensitivities on-the-fly from real-time measurements. The proposed method obtains meaningful estimates with fewer measurements when the regression model is underdetermined, in contrast with existing methods based on least-squares approaches. In addition, our method can also identify faulty measurements and handle missing data. In this work, convergence results in terms of dynamic regret are presented. Numerical tests corroborate the effectiveness of the novel approach and the robustness of missing measurements and outliers. △ Less

Submitted 9 September, 2023; v1 submitted 29 June, 2020; originally announced June 2020.

Comments: Paper to appear in the proceedings of 62nd IEEE Conference on Decision and Control (CDC 2023)

Time-Varying Convex Optimization: Time-Structured Algorithms and Applications

Authors: Andrea Simonetto, Emiliano Dall'Anese, Santiago Paternain, Geert Leus, Georgios B. Giannakis

Abstract: Optimization underpins many of the challenges that science and technology face on a daily basis. Recent years have witnessed a major shift from traditional optimization paradigms grounded on batch algorithms for medium-scale problems to challenging dynamic, time-varying, and even huge-size settings. This is driven by technological transformations that converted infrastructural and social platforms… ▽ More Optimization underpins many of the challenges that science and technology face on a daily basis. Recent years have witnessed a major shift from traditional optimization paradigms grounded on batch algorithms for medium-scale problems to challenging dynamic, time-varying, and even huge-size settings. This is driven by technological transformations that converted infrastructural and social platforms into complex and dynamic networked systems with even pervasive sensing and computing capabilities. The present paper reviews a broad class of state-of-the-art algorithms for time-varying optimization, with an eye to both algorithmic development and performance analysis. It offers a comprehensive overview of available tools and methods, and unveils open challenges in application domains of broad interest. The real-world examples presented include smart power systems, robotics, machine learning, and data analytics, highlighting domain-specific issues and solutions. The ultimate goal is to exempify wide engineering relevance of analytical tools and pertinent theoretical foundations. △ Less

Submitted 15 June, 2020; originally announced June 2020.

Comments: 14 pages, 6 figures; to appear in the Proceedings of the IEEE

Journal ref: Proceedings of the IEEE, vol 108 (11), pages 2032-2048, 2020

Personalized Demand Response via Shape-Constrained Online Learning

Authors: Ana M. Ospina, Andrea Simonetto, Emiliano Dall'Anese

Abstract: This paper formalizes a demand response task as an optimization problem featuring a known time-varying engineering cost and an unknown (dis)comfort function. Based on this model, this paper develops a feedback-based projected gradient method to solve the demand response problem in an online fashion, where: i) feedback from the user is leveraged to learn the (dis)comfort function concurrently with… ▽ More This paper formalizes a demand response task as an optimization problem featuring a known time-varying engineering cost and an unknown (dis)comfort function. Based on this model, this paper develops a feedback-based projected gradient method to solve the demand response problem in an online fashion, where: i) feedback from the user is leveraged to learn the (dis)comfort function concurrently with the execution of the algorithm; and, ii) measurements of electrical quantities are used to estimate the gradient of the known engineering cost. To learn the unknown function, a shape-constrained Gaussian Process is leveraged; this approach allows one to obtain an estimated function that is strongly convex and smooth. The performance of the online algorithm is analyzed by using metrics such as the tracking error and the dynamic regret. A numerical example is illustrated to corroborate the technical findings. △ Less

Submitted 11 June, 2020; originally announced June 2020.

Comments: Submitted to IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm) 2020

High-probability Convergence Bounds for Non-convex Stochastic Gradient Descent

Authors: Liam Madden, Emiliano Dall'Anese, Stephen Becker

Abstract: Stochastic gradient descent is one of the most common iterative algorithms used in machine learning. While being computationally cheap to implement, recent literature suggests it may have implicit regularization properties that prevent over-fitting. This paper analyzes the properties of stochastic gradient descent from a theoretical standpoint to help bridge the gap between theoretical and empiric… ▽ More Stochastic gradient descent is one of the most common iterative algorithms used in machine learning. While being computationally cheap to implement, recent literature suggests it may have implicit regularization properties that prevent over-fitting. This paper analyzes the properties of stochastic gradient descent from a theoretical standpoint to help bridge the gap between theoretical and empirical results. Most theoretical results either assume convexity or only provide convergence results in mean, while this paper proves convergence bounds in high probability without assuming convexity. Assuming strong smoothness, we prove high probability convergence bounds in two settings: (1) assuming the Polyak-Łojasiewicz inequality and norm sub-Gaussian gradient noise and (2) assuming norm sub-Weibull gradient noise. In the first setting, we combine our convergence bounds with existing generalization bounds in order to bound the true risk and show that for a certain number of epochs, convergence and generalization balance in such a way that the true risk goes to the empirical minimum as the number of samples goes to infinity. In the second setting, as an intermediate step to proving convergence, we prove a probability result of independent interest. The probability result extends Freedman-type concentration beyond the sub-exponential threshold to heavier-tailed martingale difference sequences. △ Less

Submitted 15 November, 2021; v1 submitted 9 June, 2020; originally announced June 2020.

Comments: V4: significant additions. Proved a convergence bound in the non-convex setting with norm sub-Weibull noise and proved a Freedman-type inequality for sub-Weibull martingale difference sequences. Also included numerics for both the PL and non-convex settings

Online State Estimation for Time-Varying Systems

Authors: Guido Cavraro, Emiliano Dall'Anese, Joshua Comden, Andrey Bernstein

Abstract: The paper investigates the problem of estimating the state of a time-varying system with a linear measurement model; in particular, the paper considers the case where the number of measurements available can be smaller than the number of states. In lieu of a batch linear least-squares (LS) approach -- well suited for static networks, where a sufficient number of measurements could be collected to… ▽ More The paper investigates the problem of estimating the state of a time-varying system with a linear measurement model; in particular, the paper considers the case where the number of measurements available can be smaller than the number of states. In lieu of a batch linear least-squares (LS) approach -- well suited for static networks, where a sufficient number of measurements could be collected to obtain a full-rank design matrix -- the paper proposes an online algorithm to estimate the possibly time-varying state by processing measurements as and when available. The design of the algorithm hinges on a generalized LS cost augmented with a proximal-point-type regularization. With the solution of the regularized LS problem available in closed-form, the online algorithm is written as a linear dynamical system where the state is updated based on the previous estimate and based on the new available measurements. Conditions under which the algorithmic steps are in fact a contractive map** are shown, and bounds on the estimation error are derived for different noise models. Numerical simulations are provided to corroborate the analytical findings. △ Less

Submitted 5 April, 2021; v1 submitted 31 May, 2020; originally announced June 2020.

Online Proximal-ADMM For Time-varying Constrained Convex Optimization

Authors: Yijian Zhang, Emiliano Dall'Anese, Mingyi Hong

Abstract: This paper considers a convex optimization problem with cost and constraints that evolve over time. The function to be minimized is strongly convex and possibly non-differentiable, and variables are coupled through linear constraints. In this setting, the paper proposes an online algorithm based on the alternating direction method of multipliers (ADMM), to track the optimal solution trajectory of… ▽ More This paper considers a convex optimization problem with cost and constraints that evolve over time. The function to be minimized is strongly convex and possibly non-differentiable, and variables are coupled through linear constraints. In this setting, the paper proposes an online algorithm based on the alternating direction method of multipliers (ADMM), to track the optimal solution trajectory of the time-varying problem; in particular, the proposed algorithm consists of a primal proximal gradient descent step and an appropriately perturbed dual ascent step. The paper derives tracking results, asymptotic bounds, and linear convergence results. The proposed algorithm is then specialized to a multi-area power grid optimization problem, and our numerical results verify the desired properties. △ Less

Submitted 12 January, 2021; v1 submitted 7 May, 2020; originally announced May 2020.

Bounds for the tracking error of first-order online optimization methods

Authors: Liam Madden, Stephen Becker, Emiliano Dall'Anese

Abstract: This paper investigates online algorithms for smooth time-varying optimization problems, focusing first on methods with constant step-size, momentum, and extrapolation-length. Assuming strong convexity, precise results for the tracking iterate error (the limit supremum of the norm of the difference between the optimal solution and the iterates) for online gradient descent are derived. The paper th… ▽ More This paper investigates online algorithms for smooth time-varying optimization problems, focusing first on methods with constant step-size, momentum, and extrapolation-length. Assuming strong convexity, precise results for the tracking iterate error (the limit supremum of the norm of the difference between the optimal solution and the iterates) for online gradient descent are derived. The paper then considers a general first-order framework, where a universal lower bound on the tracking iterate error is established. Furthermore, a method using "long-steps" is proposed and shown to achieve the lower bound up to a fixed constant. This method is then compared with online gradient descent for specific examples. Finally, the paper analyzes the effect of regularization when the cost is not strongly convex. With regularization, it is possible to achieve a non-regret bound. The paper ends by testing the accelerated and regularized methods on synthetic time-varying least-squares and logistic regression problems, respectively. △ Less

Submitted 6 October, 2020; v1 submitted 4 March, 2020; originally announced March 2020.

Comments: Main changes: added discussion on regret literature in section 2 and added discussion on lower bounds in section 4

Journal ref: J Optim Theory Appl 189, 437-457 (2021)

Distributed and Inexact Proximal Gradient Method for Online Convex Optimization

Authors: Nicola Bastianello, Emiliano Dall'Anese

Abstract: This paper develops and analyzes an online distributed proximal-gradient method (DPGM) for time-varying composite convex optimization problems. Each node of the network features a local cost that includes a smooth strongly convex function and a non-smooth convex function, both changing over time. By coordinating through a connected communication network, the nodes collaboratively track the traject… ▽ More This paper develops and analyzes an online distributed proximal-gradient method (DPGM) for time-varying composite convex optimization problems. Each node of the network features a local cost that includes a smooth strongly convex function and a non-smooth convex function, both changing over time. By coordinating through a connected communication network, the nodes collaboratively track the trajectory of the minimizers without exchanging their local cost functions. The DPGM is implemented in an online fashion, that is, in a setting where only a limited number of steps are implemented before the function changes. Moreover, the algorithm is analyzed in an inexact scenario, that is, with a source of additive noise, that can represent e.g. communication noise or quantization. It is shown that the tracking error of the online inexact DPGM is upper-bounded by a convergent linear system, guaranteeing convergence within a neighborhood of the optimal solution. △ Less

Submitted 1 April, 2021; v1 submitted 3 January, 2020; originally announced January 2020.

Comments: To be presented at the European Control Conference 2021 (ECC'21)

Optimization and Learning with Information Streams: Time-varying Algorithms and Applications

Authors: Emiliano Dall'Anese, Andrea Simonetto, Stephen Becker, Liam Madden

Abstract: There is a growing cross-disciplinary effort in the broad domain of optimization and learning with streams of data, applied to settings where traditional batch optimization techniques cannot produce solutions at time scales that match the inter-arrival times of the data points due to computational and/or communication bottlenecks. Special types of online algorithms can handle this situation, and t… ▽ More There is a growing cross-disciplinary effort in the broad domain of optimization and learning with streams of data, applied to settings where traditional batch optimization techniques cannot produce solutions at time scales that match the inter-arrival times of the data points due to computational and/or communication bottlenecks. Special types of online algorithms can handle this situation, and this article focuses on such time-varying optimization algorithms, with emphasis on Machine Leaning and Signal Processing, as well as data-driven Control. Approaches for the design of time-varying or online first-order optimization methods are discussed, with emphasis on algorithms that can handle errors in the gradient, as may arise when the gradient is estimated. Insights on performance metrics and accompanying claims are provided, along with evidence of cases where algorithms that are provably convergent in batch optimization may perform poorly in an online regime. The role of distributed computation is discussed. Illustrative numerical examples for a number of applications of broad interest are provided to convey key ideas. △ Less

Submitted 16 January, 2020; v1 submitted 17 October, 2019; originally announced October 2019.

Comments: Accepted for publication in IEEE Signal Processing Magazine. Limit of 40 references

Journal ref: IEEE Signal Processing Magazine, May 2020

Inexact Online Proximal-gradient Method for Time-varying Convex Optimization

Authors: Amirhossein Ajalloeian, Andrea Simonetto, Emiliano Dall'Anese

Abstract: This paper considers an online proximal-gradient method to track the minimizers of a composite convex function that may continuously evolve over time. The online proximal-gradient method is inexact, in the sense that: (i) it relies on an approximate first-order information of the smooth component of the cost; and, (ii) the proximal operator (with respect to the non-smooth term) may be computed onl… ▽ More This paper considers an online proximal-gradient method to track the minimizers of a composite convex function that may continuously evolve over time. The online proximal-gradient method is inexact, in the sense that: (i) it relies on an approximate first-order information of the smooth component of the cost; and, (ii) the proximal operator (with respect to the non-smooth term) may be computed only up to a certain precision. Under suitable assumptions, convergence of the error iterates is established for strongly convex cost functions. On the other hand, the dynamic regret is investigated when the cost is not strongly convex, under the additional assumption that the problem includes feasibility sets that are compact. Bounds are expressed in terms of the cumulative error and the path length of the optimal solutions. This suggests how to allocate resources to strike a balance between performance and precision in the gradient computation and in the proximal operator. △ Less

Submitted 22 April, 2020; v1 submitted 4 October, 2019; originally announced October 2019.

Comments: In Proceedings of ACC-2020

Personalized Optimization with User's Feedback

Authors: Andrea Simonetto, Emiliano Dall'Anese, Julien Monteil, Andrey Bernstein

Abstract: This paper develops an online algorithm to solve a time-varying optimization problem with an objective that comprises a known time-varying cost and an unknown function. This problem structure arises in a number of engineering systems and cyber-physical systems where the known function captures time-varying engineering costs, and the unknown function models user's satisfaction; in this context, the… ▽ More This paper develops an online algorithm to solve a time-varying optimization problem with an objective that comprises a known time-varying cost and an unknown function. This problem structure arises in a number of engineering systems and cyber-physical systems where the known function captures time-varying engineering costs, and the unknown function models user's satisfaction; in this context, the objective is to strike a balance between given performance metrics and user's satisfaction. Key challenges related to the problem at hand are related to (1) the time variability of the problem, and (2) the fact that learning of the user's utility function is performed concurrently with the execution of the online algorithm. This paper leverages Gaussian processes (GP) to learn the unknown cost function from noisy functional evaluation and build pertinent upper confidence bounds. Using the GP formalism, the paper then advocates time-varying optimization tools to design an online algorithm that exhibits tracking of the oracle-based optimal trajectory within an error ball, while learning the user's satisfaction function with no-regret. The algorithmic steps are inexact, to account for possible limited computational budgets or real-time implementation considerations. Numerical examples are illustrated based on a problem related to vehicle platooning. △ Less

Submitted 5 August, 2020; v1 submitted 2 May, 2019; originally announced May 2019.

Comments: 17 pages, 7 figures -- v3 with additional simulation results and comparisons

Journal ref: Automatica, Volume 131, 2021

On the Convergence of the Inexact Running Krasnosel'skii-Mann Method

Authors: Emiliano Dall'Anese, Andrea Simonetto, Andrey Bernstein

Abstract: This paper leverages a framework based on averaged operators to tackle the problem of tracking fixed points associated with maps that evolve over time. In particular, the paper considers the Krasnosel'skii-Mann method in a settings where: (i) the underlying map may change at each step of the algorithm, thus leading to a "running" implementation of the Krasnosel'skii-Mann method; and, (ii) an imper… ▽ More This paper leverages a framework based on averaged operators to tackle the problem of tracking fixed points associated with maps that evolve over time. In particular, the paper considers the Krasnosel'skii-Mann method in a settings where: (i) the underlying map may change at each step of the algorithm, thus leading to a "running" implementation of the Krasnosel'skii-Mann method; and, (ii) an imperfect information of the map may be available. An imperfect knowledge of the maps can capture cases where processors feature a finite precision or quantization errors, or the case where (part of) the map is obtained from measurements. The analytical results are applicable to inexact running algorithms for solving optimization problems, whenever the algorithmic steps can be written in the form of (a composition of) averaged operators; examples are provided for inexact running gradient methods and the forward-backward splitting method. Convergence of the average fixed-point residual is investigated for the non-expansive case; linear convergence to a unique fixed-point trajectory is showed in the case of inexact running algorithms emerging from contractive operators. △ Less

Submitted 17 April, 2019; originally announced April 2019.

Journal ref: IEEE Control Systems Letters, vol. 3 (3), pages 613 - 618, 2019

Optimal Rate of Convergence for Quasi-Stochastic Approximation

Authors: Andrey Bernstein, Yue Chen, Marcello Colombino, Emiliano Dall'Anese, Prashant Mehta, Sean Meyn

Abstract: The Robbins-Monro stochastic approximation algorithm is a foundation of many algorithmic frameworks for reinforcement learning (RL), and often an efficient approach to solving (or approximating the solution to) complex optimal control problems. However, in many cases practitioners are unable to apply these techniques because of an inherent high variance. This paper aims to provide a general founda… ▽ More The Robbins-Monro stochastic approximation algorithm is a foundation of many algorithmic frameworks for reinforcement learning (RL), and often an efficient approach to solving (or approximating the solution to) complex optimal control problems. However, in many cases practitioners are unable to apply these techniques because of an inherent high variance. This paper aims to provide a general foundation for "quasi-stochastic approximation," in which all of the processes under consideration are deterministic, much like quasi-Monte-Carlo for variance reduction in simulation. The variance reduction can be substantial, subject to tuning of pertinent parameters in the algorithm. This paper introduces a new coupling argument to establish optimal rate of convergence provided the gain is sufficiently large. These results are established for linear models, and tested also in non-ideal settings. A major application of these general results is a new class of RL algorithms for deterministic state space models. In this setting, the main contribution is a class of algorithms for approximating the value function for a given policy, using a different policy designed to introduce exploration. △ Less

Submitted 17 March, 2019; originally announced March 2019.

Online Sparse Subspace Clustering

Authors: Liam Madden, Stephen Becker, Emiliano Dall'Anese

Abstract: This paper focuses on the sparse subspace clustering problem, and develops an online algorithmic solution to cluster data points on-the-fly, without revisiting the whole dataset. The strategy involves an online solution of a sparse representation (SR) problem to build a (sparse) dictionary of similarities where points in the same subspace are considered "similar," followed by a spectral clustering… ▽ More This paper focuses on the sparse subspace clustering problem, and develops an online algorithmic solution to cluster data points on-the-fly, without revisiting the whole dataset. The strategy involves an online solution of a sparse representation (SR) problem to build a (sparse) dictionary of similarities where points in the same subspace are considered "similar," followed by a spectral clustering based on the obtained similarity matrix. When the SR cost is strongly convex, the online solution converges to within a neighborhood of the optimal time-varying batch solution. A dynamic regret analysis is performed when the SR cost is not strongly convex. △ Less

Submitted 3 June, 2019; v1 submitted 27 February, 2019; originally announced February 2019.

Comments: 4 pages, 4 figures. Copyright 2019 IEEE. Published in the 2019 IEEE Data Science Workshop (DSW 2019), scheduled for June 4-6, 2019 in Minneapolis, Minnesota

Journal ref: 2nd IEEE Data Science Workshop, June 2019 Minneapolis

Dynamic Distribution State Estimation Using Synchrophasor Data

Authors: Jianhan Song, Emiliano Dall'Anese, Andrea Simonetto, Hao Zhu

Abstract: The increasing deployment of distribution-level phasor measurement units (PMUs) calls for dynamic distribution state estimation (DDSE) approaches that tap into high-rate measurements to maintain a comprehensive view of the distribution-system state in real time. Accordingly, this paper explores the development of a fast algorithmic framework by casting the DDSE task within the time-varying optimiz… ▽ More The increasing deployment of distribution-level phasor measurement units (PMUs) calls for dynamic distribution state estimation (DDSE) approaches that tap into high-rate measurements to maintain a comprehensive view of the distribution-system state in real time. Accordingly, this paper explores the development of a fast algorithmic framework by casting the DDSE task within the time-varying optimization realm. The time-varying formulation involves a time-varying robustified least-squares approach, and it naturally models optimal trajectories for the estimated states under streaming of measurements. The formulation is based on a linear surrogate of the AC power-flow equations, and it includes an element of robustness with respect to measurement outliers. The paper then leverages a first-order prediction-correction method to achieve simple online updates that can provably track the state variables from heterogeneous measurements. This online algorithm is computationally efficient as it relies on the Hessian of the cost function without computing matrix-inverse. Convergence and bounds on the estimation errors of proposed algorithm can be analytically established. △ Less

Submitted 8 January, 2019; originally announced January 2019.

Comments: 8 pages, 7 figures

Journal ref: IEEE Transactions on Smart Grid, vol. 11 (1), 821 - 831, 2020

Aggregate Power Flexibility in Unbalanced Distribution Systems

Authors: Xin Chen, Emiliano Dall'Anese, Changhong Zhao, Na Li

Abstract: With a large-scale integration of distributed energy resources (DERs), distribution systems are expected to be capable of providing capacity support for the transmission grid. To effectively harness the collective flexibility from massive DER devices, this paper studies distribution-level power aggregation strategies for transmission-distribution interaction. In particular, this paper proposes a m… ▽ More With a large-scale integration of distributed energy resources (DERs), distribution systems are expected to be capable of providing capacity support for the transmission grid. To effectively harness the collective flexibility from massive DER devices, this paper studies distribution-level power aggregation strategies for transmission-distribution interaction. In particular, this paper proposes a method to model and quantify the aggregate power flexibility, i.e., the net power injection achievable at the substation, in unbalanced distribution systems over time. Incorporating the network constraints and multi-phase unbalanced modeling, the proposed method obtains an effective approximate feasible region of the net power injection. For any aggregate power trajectory within this region, it is proved that there exists a feasible disaggregation solution. In addition, a distributed model predictive control (MPC) framework is developed for the practical implementation of the transmission-distribution interaction. At last, we demonstrate the performances of the proposed method via numerical tests on a real-world distribution feeder with 126 multi-phase nodes. △ Less

Submitted 31 May, 2019; v1 submitted 14 December, 2018; originally announced December 2018.

Comments: 12 pages

Running Primal-Dual Gradient Method for Time-Varying Nonconvex Problems

Authors: Yujie Tang, Emiliano Dall'Anese, Andrey Bernstein, Steven Low

Abstract: This paper considers a nonconvex optimization problem that evolves over time, and addresses the synthesis and analysis of regularized primal-dual gradient methods to track a Karush-Kuhn-Tucker (KKT) trajectory. The proposed regularized primal-dual gradient methods are implemented in a running fashion, in the sense that the underlying optimization problem changes during the iterations of the algori… ▽ More This paper considers a nonconvex optimization problem that evolves over time, and addresses the synthesis and analysis of regularized primal-dual gradient methods to track a Karush-Kuhn-Tucker (KKT) trajectory. The proposed regularized primal-dual gradient methods are implemented in a running fashion, in the sense that the underlying optimization problem changes during the iterations of the algorithms. For a problem with twice continuously differentiable cost and constraints, and under a generalization of the Mangasarian-Fromovitz constraint qualification, sufficient conditions are derived for the running algorithm to track a KKT trajectory. Further, asymptotic bounds for the tracking error (as a function of the time-variability of a KKT trajectory) are obtained. A continuous-time version of the algorithm, framed as a system of differential inclusions, is also considered and analytical convergence results are derived. For the continuous-time setting, a set of sufficient conditions for the KKT trajectories not to bifurcate or merge is proposed. Illustrative numerical results inspired by a real-world application are provided. △ Less

Submitted 3 December, 2018; originally announced December 2018.

Inner Approximation of Minkowski Sums: A Union-Based Approach and Applications to Aggregated Energy Resources

Authors: Md Salman Nazir, Ian A. Hiskens, Andrey Bernstein, Emiliano Dall'Anese

Abstract: This paper develops and compares algorithms to compute inner approximations of the Minkowski sum of convex polytopes. As an application, the paper considers the computation of the feasibility set of aggregations of distributed energy resources (DERs), such as solar photovoltaic inverters, controllable loads, and storage devices. To fully account for the heterogeneity in the DERs while ensuring an… ▽ More This paper develops and compares algorithms to compute inner approximations of the Minkowski sum of convex polytopes. As an application, the paper considers the computation of the feasibility set of aggregations of distributed energy resources (DERs), such as solar photovoltaic inverters, controllable loads, and storage devices. To fully account for the heterogeneity in the DERs while ensuring an acceptable approximation accuracy, the paper leverages a union-based computation and advocates homothet-based polytope decompositions. However, union-based approaches can in general lead to high-dimensionality concerns; to alleviate this issue, this paper shows how to define candidate sets to reduce the computational complexity. Accuracy and trade-offs are analyzed through numerical simulations for illustrative examples. △ Less

Submitted 3 October, 2018; originally announced October 2018.

Comments: Accepted for presentation at the 57th IEEE Conference on Decision and Control

Online Optimization as a Feedback Controller: Stability and Tracking

Authors: Marcello Colombino, Emiliano Dall'Anese, Andrey Bernstein

Abstract: This paper develops and analyzes feedback-based online optimization methods to regulate the output of a linear time-invariant (LTI) dynamical system to the optimal solution of a time-varying convex optimization problem. The design of the algorithm is based on continuous-time primal-dual dynamics, properly modified to incorporate feedback from the LTI dynamical system, applied to a proximal augment… ▽ More This paper develops and analyzes feedback-based online optimization methods to regulate the output of a linear time-invariant (LTI) dynamical system to the optimal solution of a time-varying convex optimization problem. The design of the algorithm is based on continuous-time primal-dual dynamics, properly modified to incorporate feedback from the LTI dynamical system, applied to a proximal augmented Lagrangian function. The resultant closed-loop algorithm tracks the solution of the time-varying optimization problem without requiring knowledge of (time-varying) disturbances in the dynamical system. The analysis leverages integral quadratic constraints to provide linear matrix inequality (LMI) conditions that guarantee global exponential stability and bounded tracking error. Analytical results show that, under a sufficient time-scale separation between the dynamics of the LTI dynamical system and the algorithm, the LMI conditions can be always satisfied. The paper further proposes a modified algorithm that can track an approximate solution trajectory of the constrained optimization problem under less restrictive assumptions. As an illustrative example, the proposed algorithms are showcased for power transmission systems, to compress the time scales between secondary and tertiary control, and allow to simultaneously power re-balancing and tracking of DC optimal power flow points. △ Less

Submitted 29 May, 2018; v1 submitted 24 May, 2018; originally announced May 2018.

Asynchronous and Distributed Tracking of Time-Varying Fixed Points

Authors: Andrey Bernstein, Emiliano Dall'Anese

Abstract: This paper develops an algorithmic framework for tracking fixed points of time-varying contraction map**s. Analytical results for the tracking error are established for the cases where: (i) the underlying contraction self-map changes at each step of the algorithm; (ii) only an imperfect information of the map is available; and, (iii) the algorithm is implemented in a distributed fashion, with co… ▽ More This paper develops an algorithmic framework for tracking fixed points of time-varying contraction map**s. Analytical results for the tracking error are established for the cases where: (i) the underlying contraction self-map changes at each step of the algorithm; (ii) only an imperfect information of the map is available; and, (iii) the algorithm is implemented in a distributed fashion, with communication delays and packet drops leading to asynchronous algorithmic updates. The analytical results are applicable to several classes of problems, including time-varying contraction map**s emerging from online and asynchronous implementations of gradient-based methods for time-varying convex programs. In this domain, the proposed framework can also capture the operating principles of feedback-based online algorithms, where the online gradient steps are suitably modified to accommodate actionable feedback from an underlying physical or logical network. Examples of applications and illustrative numerical results are provided. △ Less

Submitted 12 September, 2018; v1 submitted 25 April, 2018; originally announced April 2018.

Data-based Distributionally Robust Stochastic Optimal Power Flow, Part I: Methodologies

Authors: Yi Guo, Kyri Baker, Emiliano Dall'Anese, Zechun Hu, Tyler H. Summers

Abstract: We propose a data-based method to solve a multi-stage stochastic optimal power flow (OPF) problem based on limited information about forecast error distributions. The framework explicitly combines multi-stage feedback policies with any forecasting method and historical forecast error data. The objective is to determine power scheduling policies for controllable devices in a power network to balanc… ▽ More We propose a data-based method to solve a multi-stage stochastic optimal power flow (OPF) problem based on limited information about forecast error distributions. The framework explicitly combines multi-stage feedback policies with any forecasting method and historical forecast error data. The objective is to determine power scheduling policies for controllable devices in a power network to balance operational cost and conditional value-at-risk (CVaR) of device and network constraint violations. These decisions include both nominal power schedules and reserve policies, which specify planned reactions to forecast errors in order to accommodate fluctuating renewable energy sources. Instead of assuming the uncertainties across the networks follow prescribed probability distributions, we consider ambiguity sets of distributions centered around a finite training dataset. By utilizing the Wasserstein metric to quantify differences between the empirical data-based distribution and the real unknown data-generating distribution, we formulate a multi-stage distributionally robust OPF problem to compute optimal control policies that are robust to both forecast errors and sampling errors inherent in the dataset. Two specific data-based distributionally robust stochastic OPF problems are proposed for distribution networks and transmission systems. △ Less

Submitted 25 October, 2018; v1 submitted 17 April, 2018; originally announced April 2018.

Comments: arXiv admin note: text overlap with arXiv:1706.04267

Data-based Distributionally Robust Stochastic Optimal Power Flow, Part II: Case studies

Authors: Yi Guo, Kyri Baker, Emiliano Dall'Anese, Zechun Hu, Tyler H. Summers

Abstract: This is the second part of a two-part paper on data-based distributionally robust stochastic optimal power flow (OPF). The general problem formulation and methodology have been presented in Part I [1]. Here, we present extensive numerical experiments in both distribution and transmission networks to illustrate the effectiveness and flexibility of the proposed methodology for balancing efficiency,… ▽ More This is the second part of a two-part paper on data-based distributionally robust stochastic optimal power flow (OPF). The general problem formulation and methodology have been presented in Part I [1]. Here, we present extensive numerical experiments in both distribution and transmission networks to illustrate the effectiveness and flexibility of the proposed methodology for balancing efficiency, constraint violation risk, and out-of-sample performance. On the distribution side, the method mitigates overvoltages due to high photovoltaic penetration using local energy storage devices. On the transmission side, the method reduces N-1 security line flow constraint risks due to high wind penetration using reserve policies for controllable generators. In both cases, the data-based distributionally robust model predictive control (MPC) algorithm explicitly utilizes forecast error training datasets, which can be updated online. The numerical results illustrate inherent tradeoffs between the operational costs, risks of constraints violations, and out-of-sample performance, offering systematic techniques for system operators to balance these objectives. △ Less

Submitted 25 October, 2018; v1 submitted 17 April, 2018; originally announced April 2018.

Online Primal-Dual Methods with Measurement Feedback for Time-Varying Convex Optimization

Authors: Andrey Bernstein, Emiliano Dall'Anese, Andrea Simonetto

Abstract: This paper addresses the design and analysis of feedback-based online algorithms to control systems or networked systems based on performance objectives and engineering constraints that may evolve over time. The emerging time-varying convex optimization formalism is leveraged to model optimal operational trajectories of the systems, as well as explicit local and network-level operational constrain… ▽ More This paper addresses the design and analysis of feedback-based online algorithms to control systems or networked systems based on performance objectives and engineering constraints that may evolve over time. The emerging time-varying convex optimization formalism is leveraged to model optimal operational trajectories of the systems, as well as explicit local and network-level operational constraints. Departing from existing batch and feed-forward optimization approaches, the design of the algorithms capitalizes on an online implementation of primal-dual projected-gradient methods; the gradient steps are, however, suitably modified to accommodate feedback from the system in the form of measurements - hence, the term "online optimization with feedback." By virtue of this approach, the resultant algorithms can cope with model mismatches in the algebraic representation of the system states and outputs, they avoid pervasive measurements of exogenous inputs, and they naturally lend themselves to a distributed implementation. Under suitable assumptions, analytical convergence claims are established in terms of dynamic regret. Furthermore, when the synthesis of the feedback-based online algorithms is based on a regularized Lagrangian function, Q-linear convergence to solutions of the time-varying optimization problem is shown. △ Less

Submitted 6 March, 2019; v1 submitted 13 April, 2018; originally announced April 2018.

Comments: Alphabetical order, authors contributed equally. Old title: Online Optimization with Feedback

Online Stochastic Optimization of Networked Distributed Energy Resources

Authors: Xinyang Zhou, Emiliano Dall'Anese, Lijun Chen

Abstract: This paper investigates distributed control and incentive mechanisms to coordinate distributed energy resources (DERs) with both continuous and discrete decision variables as well as device dynamics in distribution grids. We formulate a multi-period social welfare maximization problem, and based on its convex relaxation propose a distributed stochastic dual gradient algorithm for managing DERs. We… ▽ More This paper investigates distributed control and incentive mechanisms to coordinate distributed energy resources (DERs) with both continuous and discrete decision variables as well as device dynamics in distribution grids. We formulate a multi-period social welfare maximization problem, and based on its convex relaxation propose a distributed stochastic dual gradient algorithm for managing DERs. We further extend it to an online realtime setting with time-varying operating conditions, asynchronous updates by devices, and feedback being leveraged to account for nonlinear power flows as well as reduce communication overhead. The resulting algorithm provides a general online stochastic optimization algorithm for coordinating networked DERs with discrete power setpoints and dynamics to meet operational and economic objectives and constraints. We characterize the convergence of the algorithm analytically and evaluate its performance numerically. △ Less

Submitted 14 June, 2019; v1 submitted 27 November, 2017; originally announced November 2017.

Journal ref: IEEE Transactions on Automatic Control, 2019

Real-Time Feedback-Based Optimization of Distribution Grids: A Unified Approach

Authors: Andrey Bernstein, Emiliano Dall'Anese

Abstract: This paper develops an algorithmic framework for real-time optimization of distribution-level distributed energy resources (DERs). The proposed framework optimizes the operation of both DERs that are individually controllable and groups of DERs (i.e., aggregations) at an electrical point of connection that are jointly controlled. From an electrical standpoint, wye and delta single- and multi-phase… ▽ More This paper develops an algorithmic framework for real-time optimization of distribution-level distributed energy resources (DERs). The proposed framework optimizes the operation of both DERs that are individually controllable and groups of DERs (i.e., aggregations) at an electrical point of connection that are jointly controlled. From an electrical standpoint, wye and delta single- and multi-phase connections are accounted for. The algorithm enables (groups of) DERs to pursue given performance objectives, while adjusting their (aggregate) powers to respond to services requested by grid operators and to maintain electrical quantities within engineering limits. The design of the algorithm leverages a time-varying bi-level problem formulation capturing various performance objectives and engineering constraints, and an online implementation of primal-dual projected-gradient methods. The gradient steps are suitably modified to accommodate appropriate measurements from the distribution network and the DERs. By virtue of this approach, the resultant algorithm can cope with inaccuracies in the distribution-system modeling, it avoids pervasive metering to gather the state of non-controllable resources, and it naturally lends itself to a distributed implementation. Analytical stability and convergence claims are established in terms of tracking of the solution of the formulated time-varying optimization problem. The proposed method is tested in a realistic distribution system with real data. △ Less

Submitted 26 February, 2019; v1 submitted 5 November, 2017; originally announced November 2017.