Showing 1–8 of 8 results for author: Hossain, R R

Structural Impact of Grid-Forming Inverters on Power System Coherency

Authors: Sayak Mukherjee, Ramij Raja Hossain, Kaustav Chatterjee, Soumya Kundu, Kyung-Bin Kwon, Sameer Nekkalapu, Marcelo Elizondo

Abstract: This paper addresses the following fundamental research question: how does the integration of grid-forming inverters (GFMs) replacing conventional synchronous generators (SGs) impact the slow coherent eigen-structure and the low-frequency oscillatory behavior of future power systems? Due to time-scale separated dynamics, generator states inside a coherent area synchronize over a fast time-scale du… ▽ More This paper addresses the following fundamental research question: how does the integration of grid-forming inverters (GFMs) replacing conventional synchronous generators (SGs) impact the slow coherent eigen-structure and the low-frequency oscillatory behavior of future power systems? Due to time-scale separated dynamics, generator states inside a coherent area synchronize over a fast time-scale due to stronger coupling, while the areas themselves synchronize over a slower time scale. Our mathematical analysis shows that due to the large-scale integration of GFMs, the weighted Laplacian structure of the frequency dynamics is preserved, however, the entries of the Laplacian may be significantly modified based on the location and penetration levels of the GFMs. This can impact and potentially significantly alter the coherency structure of the system. We have validated our findings with numerical results using the IEEE 68-bus test system. △ Less

Submitted 15 May, 2024; originally announced May 2024.

Comments: 10 pages, 5 figures

Resilient Control of Networked Microgrids using Vertical Federated Reinforcement Learning: Designs and Real-Time Test-Bed Validations

Authors: Sayak Mukherjee, Ramij R. Hossain, Sheik M. Mohiuddin, Yuan Liu, Wei Du, Veronica Adetola, Rohit A. **siwale, Qiuhua Huang, Tianzhixi Yin, Ankit Singhal

Abstract: Improving system-level resiliency of networked microgrids is an important aspect with increased population of inverter-based resources (IBRs). This paper (1) presents resilient control design in presence of adversarial cyber-events, and proposes a novel federated reinforcement learning (Fed-RL) approach to tackle (a) model complexities, unknown dynamical behaviors of IBR devices, (b) privacy issue… ▽ More Improving system-level resiliency of networked microgrids is an important aspect with increased population of inverter-based resources (IBRs). This paper (1) presents resilient control design in presence of adversarial cyber-events, and proposes a novel federated reinforcement learning (Fed-RL) approach to tackle (a) model complexities, unknown dynamical behaviors of IBR devices, (b) privacy issues regarding data sharing in multi-party-owned networked grids, and (2) transfers learned controls from simulation to hardware-in-the-loop test-bed, thereby bridging the gap between simulation and real world. With these multi-prong objectives, first, we formulate a reinforcement learning (RL) training setup generating episodic trajectories with adversaries (attack signal) injected at the primary controllers of the grid forming (GFM) inverters where RL agents (or controllers) are being trained to mitigate the injected attacks. For networked microgrids, the horizontal Fed-RL method involving distinct independent environments is not appropriate, leading us to develop vertical variant Federated Soft Actor-Critic (FedSAC) algorithm to grasp the interconnected dynamics of networked microgrid. Next, utilizing OpenAI Gym interface, we built a custom simulation set-up in GridLAB-D/HELICS co-simulation platform, named Resilient RL Co-simulation (ResRLCoSIM), to train the RL agents with IEEE 123-bus benchmark test systems comprising 3 interconnected microgrids. Finally, the learned policies in simulation world are transferred to the real-time hardware-in-the-loop test-bed set-up developed using high-fidelity Hypersim platform. Experiments show that the simulator-trained RL controllers produce convincing results with the real-time test-bed set-up, validating the minimization of sim-to-real gap. △ Less

Submitted 20 November, 2023; originally announced November 2023.

Comments: 10 pages, 7 figures

Enhancing Cyber Resilience of Networked Microgrids using Vertical Federated Reinforcement Learning

Authors: Sayak Mukherjee, Ramij R. Hossain, Yuan Liu, Wei Du, Veronica Adetola, Sheik M. Mohiuddin, Qiuhua Huang, Tianzhixi Yin, Ankit Singhal

Abstract: This paper presents a novel federated reinforcement learning (Fed-RL) methodology to enhance the cyber resiliency of networked microgrids. We formulate a resilient reinforcement learning (RL) training setup which (a) generates episodic trajectories injecting adversarial actions at primary control reference signals of the grid forming (GFM) inverters and (b) trains the RL agents (or controllers) to… ▽ More This paper presents a novel federated reinforcement learning (Fed-RL) methodology to enhance the cyber resiliency of networked microgrids. We formulate a resilient reinforcement learning (RL) training setup which (a) generates episodic trajectories injecting adversarial actions at primary control reference signals of the grid forming (GFM) inverters and (b) trains the RL agents (or controllers) to alleviate the impact of the injected adversaries. To circumvent data-sharing issues and concerns for proprietary privacy in multi-party-owned networked grids, we bring in the aspects of federated machine learning and propose a novel Fed-RL algorithm to train the RL agents. To this end, the conventional horizontal Fed-RL approaches using decoupled independent environments fail to capture the coupled dynamics in a networked microgrid, which leads us to propose a multi-agent vertically federated variation of actor-critic algorithms, namely federated soft actor-critic (FedSAC) algorithm. We created a customized simulation setup encapsulating microgrid dynamics in the GridLAB-D/HELICS co-simulation platform compatible with the OpenAI Gym interface for training RL agents. Finally, the proposed methodology is validated with numerical examples of modified IEEE 123-bus benchmark test systems consisting of three coupled microgrids. △ Less

Submitted 17 December, 2022; originally announced December 2022.

Comments: 13 pages, 5 figures

Efficient Learning of Voltage Control Strategies via Model-based Deep Reinforcement Learning

Authors: Ramij R. Hossain, Tianzhixi Yin, Yan Du, Renke Huang, Jie Tan, Wenhao Yu, Yuan Liu, Qiuhua Huang

Abstract: This article proposes a model-based deep reinforcement learning (DRL) method to design emergency control strategies for short-term voltage stability problems in power systems. Recent advances show promising results in model-free DRL-based methods for power systems, but model-free methods suffer from poor sample efficiency and training time, both critical for making state-of-the-art DRL algorithms… ▽ More This article proposes a model-based deep reinforcement learning (DRL) method to design emergency control strategies for short-term voltage stability problems in power systems. Recent advances show promising results in model-free DRL-based methods for power systems, but model-free methods suffer from poor sample efficiency and training time, both critical for making state-of-the-art DRL algorithms practically applicable. DRL-agent learns an optimal policy via a trial-and-error method while interacting with the real-world environment. And it is desirable to minimize the direct interaction of the DRL agent with the real-world power grid due to its safety-critical nature. Additionally, state-of-the-art DRL-based policies are mostly trained using a physics-based grid simulator where dynamic simulation is computationally intensive, lowering the training efficiency. We propose a novel model-based-DRL framework where a deep neural network (DNN)-based dynamic surrogate model, instead of a real-world power-grid or physics-based simulation, is utilized with the policy learning framework, making the process faster and sample efficient. However, stabilizing model-based DRL is challenging because of the complex system dynamics of large-scale power systems. We solved these issues by incorporating imitation learning to have a warm start in policy learning, reward-sha**, and multi-step surrogate loss. Finally, we achieved 97.5% sample efficiency and 87.7% training efficiency for an application to the IEEE 300-bus test system. △ Less

Submitted 5 December, 2022; originally announced December 2022.

Data-Driven Linear Koopman Embedding for Networked Systems: Model-Predictive Grid Control

Authors: Ramij R. Hossain, Rahmat Adesunkanmi, Ratnesh Kumar

Abstract: This paper presents a data-learned linear Koopman embedding of nonlinear networked dynamics and uses it to enable real-time model predictive emergency voltage control in a power network. The approach involves a novel data-driven ``basis-dictionary free" lifting of the system dynamics into a higher dimensional linear space over which an MPC (model predictive control) is exercised, making it both sc… ▽ More This paper presents a data-learned linear Koopman embedding of nonlinear networked dynamics and uses it to enable real-time model predictive emergency voltage control in a power network. The approach involves a novel data-driven ``basis-dictionary free" lifting of the system dynamics into a higher dimensional linear space over which an MPC (model predictive control) is exercised, making it both scalable and rapid for practical real-time implementation. A Koopman-inspired deep neural network (KDNN) encoder-decoder architecture for the linear embedding of the underlying dynamics under distributed controls is presented, in which the end-to-end components of the KDNN comprising of a triple of transforms is learned from the system trajectory data in one go: A Neural Network (NN)-based lifting to a higher dimension, a linear dynamics within that higher dimension, and an NN-based projection to the original space. This data-learned approach relieves the burden of the ad-hoc selection of the nonlinear basis functions (e.g., polynomial or radial) used in conventional approaches for lifting to higher dimensional linear space. We validate the efficacy and robustness of the approach via application to the standard IEEE 39-bus system. △ Less

Submitted 28 October, 2022; v1 submitted 2 June, 2022; originally announced June 2022.

Journal ref: IEEE Systems Journal ( Volume: 17, Issue: 3, September 2023)

Distributed-MPC with Data-Driven Estimation of Bus Admittance Matrix in Voltage Control

Authors: Ramij R. Hossain, Ratnesh Kumar

Abstract: This article presents a distributed model-predictive control (MPC) design for real-time voltage control in power systems, including an online method to estimate the bus admittance matrix $\mathbf{Y}$ to let it be time-varying and unknown a priori. The prevalent control designs are either (a) centralized, providing optimal solutions but less scalable and susceptible to single-point failures/attacks… ▽ More This article presents a distributed model-predictive control (MPC) design for real-time voltage control in power systems, including an online method to estimate the bus admittance matrix $\mathbf{Y}$ to let it be time-varying and unknown a priori. The prevalent control designs are either (a) centralized, providing optimal solutions but less scalable and susceptible to single-point failures/attacks, or (b) decentralized or localized, having increased scalability and attack resilience but are suboptimal. The proposed distributed solution offers the attractive features of both methodologies, where neighboring nodes share state information to attain a globally optimal solution. In addition, the presented framework provides a data-driven estimation of Y to circumvent the challenging issue of acquiring accurate knowledge of the line impedance (required to form Y). We first introduce the centralized version of the predictive voltage control problem and then transfer it to a distributed version. The distributed version is solved via the alternating direction method of multipliers (ADMM), using only local measurements and communication leveraging the graph structure of the power system. The proposed framework is resilient to prediction uncertainty, modeling error, and communication link failure, and the in-built redundancy within the proposed framework supports anomaly detection in cyberattacks. We validate the proposed methodology for IEEE-30 bus, IEEE-57 bus transmission systems, and IEEE-123 bus distribution systems. △ Less

Submitted 28 October, 2022; v1 submitted 28 February, 2022; originally announced February 2022.

Data-Driven Pole Placement in LMI Regions with Robustness Constraints

Authors: Sayak Mukherjee, Ramij R. Hossain

Abstract: This paper proposes a robust learning methodology to place the closed-loop poles in desired convex regions in the complex plane. We considered the system state and input matrices to be unknown and can only use the measurements of the system trajectories. The closed-loop pole placement problem in the linear matrix inequality (LMI) regions is considered a classic robust control problem; however, tha… ▽ More This paper proposes a robust learning methodology to place the closed-loop poles in desired convex regions in the complex plane. We considered the system state and input matrices to be unknown and can only use the measurements of the system trajectories. The closed-loop pole placement problem in the linear matrix inequality (LMI) regions is considered a classic robust control problem; however, that requires knowledge about the state and input matrices of the linear system. We bring in ideas from the behavioral system theory and persistency of excitation condition-based fundamental lemma to develop a data-driven counterpart that satisfies multiple closed-loop robustness specifications, such as $\mathcal{D}$-stability and mixed $H_2/H_{\infty}$ performance specifications. Our formulations lead to data-driven semi-definite programs (SDPs) that are coupled with sufficient theoretical guarantees. We validate the theoretical results with numerical simulations on a third-order dynamic system. △ Less

Submitted 12 November, 2021; originally announced November 2021.

Comments: This version contains 12 pages

MPC-based Realtime Power System Control with DNN-based Prediction/Sensitivity-Estimation

Authors: Ramij Raja Hossain, Ratnesh Kumar

Abstract: This paper presents a model predictive control (MPC)-based online real-time adaptive control scheme for emergency voltage control in power systems. Despite tremendous success in various applications, real-time implementation of MPC for control in power systems has not been successful due to its online computational burden for large-sized systems that takes more time than available between the two… ▽ More This paper presents a model predictive control (MPC)-based online real-time adaptive control scheme for emergency voltage control in power systems. Despite tremendous success in various applications, real-time implementation of MPC for control in power systems has not been successful due to its online computational burden for large-sized systems that takes more time than available between the two control decisions. This long-standing problem is addressed here by develo** a novel MPC-based adaptive control framework which (i) adapts the nominal offline computed control, by successive control corrections, at each control decision point using the latest measurements, (ii) utilizes data-driven approach for prediction of voltage trajectory and its sensitivity with respect to control using trained deep neural networks (DNNs). In addition, a realistic coordination scheme among control inputs of static var compensators (SVC), load-shedding (LS), and load tap-changers (LTC) is presented with a goal of maintaining bus voltages within a predefined permissible range, where the delayed effect of LTC action is also incorporated in a novel way. The performance of the proposed scheme is validated for IEEE 9-bus as well as 39-bus systems, with $\pm 20\%$ variations in nominal loading conditions. We also show that the proposed new scheme speeds up the online computation by a factor of 20 bringing it down to under one-tenth the control interval, making the MPC-based power system control practically feasible. △ Less

Submitted 5 June, 2021; originally announced June 2021.