Showing 1–4 of 4 results for author: Palensky, P

A Constraint Enforcement Deep Reinforcement Learning Framework for Optimal Energy Storage Systems Dispatch

Authors: Shengren Hou, Edgar Mauricio Salazar Duque, Peter Palensky, Pedro P. Vergara

Abstract: The optimal dispatch of energy storage systems (ESSs) presents formidable challenges due to the uncertainty introduced by fluctuations in dynamic prices, demand consumption, and renewable-based energy generation. By exploiting the generalization capabilities of deep neural networks (DNNs), deep reinforcement learning (DRL) algorithms can learn good-quality control models that adaptively respond to… ▽ More The optimal dispatch of energy storage systems (ESSs) presents formidable challenges due to the uncertainty introduced by fluctuations in dynamic prices, demand consumption, and renewable-based energy generation. By exploiting the generalization capabilities of deep neural networks (DNNs), deep reinforcement learning (DRL) algorithms can learn good-quality control models that adaptively respond to distribution networks' stochastic nature. However, current DRL algorithms lack the capabilities to enforce operational constraints strictly, often even providing unfeasible control actions. To address this issue, we propose a DRL framework that effectively handles continuous action spaces while strictly enforcing the environments and action space operational constraints during online operation. Firstly, the proposed framework trains an action-value function modeled using DNNs. Subsequently, this action-value function is formulated as a mixed-integer programming (MIP) formulation enabling the consideration of the environment's operational constraints. Comprehensive numerical simulations show the superior performance of the proposed MIP-DRL framework, effectively enforcing all constraints while delivering high-quality dispatch decisions when compared with state-of-the-art DRL algorithms and the optimal solution obtained with a perfect forecast of the stochastic variables. △ Less

Submitted 26 July, 2023; originally announced July 2023.

Comments: This paper has been submitted to a publication in a journal. This corresponds to the submitted version. After acceptance, it may be removed depending on the journal's requirements for copyright

Dynamic Anomaly Detection with High-fidelity Simulators: A Convex Optimization Approach

Authors: Kaikai Pan, Peter Palensky, Peyman Mohajerin Esfahani

Abstract: The main objective of this article is to develop scalable dynamic anomaly detectors when high-fidelity simulators of power systems are at our disposal. On the one hand, mathematical models of these high-fidelity simulators are typically "intractable" to apply existing model-based approaches. On the other hand, pure data-driven methods developed primarily in the machine learning literature neglect… ▽ More The main objective of this article is to develop scalable dynamic anomaly detectors when high-fidelity simulators of power systems are at our disposal. On the one hand, mathematical models of these high-fidelity simulators are typically "intractable" to apply existing model-based approaches. On the other hand, pure data-driven methods developed primarily in the machine learning literature neglect our knowledge about the underlying dynamics of the systems. In this study, we combine tools from these two mainstream approaches to develop a diagnosis filter that utilizes the knowledge of both the dynamical system as well as the simulation data of the high-fidelity simulators. The proposed diagnosis filter aims to achieve two desired features: (i) performance robustness with respect to model mismatch; (ii) high scalability. To this end, we propose a tractable (convex) optimization-based reformulation in which decisions are the filter parameters, the model-based information introduces feasible sets, and the data from the simulator forms the objective function to-be-minimized regarding the effect of model mismatch on the filter performance. To validate the theoretical results, we implement the developed diagnosis filter in DIgSILENT PowerFactory to detect false data injection attacks on the Automatic Generation Control measurements in the three-area IEEE 39-bus system. △ Less

Submitted 6 October, 2020; v1 submitted 28 April, 2020; originally announced April 2020.

Comments: 19 pages

False Data Injection Attacks on Hybrid AC/HVDC Interconnected System with Virtual Inertia -- Vulnerability, Impact and Detection

Authors: Kaikai Pan, Elyas Rakhshani, Peter Palensky

Abstract: Power systems are moving towards hybrid AC/DC grids with the integration of HVDC links, renewable resources and energy storage modules. New models of frequency control have to consider the complex interactions between these components. Meanwhile, more attention should be paid to cyber security concerns as these control strategies highly depend on data communications which may be exposed to cyber a… ▽ More Power systems are moving towards hybrid AC/DC grids with the integration of HVDC links, renewable resources and energy storage modules. New models of frequency control have to consider the complex interactions between these components. Meanwhile, more attention should be paid to cyber security concerns as these control strategies highly depend on data communications which may be exposed to cyber attacks. In this regard, this article aims to analyze the false data injection (FDI) attacks on the AC/DC interconnected system with virtual inertia and develop advanced diagnosis tools to reveal their occurrence. We build an optimization-based framework for the purpose of vulnerability and attack impact analysis. Considering the attack impact on the system frequency stability, it is shown that the hybrid grid with parallel AC/DC links and emulated inertia is more vulnerable to the FDI attacks, compared with the one without virtual inertia and the normal AC system. We then propose a detection approach to detect and isolate each FDI intrusion with a sufficient fast response, and even recover the attack value. In addition to theoretical results, the effectiveness of the proposed methods is validated through simulations on the two-area AC/DC interconnected system with virtual inertia emulation capabilities. △ Less

Submitted 30 May, 2020; v1 submitted 20 January, 2020; originally announced January 2020.

From Static to Dynamic Anomaly Detection with Application to Power System Cyber Security

Authors: Kaikai Pan, Peter Palensky, Peyman Mohajerin Esfahani

Abstract: Develo** advanced diagnosis tools to detect cyber attacks is the key to security of power systems. It has been shown that multivariate data injection attacks can bypass bad data detection schemes typically built on static behavior of the systems, which misleads operators to disruptive decisions. In this article, we depart from the existing static viewpoint to develop a diagnosis filter that capt… ▽ More Develo** advanced diagnosis tools to detect cyber attacks is the key to security of power systems. It has been shown that multivariate data injection attacks can bypass bad data detection schemes typically built on static behavior of the systems, which misleads operators to disruptive decisions. In this article, we depart from the existing static viewpoint to develop a diagnosis filter that captures the dynamics signatures of such a multivariate intrusion. To this end, we introduce a dynamic residual generator approach formulated as robust optimization programs in order to detect a class of disruptive multivariate attacks that potentially remain stealthy in view of a static bad data detector. We investigate two possible desired features: (i) a non-zero transient and (ii) a non-zero steady-state behavior of the residual generator in the presence of an attack. In case (i), the problem is reformulated as a finite, but possibly non-convex, optimization program. We further develop a linear programming relaxation that improves the scalability, and as such practicality, of the diagnosis filter design. In case (ii), it turns out that the resulting robust program admits an exact convex reformulation, yielding a Nash equilibrium between the attacker and the residual generator. This assertion has an interesting implication: the proposed approach is not conservative in the sense that the additional knowledge of the worst-case attack does not improve the diagnosis performance. To illustrate our theoretical results, we implement the proposed diagnosis filter to detect multivariate attacks on the system measurements deployed to generate the so-called Automatic Generation Control signals in a three-area IEEE 39-bus system. △ Less

Submitted 22 September, 2019; v1 submitted 19 April, 2019; originally announced April 2019.