-
Safe Reinforcement Learning for Strategic Bidding of Virtual Power Plants in Day-Ahead Markets
Authors:
Ognjen Stanojev,
Lesia Mitridati,
Riccardo de Nardis di Prata,
Gabriela Hug
Abstract:
This paper presents a novel safe reinforcement learning algorithm for strategic bidding of Virtual Power Plants (VPPs) in day-ahead electricity markets. The proposed algorithm utilizes the Deep Deterministic Policy Gradient (DDPG) method to learn competitive bidding policies without requiring an accurate market model. Furthermore, to account for the complex internal physical constraints of VPPs we…
▽ More
This paper presents a novel safe reinforcement learning algorithm for strategic bidding of Virtual Power Plants (VPPs) in day-ahead electricity markets. The proposed algorithm utilizes the Deep Deterministic Policy Gradient (DDPG) method to learn competitive bidding policies without requiring an accurate market model. Furthermore, to account for the complex internal physical constraints of VPPs we introduce two enhancements to the DDPG method. Firstly, a projection-based safety shield that restricts the agent's actions to the feasible space defined by the non-linear power flow equations and operating constraints of distributed energy resources is derived. Secondly, a penalty for the shield activation in the reward function that incentivizes the agent to learn a safer policy is introduced. A case study based on the IEEE 13-bus network demonstrates the effectiveness of the proposed approach in enabling the agent to learn a highly competitive, safe strategic policy.
△ Less
Submitted 12 September, 2023; v1 submitted 11 July, 2023;
originally announced July 2023.
-
Differential Privacy for Stackelberg Games
Authors:
Ferdinando Fioretto,
Lesia Mitridati,
Pascal Van Hentenryck
Abstract:
This paper introduces a differentially private (DP) mechanism to protect the information exchanged during the coordination of sequential and interdependent markets. This coordination represents a classic Stackelberg game and relies on the exchange of sensitive information between the system agents. The paper is motivated by the observation that the perturbation introduced by traditional DP mechani…
▽ More
This paper introduces a differentially private (DP) mechanism to protect the information exchanged during the coordination of sequential and interdependent markets. This coordination represents a classic Stackelberg game and relies on the exchange of sensitive information between the system agents. The paper is motivated by the observation that the perturbation introduced by traditional DP mechanisms fundamentally changes the underlying optimization problem and even leads to unsatisfiable instances. To remedy such limitation, the paper introduces the Privacy-Preserving Stackelberg Mechanism (PPSM), a framework that enforces the notions of feasibility and fidelity of the privacy-preserving information to the original problem objective. PPSM complies with the notion of differential privacy and ensures that the outcomes of the privacy-preserving coordination mechanism are close-to-optimality for each agent. Experimental results on several gas and electricity market benchmarks based on a real case study demonstrate the effectiveness of the approach.
△ Less
Submitted 1 February, 2020;
originally announced February 2020.
-
PPSM: A Privacy-Preserving Stackelberg Mechanism: Privacy Guarantees for the Coordination of Sequential Electricity and Gas Markets
Authors:
Ferdinando Fioretto,
Lesia Mitridati,
Pascal Van Hentenryck
Abstract:
This paper introduces a differentially private mechanism to protect the information exchanged during the coordination of the sequential market-clearing of electricity and natural gas systems. The coordination between these sequential and interdependent markets represents a classic Stackelberg game and relies on the exchange of sensitive information between the system agents, including the supply a…
▽ More
This paper introduces a differentially private mechanism to protect the information exchanged during the coordination of the sequential market-clearing of electricity and natural gas systems. The coordination between these sequential and interdependent markets represents a classic Stackelberg game and relies on the exchange of sensitive information between the system agents, including the supply and demand bids in each market or the characteristics of the systems. The paper is motivated by the observation that traditional differential privacy mechanisms are unsuitable for the problem of interest: The perturbation introduced by these mechanisms fundamentally changes the underlying optimization problem and even leads to unsatisfiable instances. To remedy such limitation, the paper introduces the Privacy-Preserving Stackelberg Mechanism (PPSM), a framework that enforces the notions of consistency and fidelity of the privacy-preserving information to the original problem objective. The PPSM has strong properties: It complies with the notion of differential privacy and ensures that the outcomes of the privacy-preserving coordination mechanisms are close-to-optimality for each agent. The fidelity property is analyzed by providing theoretical guarantees on the cost of privacy of PPSM and experimental results on several gas and electricity market benchmarks based on a real case study demonstrate the effectiveness of the approach.
△ Less
Submitted 22 November, 2019;
originally announced November 2019.
