-
Safe Exploration Method for Reinforcement Learning under Existence of Disturbance
Authors:
Yoshihiro Okawa,
Tomotake Sasaki,
Hitoshi Yanami,
Toru Namerikawa
Abstract:
Recent rapid developments in reinforcement learning algorithms have been giving us novel possibilities in many fields. However, due to their exploring property, we have to take the risk into consideration when we apply those algorithms to safety-critical problems especially in real environments. In this study, we deal with a safe exploration problem in reinforcement learning under the existence of…
▽ More
Recent rapid developments in reinforcement learning algorithms have been giving us novel possibilities in many fields. However, due to their exploring property, we have to take the risk into consideration when we apply those algorithms to safety-critical problems especially in real environments. In this study, we deal with a safe exploration problem in reinforcement learning under the existence of disturbance. We define the safety during learning as satisfaction of the constraint conditions explicitly defined in terms of the state and propose a safe exploration method that uses partial prior knowledge of a controlled object and disturbance. The proposed method assures the satisfaction of the explicit state constraints with a pre-specified probability even if the controlled object is exposed to a stochastic disturbance following a normal distribution. As theoretical results, we introduce sufficient conditions to construct conservative inputs not containing an exploring aspect used in the proposed method and prove that the safety in the above explained sense is guaranteed with the proposed method. Furthermore, we illustrate the validity and effectiveness of the proposed method through numerical simulations of an inverted pendulum and a four-bar parallel link robot manipulator.
△ Less
Submitted 20 March, 2023; v1 submitted 30 September, 2022;
originally announced September 2022.
-
Improving Rate of Convergence via Gain Adaptation in Multi-Agent Distributed ADMM Framework
Authors:
Towfiq Rahman,
Zhihua Qu,
Toru Namerikawa
Abstract:
In this paper, the alternating direction method of multipliers (ADMM) is investigated for distributed optimization problems in a networked multi-agent system. In particular, a new adaptive-gain ADMM algorithm is derived in a closed form and under the standard convex property in order to greatly speed up convergence of ADMM-based distributed optimization. Using Lyapunov direct approach, the propose…
▽ More
In this paper, the alternating direction method of multipliers (ADMM) is investigated for distributed optimization problems in a networked multi-agent system. In particular, a new adaptive-gain ADMM algorithm is derived in a closed form and under the standard convex property in order to greatly speed up convergence of ADMM-based distributed optimization. Using Lyapunov direct approach, the proposed solution embeds control gains into weighted network matrix among the agents and uses those weights as adaptive penalty gains in the augmented Lagrangian. It is shown that the proposed closed loop gain adaptation scheme significantly improves the convergence time of underlying ADMM optimization. Convergence analysis is provided and simulation results are included to demonstrate the effectiveness of the proposed scheme.
△ Less
Submitted 24 February, 2020;
originally announced February 2020.
-
Data-Driven Wide-Area Control Design of Power System Using the Passivity Shortage Framework
Authors:
Ying Xu,
Zhihua Qu,
Roland Harvey,
Toru Namerikawa
Abstract:
A novel wide-area control design is presented to mitigate inter-area power frequency oscillations. A large-scale power system is decomposed into a network of passivity-short subsystems whose nonlinear interconnections have a state-dependent affine form, and by utilizing the passivity shortage framework a two-level design procedure is developed. At the lower level, any generator control can be view…
▽ More
A novel wide-area control design is presented to mitigate inter-area power frequency oscillations. A large-scale power system is decomposed into a network of passivity-short subsystems whose nonlinear interconnections have a state-dependent affine form, and by utilizing the passivity shortage framework a two-level design procedure is developed. At the lower level, any generator control can be viewed as one that makes the generator passivity-short and $L_2$ stable, and the stability impact of the lower-level control on the overall system can be characterized in terms of two parameters. While the system is nonlinear, the impact parameters can be optimized by solving a data-driven matrix inequality (DMI), and the high-level wide-area control is then designed by solving another Lyapunov matrix inequality in terms of the design parameters. The proposed methodology makes the design modular, and the resulting control is adaptive with respect to operating conditions of the power system. A test system is used to illustrate the proposed design, including DMI and the wide-area control, and simulation results demonstrate effectiveness in dam** out inter-area oscillations.
△ Less
Submitted 18 July, 2019;
originally announced July 2019.
