-
MARLP: Time-series Forecasting Control for Agricultural Managed Aquifer Recharge
Authors:
Yuning Chen,
Kang Yang,
Zhiyu An,
Brady Holder,
Luke Paloutzian,
Khaled Bali,
Wan Du
Abstract:
The rapid decline in groundwater around the world poses a significant challenge to sustainable agriculture. To address this issue, agricultural managed aquifer recharge (Ag-MAR) is proposed to recharge the aquifer by artificially flooding agricultural lands using surface water. Ag-MAR requires a carefully selected flooding schedule to avoid affecting the oxygen absorption of crop roots. However, c…
▽ More
The rapid decline in groundwater around the world poses a significant challenge to sustainable agriculture. To address this issue, agricultural managed aquifer recharge (Ag-MAR) is proposed to recharge the aquifer by artificially flooding agricultural lands using surface water. Ag-MAR requires a carefully selected flooding schedule to avoid affecting the oxygen absorption of crop roots. However, current Ag-MAR scheduling does not take into account complex environmental factors such as weather and soil oxygen, resulting in crop damage and insufficient recharging amounts. This paper proposes MARLP, the first end-to-end data-driven control system for Ag-MAR. We first formulate Ag-MAR as an optimization problem. To that end, we analyze four-year in-field datasets, which reveal the multi-periodicity feature of the soil oxygen level trends and the opportunity to use external weather forecasts and flooding proposals as exogenous clues for soil oxygen prediction. Then, we design a two-stage forecasting framework. In the first stage, it extracts both the cross-variate dependency and the periodic patterns from historical data to conduct preliminary forecasting. In the second stage, it uses weather-soil and flooding-soil causality to facilitate an accurate prediction of soil oxygen levels. Finally, we conduct model predictive control (MPC) for Ag-MAR flooding. To address the challenge of large action spaces, we devise a heuristic planning module to reduce the number of flooding proposals to enable the search for optimal solutions. Real-world experiments show that MARLP reduces the oxygen deficit ratio by 86.8% while improving the recharging amount in unit time by 35.8%, compared with the previous four years.
△ Less
Submitted 1 July, 2024;
originally announced July 2024.
-
Differentiable Distributionally Robust Optimization Layers
Authors:
Xutao Ma,
Chao Ning,
Wenli Du
Abstract:
In recent years, there has been a growing research interest in decision-focused learning, which embeds optimization problems as a layer in learning pipelines and demonstrates a superior performance than the prediction-focused approach. However, for distributionally robust optimization (DRO), a popular paradigm for decision-making under uncertainty, it is still unknown how to embed it as a layer, i…
▽ More
In recent years, there has been a growing research interest in decision-focused learning, which embeds optimization problems as a layer in learning pipelines and demonstrates a superior performance than the prediction-focused approach. However, for distributionally robust optimization (DRO), a popular paradigm for decision-making under uncertainty, it is still unknown how to embed it as a layer, i.e., how to differentiate decisions with respect to an ambiguity set. In this paper, we develop such differentiable DRO layers for generic mixed-integer DRO problems with parameterized second-order conic ambiguity sets and discuss its extension to Wasserstein ambiguity sets. To differentiate the mixed-integer decisions, we propose a novel dual-view methodology by handling continuous and discrete parts of decisions via different principles. Specifically, we construct a differentiable energy-based surrogate to implement the dual-view methodology and use importance sampling to estimate its gradient. We further prove that such a surrogate enjoys the asymptotic convergency under regularization. As an application of the proposed differentiable DRO layers, we develop a novel decision-focused learning pipeline for contextual distributionally robust decision-making tasks and compare it with the prediction-focused approach in experiments.
△ Less
Submitted 24 June, 2024;
originally announced June 2024.
-
Go Beyond Black-box Policies: Rethinking the Design of Learning Agent for Interpretable and Verifiable HVAC Control
Authors:
Zhiyu An,
Xianzhong Ding,
Wan Du
Abstract:
Recent research has shown the potential of Model-based Reinforcement Learning (MBRL) to enhance energy efficiency of Heating, Ventilation, and Air Conditioning (HVAC) systems. However, existing methods rely on black-box thermal dynamics models and stochastic optimizers, lacking reliability guarantees and posing risks to occupant health. In this work, we overcome the reliability bottleneck by redes…
▽ More
Recent research has shown the potential of Model-based Reinforcement Learning (MBRL) to enhance energy efficiency of Heating, Ventilation, and Air Conditioning (HVAC) systems. However, existing methods rely on black-box thermal dynamics models and stochastic optimizers, lacking reliability guarantees and posing risks to occupant health. In this work, we overcome the reliability bottleneck by redesigning HVAC controllers using decision trees extracted from existing thermal dynamics models and historical data. Our decision tree-based policies are deterministic, verifiable, interpretable, and more energy-efficient than current MBRL methods. First, we introduce a novel verification criterion for RL agents in HVAC control based on domain knowledge. Second, we develop a policy extraction procedure that produces a verifiable decision tree policy. We found that the high dimensionality of the thermal dynamics model input hinders the efficiency of policy extraction. To tackle the dimensionality challenge, we leverage importance sampling conditioned on historical data distributions, significantly improving policy extraction efficiency. Lastly, we present an offline verification algorithm that guarantees the reliability of a control policy. Extensive experiments show that our method saves 68.4% more energy and increases human comfort gain by 14.8% compared to the state-of-the-art method, in addition to an 1127x reduction in computation overhead. Our code and data are available at https://github.com/ryeii/Veri_HVAC
△ Less
Submitted 29 February, 2024;
originally announced March 2024.
-
Resilient Control of Networked Microgrids using Vertical Federated Reinforcement Learning: Designs and Real-Time Test-Bed Validations
Authors:
Sayak Mukherjee,
Ramij R. Hossain,
Sheik M. Mohiuddin,
Yuan Liu,
Wei Du,
Veronica Adetola,
Rohit A. **siwale,
Qiuhua Huang,
Tianzhixi Yin,
Ankit Singhal
Abstract:
Improving system-level resiliency of networked microgrids is an important aspect with increased population of inverter-based resources (IBRs). This paper (1) presents resilient control design in presence of adversarial cyber-events, and proposes a novel federated reinforcement learning (Fed-RL) approach to tackle (a) model complexities, unknown dynamical behaviors of IBR devices, (b) privacy issue…
▽ More
Improving system-level resiliency of networked microgrids is an important aspect with increased population of inverter-based resources (IBRs). This paper (1) presents resilient control design in presence of adversarial cyber-events, and proposes a novel federated reinforcement learning (Fed-RL) approach to tackle (a) model complexities, unknown dynamical behaviors of IBR devices, (b) privacy issues regarding data sharing in multi-party-owned networked grids, and (2) transfers learned controls from simulation to hardware-in-the-loop test-bed, thereby bridging the gap between simulation and real world. With these multi-prong objectives, first, we formulate a reinforcement learning (RL) training setup generating episodic trajectories with adversaries (attack signal) injected at the primary controllers of the grid forming (GFM) inverters where RL agents (or controllers) are being trained to mitigate the injected attacks. For networked microgrids, the horizontal Fed-RL method involving distinct independent environments is not appropriate, leading us to develop vertical variant Federated Soft Actor-Critic (FedSAC) algorithm to grasp the interconnected dynamics of networked microgrid. Next, utilizing OpenAI Gym interface, we built a custom simulation set-up in GridLAB-D/HELICS co-simulation platform, named Resilient RL Co-simulation (ResRLCoSIM), to train the RL agents with IEEE 123-bus benchmark test systems comprising 3 interconnected microgrids. Finally, the learned policies in simulation world are transferred to the real-time hardware-in-the-loop test-bed set-up developed using high-fidelity Hypersim platform. Experiments show that the simulator-trained RL controllers produce convincing results with the real-time test-bed set-up, validating the minimization of sim-to-real gap.
△ Less
Submitted 20 November, 2023;
originally announced November 2023.
-
Safety in Traffic Management Systems: A Comprehensive Survey
Authors:
Wenlu Du,
Ankan Dash,
**g Li,
Hua Wei,
Guiling Wang
Abstract:
Traffic management systems play a vital role in ensuring safe and efficient transportation on roads. However, the use of advanced technologies in traffic management systems has introduced new safety challenges. Therefore, it is important to ensure the safety of these systems to prevent accidents and minimize their impact on road users. In this survey, we provide a comprehensive review of the liter…
▽ More
Traffic management systems play a vital role in ensuring safe and efficient transportation on roads. However, the use of advanced technologies in traffic management systems has introduced new safety challenges. Therefore, it is important to ensure the safety of these systems to prevent accidents and minimize their impact on road users. In this survey, we provide a comprehensive review of the literature on safety in traffic management systems. Specifically, we discuss the different safety issues that arise in traffic management systems, the current state of research on safety in these systems, and the techniques and methods proposed to ensure the safety of these systems. We also identify the limitations of the existing research and suggest future research directions.
△ Less
Submitted 11 August, 2023;
originally announced August 2023.
-
Optimizing Irrigation Efficiency using Deep Reinforcement Learning in the Field
Authors:
Xianzhong Ding,
Wan Du
Abstract:
Agricultural irrigation is a significant contributor to freshwater consumption. However, the current irrigation systems used in the field are not efficient. They rely mainly on soil moisture sensors and the experience of growers, but do not account for future soil moisture loss. Predicting soil moisture loss is challenging because it is influenced by numerous factors, including soil texture, weath…
▽ More
Agricultural irrigation is a significant contributor to freshwater consumption. However, the current irrigation systems used in the field are not efficient. They rely mainly on soil moisture sensors and the experience of growers, but do not account for future soil moisture loss. Predicting soil moisture loss is challenging because it is influenced by numerous factors, including soil texture, weather conditions, and plant characteristics. This paper proposes a solution to improve irrigation efficiency, which is called DRLIC. DRLIC is a sophisticated irrigation system that uses deep reinforcement learning (DRL) to optimize its performance. The system employs a neural network, known as the DRL control agent, which learns an optimal control policy that considers both the current soil moisture measurement and the future soil moisture loss. We introduce an irrigation reward function that enables our control agent to learn from previous experiences. However, there may be instances where the output of our DRL control agent is unsafe, such as irrigating too much or too little water. To avoid damaging the health of the plants, we implement a safety mechanism that employs a soil moisture predictor to estimate the performance of each action. If the predicted outcome is deemed unsafe, we perform a relatively-conservative action instead. To demonstrate the real-world application of our approach, we developed an irrigation system that comprises sprinklers, sensing and control nodes, and a wireless network. We evaluate the performance of DRLIC by deploying it in a testbed consisting of six almond trees. During a 15-day in-field experiment, we compared the water consumption of DRLIC with a widely-used irrigation scheme. Our results indicate that DRLIC outperformed the traditional irrigation method by achieving a water savings of up to 9.52%.
△ Less
Submitted 3 April, 2023;
originally announced April 2023.
-
A Universal Identity Backdoor Attack against Speaker Verification based on Siamese Network
Authors:
Haodong Zhao,
Wei Du,
Junjie Guo,
Gongshen Liu
Abstract:
Speaker verification has been widely used in many authentication scenarios. However, training models for speaker verification requires large amounts of data and computing power, so users often use untrustworthy third-party data or deploy third-party models directly, which may create security risks. In this paper, we propose a backdoor attack for the above scenario. Specifically, for the Siamese ne…
▽ More
Speaker verification has been widely used in many authentication scenarios. However, training models for speaker verification requires large amounts of data and computing power, so users often use untrustworthy third-party data or deploy third-party models directly, which may create security risks. In this paper, we propose a backdoor attack for the above scenario. Specifically, for the Siamese network in the speaker verification system, we try to implant a universal identity in the model that can simulate any enrolled speaker and pass the verification. So the attacker does not need to know the victim, which makes the attack more flexible and stealthy. In addition, we design and compare three ways of selecting attacker utterances and two ways of poisoned training for the GE2E loss function in different scenarios. The results on the TIMIT and Voxceleb1 datasets show that our approach can achieve a high attack success rate while guaranteeing the normal verification accuracy. Our work reveals the vulnerability of the speaker verification system and provides a new perspective to further improve the robustness of the system.
△ Less
Submitted 28 March, 2023;
originally announced March 2023.
-
Multi-zone HVAC Control with Model-Based Deep Reinforcement Learning
Authors:
Xianzhong Ding,
Alberto Cerpa,
Wan Du
Abstract:
In this paper, we conduct a set of experiments to analyze the limitations of current MBRL-based HVAC control methods, in terms of model uncertainty and controller effectiveness. Using the lessons learned, we develop MB2C, a novel MBRL-based HVAC control system that can achieve high control performance with excellent sample efficiency. MB2C learns the building dynamics by employing an ensemble of e…
▽ More
In this paper, we conduct a set of experiments to analyze the limitations of current MBRL-based HVAC control methods, in terms of model uncertainty and controller effectiveness. Using the lessons learned, we develop MB2C, a novel MBRL-based HVAC control system that can achieve high control performance with excellent sample efficiency. MB2C learns the building dynamics by employing an ensemble of environment-conditioned neural networks. It then applies a new control method, Model Predictive Path Integral (MPPI), for HVAC control. It produces candidate action sequences by using an importance sampling weighted algorithm that scales better to high state and action dimensions of multi-zone buildings. We evaluate MB2C using EnergyPlus simulations in a five-zone office building. The results show that MB2C can achieve 8.23% more energy savings compared to the state-of-the-art MBRL solution while maintaining similar thermal comfort. MB2C can reduce the training data set by an order of magnitude (10.52x) while achieving comparable performance to MFRL approaches.
△ Less
Submitted 1 February, 2023;
originally announced February 2023.
-
Hierarchical Disentangled Representation for Invertible Image Denoising and Beyond
Authors:
Wenchao Du,
Hu Chen,
Yi Zhang,
H. Yang
Abstract:
Image denoising is a typical ill-posed problem due to complex degradation. Leading methods based on normalizing flows have tried to solve this problem with an invertible transformation instead of a deterministic map**. However, the implicit bijective map** is not explored well. Inspired by a latent observation that noise tends to appear in the high-frequency part of the image, we propose a ful…
▽ More
Image denoising is a typical ill-posed problem due to complex degradation. Leading methods based on normalizing flows have tried to solve this problem with an invertible transformation instead of a deterministic map**. However, the implicit bijective map** is not explored well. Inspired by a latent observation that noise tends to appear in the high-frequency part of the image, we propose a fully invertible denoising method that injects the idea of disentangled learning into a general invertible neural network to split noise from the high-frequency part. More specifically, we decompose the noisy image into clean low-frequency and hybrid high-frequency parts with an invertible transformation and then disentangle case-specific noise and high-frequency components in the latent space. In this way, denoising is made tractable by inversely merging noiseless low and high-frequency parts. Furthermore, we construct a flexible hierarchical disentangling framework, which aims to decompose most of the low-frequency image information while disentangling noise from the high-frequency part in a coarse-to-fine manner. Extensive experiments on real image denoising, JPEG compressed artifact removal, and medical low-dose CT image restoration have demonstrated that the proposed method achieves competing performance on both quantitative metrics and visual quality, with significantly less computational cost.
△ Less
Submitted 30 January, 2023;
originally announced January 2023.
-
Exploring Deep Reinforcement Learning for Holistic Smart Building Control
Authors:
Xianzhong Ding,
Alberto Cerpa,
Wan Du
Abstract:
In this paper, we take a holistic approach to deal with the tradeoffs between energy use and comfort in commercial buildings. We developed a system called OCTOPUS, which employs a novel deep reinforcement learning (DRL) framework that uses a data-driven approach to find the optimal control sequences of all building's subsystems, including HVAC, lighting, blind and window systems. The DRL architect…
▽ More
In this paper, we take a holistic approach to deal with the tradeoffs between energy use and comfort in commercial buildings. We developed a system called OCTOPUS, which employs a novel deep reinforcement learning (DRL) framework that uses a data-driven approach to find the optimal control sequences of all building's subsystems, including HVAC, lighting, blind and window systems. The DRL architecture includes a novel reward function that allows the framework to explore the tradeoffs between energy use and users' comfort, while at the same time enabling the solution of the high-dimensional control problem due to the interactions of four different building subsystems. In order to cope with OCTOPUS's data training requirements, we argue that calibrated simulations that match the target building operational points are the vehicle to generate enough data to be able to train our DRL framework to find the control solution for the target building. In our work, we trained OCTOPUS with 10-year weather data and a building model that is implemented in the EnergyPlus building simulator, which was calibrated using data from a real production building. Through extensive simulations, we demonstrate that OCTOPUS can achieve 14.26% and 8.1% energy savings compared with the state-of-the-art rule-based method in a LEED Gold Certified building and the latest DRL-based method available in the literature respectively, while maintaining human comfort within a desired range.
△ Less
Submitted 26 January, 2023;
originally announced January 2023.
-
Transformer and GAN Based Super-Resolution Reconstruction Network for Medical Images
Authors:
Weizhi Du,
Harvery Tian
Abstract:
Because of the necessity to obtain high-quality images with minimal radiation doses, such as in low-field magnetic resonance imaging, super-resolution reconstruction in medical imaging has become more popular (MRI). However, due to the complexity and high aesthetic requirements of medical imaging, image super-resolution reconstruction remains a difficult challenge. In this paper, we offer a deep l…
▽ More
Because of the necessity to obtain high-quality images with minimal radiation doses, such as in low-field magnetic resonance imaging, super-resolution reconstruction in medical imaging has become more popular (MRI). However, due to the complexity and high aesthetic requirements of medical imaging, image super-resolution reconstruction remains a difficult challenge. In this paper, we offer a deep learning-based strategy for reconstructing medical images from low resolutions utilizing Transformer and Generative Adversarial Networks (T-GAN). The integrated system can extract more precise texture information and focus more on important locations through global image matching after successfully inserting Transformer into the generative adversarial network for picture reconstruction. Furthermore, we weighted the combination of content loss, adversarial loss, and adversarial feature loss as the final multi-task loss function during the training of our proposed model T-GAN. In comparison to established measures like PSNR and SSIM, our suggested T-GAN achieves optimal performance and recovers more texture features in super-resolution reconstruction of MRI scanned images of the knees and belly.
△ Less
Submitted 26 December, 2022;
originally announced December 2022.
-
Enhancing Cyber Resilience of Networked Microgrids using Vertical Federated Reinforcement Learning
Authors:
Sayak Mukherjee,
Ramij R. Hossain,
Yuan Liu,
Wei Du,
Veronica Adetola,
Sheik M. Mohiuddin,
Qiuhua Huang,
Tianzhixi Yin,
Ankit Singhal
Abstract:
This paper presents a novel federated reinforcement learning (Fed-RL) methodology to enhance the cyber resiliency of networked microgrids. We formulate a resilient reinforcement learning (RL) training setup which (a) generates episodic trajectories injecting adversarial actions at primary control reference signals of the grid forming (GFM) inverters and (b) trains the RL agents (or controllers) to…
▽ More
This paper presents a novel federated reinforcement learning (Fed-RL) methodology to enhance the cyber resiliency of networked microgrids. We formulate a resilient reinforcement learning (RL) training setup which (a) generates episodic trajectories injecting adversarial actions at primary control reference signals of the grid forming (GFM) inverters and (b) trains the RL agents (or controllers) to alleviate the impact of the injected adversaries. To circumvent data-sharing issues and concerns for proprietary privacy in multi-party-owned networked grids, we bring in the aspects of federated machine learning and propose a novel Fed-RL algorithm to train the RL agents. To this end, the conventional horizontal Fed-RL approaches using decoupled independent environments fail to capture the coupled dynamics in a networked microgrid, which leads us to propose a multi-agent vertically federated variation of actor-critic algorithms, namely federated soft actor-critic (FedSAC) algorithm. We created a customized simulation setup encapsulating microgrid dynamics in the GridLAB-D/HELICS co-simulation platform compatible with the OpenAI Gym interface for training RL agents. Finally, the proposed methodology is validated with numerical examples of modified IEEE 123-bus benchmark test systems consisting of three coupled microgrids.
△ Less
Submitted 17 December, 2022;
originally announced December 2022.
-
Network Topology Inference based on Timing Meta-Data
Authors:
Wenbo Du,
Tao Tan,
Haijun Zhang,
Xianbin Cao,
Gang Yan,
Osvaldo Simeone
Abstract:
Consider a processor having access only to meta-data consisting of the timings of data packets and acknowledgment (ACK) packets from all nodes in a network. The meta-data report the source node of each packet, but not the destination nodes or the contents of the packets. The goal of the processor is to infer the network topology based solely on such information. Prior work leveraged causality metr…
▽ More
Consider a processor having access only to meta-data consisting of the timings of data packets and acknowledgment (ACK) packets from all nodes in a network. The meta-data report the source node of each packet, but not the destination nodes or the contents of the packets. The goal of the processor is to infer the network topology based solely on such information. Prior work leveraged causality metrics to identify which links are active. If the data timings and ACK timings of two nodes -- say node 1 and node 2, respectively -- are causally related, this may be taken as evidence that node 1 is communicating to node 2 (which sends back ACK packets to node 1). This paper starts with the observation that packet losses can weaken the causality relationship between data and ACK timing streams. To obviate this problem, a new Expectation Maximization (EM)-based algorithm is introduced -- EM-causality discovery algorithm (EM-CDA) -- which treats packet losses as latent variables. EM-CDA iterates between the estimation of packet losses and the evaluation of causality metrics. The method is validated through extensive experiments in wireless sensor networks on the NS-3 simulation platform.
△ Less
Submitted 11 October, 2022;
originally announced October 2022.
-
Enhanced exemplar autoencoder with cycle consistency loss in any-to-one voice conversion
Authors:
Weida Liang,
Lantian Li,
Wenqiang Du,
Dong Wang
Abstract:
Recent research showed that an autoencoder trained with speech of a single speaker, called exemplar autoencoder (eAE), can be used for any-to-one voice conversion (VC). Compared to large-scale many-to-many models such as AutoVC, the eAE model is easy and fast in training, and may recover more details of the target speaker.
To ensure VC quality, the latent code should represent and only represent…
▽ More
Recent research showed that an autoencoder trained with speech of a single speaker, called exemplar autoencoder (eAE), can be used for any-to-one voice conversion (VC). Compared to large-scale many-to-many models such as AutoVC, the eAE model is easy and fast in training, and may recover more details of the target speaker.
To ensure VC quality, the latent code should represent and only represent content information. However, this is not easy to attain for eAE as it is unaware of any speaker variation in model training. To tackle the problem, we propose a simple yet effective approach based on a cycle consistency loss. Specifically, we train eAEs of multiple speakers with a shared encoder, and meanwhile encourage the speech reconstructed from any speaker-specific decoder to get a consistent latent code as the original speech when cycled back and encoded again. Experiments conducted on the AISHELL-3 corpus showed that this new approach improved the baseline eAE consistently. The source code and examples are available at the project page: http://project.cslt.org/.
△ Less
Submitted 11 April, 2022; v1 submitted 8 April, 2022;
originally announced April 2022.
-
Unsupervised Learning Based Focal Stack Camera Depth Estimation
Authors:
Zhengyu Huang,
Weizhi Du,
Theodore B. Norris
Abstract:
We propose an unsupervised deep learning based method to estimate depth from focal stack camera images. On the NYU-v2 dataset, our method achieves much better depth estimation accuracy compared to single-image based methods.
We propose an unsupervised deep learning based method to estimate depth from focal stack camera images. On the NYU-v2 dataset, our method achieves much better depth estimation accuracy compared to single-image based methods.
△ Less
Submitted 9 August, 2022; v1 submitted 13 March, 2022;
originally announced March 2022.
-
C-P Map: A Novel Evaluation Toolkit for Speaker Verification
Authors:
Lantian Li,
Di Wang,
Wenqiang Du,
Dong Wang
Abstract:
Evaluation trials are used to probe performance of automatic speaker verification (ASV) systems. In spite of the clear importance and impact, evaluation trials have not been seriously treated in research and engineering practice. This paper firstly presents a theoretical analysis on evaluation trials and highlights potential bias with the most popular cross-pairing approach used in trials design.…
▽ More
Evaluation trials are used to probe performance of automatic speaker verification (ASV) systems. In spite of the clear importance and impact, evaluation trials have not been seriously treated in research and engineering practice. This paper firstly presents a theoretical analysis on evaluation trials and highlights potential bias with the most popular cross-pairing approach used in trials design. To interpret and settle this problem, we define the concept of trial config and C-P map derived from it. The C-P map measures the performance of an ASV system on various trial configs in a 2-dimensional map. On the map, each location represents a particular trial config and its corresponding color represents the system performance. Experiments conducted on representative ASV systems show that the proposed C-P map offers a powerful evaluation toolkit for ASV performance analysis and comparison. The source code for C-P map has been release at https://gitlab.com/csltstu/sunine.
△ Less
Submitted 6 March, 2022;
originally announced March 2022.
-
A Novel Sample-efficient Deep Reinforcement Learning with Episodic Policy Transfer for PID-Based Control in Cardiac Catheterization Robots
Authors:
Olatunji Mumini Omisore,
Toluwanimi Akinyemi,
Wenke Duan,
Wen**g Du,
Lei Wang
Abstract:
Robotic catheterization is typically used for percutaneous coronary intervention procedures nowadays and it involves steering flexible endovascular tools to open up occlusion in the coronaries. In this study, a sample-efficient deep reinforcement learning with episodic policy transfer is, for the first time, used for motion control during robotic catheterization with fully adaptive PID tuning stra…
▽ More
Robotic catheterization is typically used for percutaneous coronary intervention procedures nowadays and it involves steering flexible endovascular tools to open up occlusion in the coronaries. In this study, a sample-efficient deep reinforcement learning with episodic policy transfer is, for the first time, used for motion control during robotic catheterization with fully adaptive PID tuning strategy. The reinforcement model aids the agent to continuously learn from its interactions in its environment and adaptively tune PID control gains for axial navigation of endovascular tool. The model was validated for axial motion control of a robotic system designed for intravascular catheterization. Simulation and experimental trials were done to validate the application of the model, and results obtained shows it could self-tune PID gains appropriately for motion control of a robotic catheter system. Performance comparison with conventional methods in average of 10 trials shows the agent tunes the gain better with error of 0.003 mm. Thus, the proposed model would offer more stable set-point motion control robotic catheterization.
△ Less
Submitted 28 October, 2021;
originally announced October 2021.
-
Explaining the Attention Mechanism of End-to-End Speech Recognition Using Decision Trees
Authors:
Yuanchao Wang,
Wenji Du,
Chenghao Cai,
Yanyan Xu
Abstract:
The attention mechanism has largely improved the performance of end-to-end speech recognition systems. However, the underlying behaviours of attention is not yet clearer. In this study, we use decision trees to explain how the attention mechanism impact itself in speech recognition. The results indicate that attention levels are largely impacted by their previous states rather than the encoder and…
▽ More
The attention mechanism has largely improved the performance of end-to-end speech recognition systems. However, the underlying behaviours of attention is not yet clearer. In this study, we use decision trees to explain how the attention mechanism impact itself in speech recognition. The results indicate that attention levels are largely impacted by their previous states rather than the encoder and decoder patterns. Additionally, the default attention mechanism seems to put more weights on closer states, but behaves poorly on modelling long-term dependencies of attention states.
△ Less
Submitted 7 October, 2021;
originally announced October 2021.
-
Coordinated Frequency and Voltage Regulation of Grid-Following and Grid-Forming Inverters
Authors:
Ankit Singhal,
Thanh Long Vu,
Wei Du
Abstract:
In a purely inverter-based microgrid, both grid-forming (GFM) and grid-following (GFL) inverters will have a crucial role to play in frequency/voltage regulation and maintaining power sharing through their grid support capabilities. Consequently, the coordination between these two technologies becomes increasingly important for optimal system performance. However, the existing work does not consid…
▽ More
In a purely inverter-based microgrid, both grid-forming (GFM) and grid-following (GFL) inverters will have a crucial role to play in frequency/voltage regulation and maintaining power sharing through their grid support capabilities. Consequently, the coordination between these two technologies becomes increasingly important for optimal system performance. However, the existing work does not consider GFL's potential to participate in a secondary control in coordination with GFM, thus not able to utilize the full capability of inverter resources. In this paper, we show that it is possible to fully coordinate the GFL and GFM inverters to achieve accurate power sharing, frequency/voltage regulation, and circulating var mitigation in networked microgrids even without the support of any synchronous generators or the bulk power system. We use the leader-follower consensus framework to develop a GFM-GFL coordination control. The effectiveness of the proposed coordination is verified under different disturbances and communication degradation. In addition, we find that the proposed fully-coordinated secondary control outperforms other approaches such as un-coordinated and partially-coordinated secondary controls, in aspects of load sharing and frequency and voltage regulation. Overall, this study emphasizes the need and benefits of GFL-GFM coordination in microgrids.
△ Less
Submitted 28 October, 2021; v1 submitted 11 December, 2020;
originally announced December 2020.
-
Distributed Small-Signal Stability Conditions for Inverter-Based Unbalanced Microgrids
Authors:
Sai Pushpak Nandanoori,
Soumya Kundu,
Wei Du,
Francis Tuffner,
Kevin Schneider
Abstract:
The proliferation of inverter-based generation and advanced sensing, controls, and communication infrastructure have facilitated the accelerated deployment of microgrids. A coordinated network of microgrids can maintain reliable power delivery to critical facilities during extreme events. Low inertia offered by the power electronics interfaced energy resources however, can present significant chal…
▽ More
The proliferation of inverter-based generation and advanced sensing, controls, and communication infrastructure have facilitated the accelerated deployment of microgrids. A coordinated network of microgrids can maintain reliable power delivery to critical facilities during extreme events. Low inertia offered by the power electronics interfaced energy resources however, can present significant challenges to ensuring stable operation of the microgrids. In this work, distributed small-signal stability conditions for inverter-based microgrids are developed that involve the droop controller parameters and the network parameters such as line impedances, loads, etc. The distributed closed-form parametric stability conditions derived in this paper can be verified in a computationally efficient manner, facilitating the reliable design and operations of networks of microgrids. Dynamic phasor models have been used to capture the effects of electromagnetic transients. Numerical results are presented, along with PSCAD simulations, to validate the analytical stability conditions. Effects of design choices, such as the conductor types, and inverter sizes, on the small-signal stability of inverter-based microgrids are investigated to identify interpretable stable or unstable region estimates.
△ Less
Submitted 17 March, 2020;
originally announced March 2020.
-
Exploration of Surgeons' Natural Skills for Robotic Catheterization
Authors:
Olatunji Mumini Omisore,
Wen**g Du,
Tao Zhou,
Shipeng Han,
Kamen Ivanov,
Yousef Al-Handarish,
Lei Wang
Abstract:
Despite having the robotic catheter systems which have recently emerged as safe way of performing cardiovascular interventions, a number of important challenges are yet to be investigated. One of them is exploration of surgeons' natural skills during vascular catheterization with robotic systems. In this study, surgeons' natural hand motions were investigated for identification of four basic movem…
▽ More
Despite having the robotic catheter systems which have recently emerged as safe way of performing cardiovascular interventions, a number of important challenges are yet to be investigated. One of them is exploration of surgeons' natural skills during vascular catheterization with robotic systems. In this study, surgeons' natural hand motions were investigated for identification of four basic movements used for intravascular catheterization. Controlled experiment was setup to acquire surface electromyography (sEMG) signals from six muscles that are innervated when a subject with catheterization skills made the four movements in open settings. k-means and k-NN models were implemented over average EMG and root means square features to uniquely identify the movements. The result shows great potentials of sEMG analysis towards designing intelligent cyborg control for safe and efficient robotic catheterization.
△ Less
Submitted 5 March, 2020;
originally announced March 2020.
-
A Survey on LoRa Networking: Research Problems, Current Solutions and Open Issues
Authors:
Jothi Prasanna Shanmuga Sundaram,
Wan Du,
Zhiwei Zhao
Abstract:
Wireless networks have been widely deployed for many Internet-of-Things (IoT) applications, like smart cities and precision agriculture. Low Power Wide Area Networking (LPWAN) is an emerging IoT networking paradigm to meet three key requirements of IoT applications, i.e., low cost, large scale deployment and high energy efficiency. Among all available LPWAN technologies, LoRa networking has attrac…
▽ More
Wireless networks have been widely deployed for many Internet-of-Things (IoT) applications, like smart cities and precision agriculture. Low Power Wide Area Networking (LPWAN) is an emerging IoT networking paradigm to meet three key requirements of IoT applications, i.e., low cost, large scale deployment and high energy efficiency. Among all available LPWAN technologies, LoRa networking has attracted much attention from both academia and industry, since it specifies an open standard and allows us to build autonomous LPWAN networks without any third-party infrastructure. Many LoRa networks have been developed recently, e.g., managing solar plants in Carson City, Nevada, USA and power monitoring in Lyon and Grenoble, France. However, there are still many research challenges to develop practical LoRa networks, e.g., link coordination, resource allocation, reliable transmissions and security. This article provides a comprehensive survey on LoRa networks, including the technical challenges of deploying LoRa networks and recent solutions. Based on our detailed analysis of current solutions, some open issues of LoRa networking are discussed. The goal of this survey paper is to inspire more works on improving the performance of LoRa networks and enabling more practical deployments.
△ Less
Submitted 23 August, 2019;
originally announced August 2019.
-
Retrieving Similar Trajectories from Cellular Data at City Scale
Authors:
Zhihao Shen,
Wan Du,
Xi Zhao,
Jianhua Zou
Abstract:
Retrieving similar trajectories from a large trajectory dataset is important for a variety of applications, like transportation planning and mobility analysis. Unlike previous works based on fine-grained GPS trajectories, this paper investigates the feasibility of identifying similar trajectories from cellular data observed by mobile infrastructure, which provide more comprehensive coverage. To ha…
▽ More
Retrieving similar trajectories from a large trajectory dataset is important for a variety of applications, like transportation planning and mobility analysis. Unlike previous works based on fine-grained GPS trajectories, this paper investigates the feasibility of identifying similar trajectories from cellular data observed by mobile infrastructure, which provide more comprehensive coverage. To handle the large localization errors and low sample rates of cellular data, we develop a holistic system, cellSim, which seamlessly integrates map matching and similar trajectory search. A set of map matching techniques are proposed to transform cell tower sequences into moving trajectories on a road map by considering the unique features of cellular data, like the dynamic density of cell towers and bidirectional roads. To further improve the accuracy of similarity search, map matching outputs M trajectory candidates of different confidence, and a new similarity measure scheme is developed to process the map matching results. Meanwhile, M is dynamically adapted to maintain a low false positive rate of the similarity search, and two pruning schemes are proposed to minimize the computation overhead. Extensive experiments on a large-scale dataset and real-world trajectories of 1701 km reveal that cellSim provides high accuracy (precision 62.4% and recall of 89.8%).
△ Less
Submitted 30 October, 2019; v1 submitted 19 July, 2019;
originally announced July 2019.
-
Differential Evolution with Event-Triggered Impulsive Control
Authors:
Wei Du,
Sunney Yung Sun Leung,
Yang Tang,
Athanasios V. Vasilakos
Abstract:
Differential evolution (DE) is a simple but powerful evolutionary algorithm, which has been widely and successfully used in various areas. In this paper, an event-triggered impulsive control scheme (ETI) is introduced to improve the performance of DE. Impulsive control, the concept of which derives from control theory, aims at regulating the states of a network by instantly adjusting the states of…
▽ More
Differential evolution (DE) is a simple but powerful evolutionary algorithm, which has been widely and successfully used in various areas. In this paper, an event-triggered impulsive control scheme (ETI) is introduced to improve the performance of DE. Impulsive control, the concept of which derives from control theory, aims at regulating the states of a network by instantly adjusting the states of a fraction of nodes at certain instants, and these instants are determined by event-triggered mechanism (ETM). By introducing impulsive control and ETM into DE, we hope to change the search performance of the population in a positive way after revising the positions of some individuals at certain moments. At the end of each generation, the impulsive control operation is triggered when the update rate of the population declines or equals to zero. In detail, inspired by the concepts of impulsive control, two types of impulses are presented within the framework of DE in this paper: stabilizing impulses and destabilizing impulses. Stabilizing impulses help the individuals with lower rankings instantly move to a desired state determined by the individuals with better fitness values. Destabilizing impulses randomly alter the positions of inferior individuals within the range of the current population. By means of intelligently modifying the positions of a part of individuals with these two kinds of impulses, both exploitation and exploration abilities of the whole population can be meliorated. In addition, the proposed ETI is flexible to be incorporated into several state-of-the-art DE variants. Experimental results over the CEC 2014 benchmark functions exhibit that the developed scheme is simple yet effective, which significantly improves the performance of the considered DE algorithms.
△ Less
Submitted 24 December, 2015; v1 submitted 16 December, 2015;
originally announced December 2015.
