-
Situation-Aware Deep Reinforcement Learning for Autonomous Nonlinear Mobility Control in Cyber-Physical Loitering Munition Systems
Authors:
Hyunsoo Lee,
Soohyun Park,
Won Joon Yun,
Soyi Jung,
Joongheon Kim
Abstract:
According to the rapid development of drone technologies, drones are widely used in many applications including military domains. In this paper, a novel situation-aware DRL- based autonomous nonlinear drone mobility control algorithm in cyber-physical loitering munition applications. On the battlefield, the design of DRL-based autonomous control algorithm is not straightforward because real-world…
▽ More
According to the rapid development of drone technologies, drones are widely used in many applications including military domains. In this paper, a novel situation-aware DRL- based autonomous nonlinear drone mobility control algorithm in cyber-physical loitering munition applications. On the battlefield, the design of DRL-based autonomous control algorithm is not straightforward because real-world data gathering is generally not available. Therefore, the approach in this paper is that cyber-physical virtual environment is constructed with Unity environment. Based on the virtual cyber-physical battlefield scenarios, a DRL-based automated nonlinear drone mobility control algorithm can be designed, evaluated, and visualized. Moreover, many obstacles exist which is harmful for linear trajectory control in real-world battlefield scenarios. Thus, our proposed autonomous nonlinear drone mobility control algorithm utilizes situation-aware components those are implemented with a Raycast function in Unity virtual scenarios. Based on the gathered situation-aware information, the drone can autonomously and nonlinearly adjust its trajectory during flight. Therefore, this approach is obviously beneficial for avoiding obstacles in obstacle-deployed battlefields. Our visualization-based performance evaluation shows that the proposed algorithm is superior from the other linear mobility control algorithms.
△ Less
Submitted 31 December, 2022;
originally announced January 2023.
-
Self-Configurable Stabilized Real-Time Detection Learning for Autonomous Driving Applications
Authors:
Won Joon Yun,
Soohyun Park,
Joongheon Kim,
David Mohaisen
Abstract:
Guaranteeing real-time and accurate object detection simultaneously is paramount in autonomous driving environments. However, the existing object detection neural network systems are characterized by a tradeoff between computation time and accuracy, making it essential to optimize such a tradeoff. Fortunately, in many autonomous driving environments, images come in a continuous form, providing an…
▽ More
Guaranteeing real-time and accurate object detection simultaneously is paramount in autonomous driving environments. However, the existing object detection neural network systems are characterized by a tradeoff between computation time and accuracy, making it essential to optimize such a tradeoff. Fortunately, in many autonomous driving environments, images come in a continuous form, providing an opportunity to use optical flow. In this paper, we improve the performance of an object detection neural network utilizing optical flow estimation. In addition, we propose a Lyapunov optimization framework for time-average performance maximization subject to stability. It adaptively determines whether to use optical flow to suit the dynamic vehicle environment, thereby ensuring the vehicle's queue stability and the time-average maximum performance simultaneously. To verify the key ideas, we conduct numerical experiments with various object detection neural networks and optical flow estimation networks. In addition, we demonstrate the self-configurable stabilized detection with YOLOv3-tiny and FlowNet2-S, which are the real-time object detection network and an optical flow estimation network, respectively. In the demonstration, our proposed framework improves the accuracy by 3.02%, the number of detected objects by 59.6%, and the queue stability for computing capabilities.
△ Less
Submitted 28 September, 2022;
originally announced September 2022.
-
Cooperative Multi-Agent Deep Reinforcement Learning for Reliable Surveillance via Autonomous Multi-UAV Control
Authors:
Won Joon Yun,
Soohyun Park,
Joongheon Kim,
MyungJae Shin,
Soyi Jung,
David A. Mohaisen,
Jae-Hyun Kim
Abstract:
CCTV-based surveillance using unmanned aerial vehicles (UAVs) is considered a key technology for security in smart city environments. This paper creates a case where the UAVs with CCTV-cameras fly over the city area for flexible and reliable surveillance services. UAVs should be deployed to cover a large area while minimize overlap** and shadow areas for a reliable surveillance system. However,…
▽ More
CCTV-based surveillance using unmanned aerial vehicles (UAVs) is considered a key technology for security in smart city environments. This paper creates a case where the UAVs with CCTV-cameras fly over the city area for flexible and reliable surveillance services. UAVs should be deployed to cover a large area while minimize overlap** and shadow areas for a reliable surveillance system. However, the operation of UAVs is subject to high uncertainty, necessitating autonomous recovery systems. This work develops a multi-agent deep reinforcement learning-based management scheme for reliable industry surveillance in smart city applications. The core idea this paper employs is autonomously replenishing the UAV's deficient network requirements with communications. Via intensive simulations, our proposed algorithm outperforms the state-of-the-art algorithms in terms of surveillance coverage, user support capability, and computational costs.
△ Less
Submitted 15 January, 2022;
originally announced January 2022.
-
Quality-Aware Deep Reinforcement Learning for Streaming in Infrastructure-Assisted Connected Vehicles
Authors:
Won Joon Yun,
Dohyun Kwon,
Minseok Choi,
Joongheon Kim,
Guiseppe Caire,
Andreas F. Molisch
Abstract:
This paper proposes a deep reinforcement learning-based video streaming scheme for mobility-aware vehicular networks, e.g., vehicles on the highway. We consider infrastructure-assisted and mmWave-based scenarios in which the macro base station (MBS) cannot directly provide the streaming service to vehicles due to the short range of mmWave beams so that small mmWave base stations (mBSs) along the r…
▽ More
This paper proposes a deep reinforcement learning-based video streaming scheme for mobility-aware vehicular networks, e.g., vehicles on the highway. We consider infrastructure-assisted and mmWave-based scenarios in which the macro base station (MBS) cannot directly provide the streaming service to vehicles due to the short range of mmWave beams so that small mmWave base stations (mBSs) along the road deliver the desired videos to users. For a smoother streaming service, the MBS proactively pushes video chunks to mBSs. This is done to support vehicles that are currently covered and/or will be by each mBS. We formulate the dynamic video delivery scheme that adaptively determines 1) which content, 2) what quality and 3) how many chunks to be proactively delivered from the MBS to mBSs using Markov decision process (MDP). Since it is difficult for the MBS to track all the channel conditions and the network states have extensive dimensions, we adopt the deep deterministic policy gradient (DDPG) algorithm for the DRL-based video delivery scheme. This paper finally shows that the DRL agent learns a streaming policy that pursues high average quality while limiting packet drops, avoiding playback stalls, reducing quality fluctuations and saving backhaul usage.
△ Less
Submitted 12 October, 2021;
originally announced November 2021.
-
Joint Pilot Design and Channel Estimation using Deep Residual Learning for Multi-Cell Massive MIMO under Hardware Impairments
Authors:
Byungju Lim,
Won Joon Yun,
Joongheon Kim,
Young-Chai Ko
Abstract:
In multi-cell massive MIMO systems, channel estimation is deteriorated by pilot contamination and the effects of pilot contamination become more severe due to hardware impairments. In this paper, we propose a joint pilot design and channel estimation based on deep residual learning in order to mitigate the effects of pilot contamination under the consideration of hardware impairments. We first inv…
▽ More
In multi-cell massive MIMO systems, channel estimation is deteriorated by pilot contamination and the effects of pilot contamination become more severe due to hardware impairments. In this paper, we propose a joint pilot design and channel estimation based on deep residual learning in order to mitigate the effects of pilot contamination under the consideration of hardware impairments. We first investigate a conventional linear minimum mean square error (LMMSE) based channel estimator to suppress the interference caused by pilot contamination. After that, a deep learning based pilot design is proposed to minimize the mean square error (MSE) of LMMSE channel estimation, which is utilized to the joint pilot design and channel estimator for transfer learning approach. For the channel estimator, we use a deep residual learning which extracts the features of interference caused by pilot contamination and eliminates them to estimate the channel information. Simulation results demonstrate that the proposed joint pilot design and channel estimator outperforms the conventional approach in multi-cell massive MIMO scenarios. Furthermore, the joint pilot design and channel estimator using transfer learning enhances the estimation performance by reducing the effects of pilot contamination when the prior knowledge of pilot contamination cannot be exploited.
△ Less
Submitted 10 August, 2021;
originally announced August 2021.
