-
Resilient Path Planning for UAVs in Data Collection under Adversarial Attacks
Authors:
Xueyuan Wang,
M. Cenk Gursoy
Abstract:
In this paper, we investigate jamming-resilient UAV path planning strategies for data collection in Internet of Things (IoT) networks, in which the typical UAV can learn the optimal trajectory to elude such jamming attacks. Specifically, the typical UAV is required to collect data from multiple distributed IoT nodes under collision avoidance, mission completion deadline, and kinematic constraints…
▽ More
In this paper, we investigate jamming-resilient UAV path planning strategies for data collection in Internet of Things (IoT) networks, in which the typical UAV can learn the optimal trajectory to elude such jamming attacks. Specifically, the typical UAV is required to collect data from multiple distributed IoT nodes under collision avoidance, mission completion deadline, and kinematic constraints in the presence of jamming attacks. We first design a fixed ground jammer with continuous jamming attack and periodical jamming attack strategies to jam the link between the typical UAV and IoT nodes. Defensive strategies involving a reinforcement learning (RL) based virtual jammer and the adoption of higher SINR thresholds are proposed to counteract against such attacks. Secondly, we design an intelligent UAV jammer, which utilizes the RL algorithm to choose actions based on its observation. Then, an intelligent UAV anti-jamming strategy is constructed to deal with such attacks, and the optimal trajectory of the typical UAV is obtained via dueling double deep Q-network (D3QN). Simulation results show that both non-intelligent and intelligent jamming attacks have significant influence on the UAV's performance, and the proposed defense strategies can recover the performance close to that in no-jammer scenarios.
△ Less
Submitted 11 December, 2023;
originally announced January 2024.
-
Robust and Decentralized Reinforcement Learning for UAV Path Planning in IoT Networks
Authors:
Xueyuan Wang,
M. Cenk Gursoy
Abstract:
Unmanned aerial vehicle (UAV)-based networks and Internet of Things (IoT) are being considered as integral components of current and next-generation wireless networks.
In particular, UAVs can provide IoT devices with seamless connectivity and high coverage and this can be accomplished with effective UAV path planning.
In this article, we study robust and decentralized UAV path planning for dat…
▽ More
Unmanned aerial vehicle (UAV)-based networks and Internet of Things (IoT) are being considered as integral components of current and next-generation wireless networks.
In particular, UAVs can provide IoT devices with seamless connectivity and high coverage and this can be accomplished with effective UAV path planning.
In this article, we study robust and decentralized UAV path planning for data collection in IoT networks in the presence of other noncooperative UAVs and adversarial jamming attacks. We address three different practical scenarios, including single UAV path planning, UAV swarm path planning, and single UAV path planning in the presence of an intelligent mobile UAV jammer. We advocate a reinforcement learning framework for UAV path planning in these three scenarios under practical constraints. The simulation results demonstrate that with learning-based path planning, the UAVs can complete their missions with high success rates and data collection rates. In addition, the UAVs can adapt and execute different trajectories as a defensive measure against the intelligent jammer.
△ Less
Submitted 11 December, 2023;
originally announced December 2023.
-
Learning-Based UAV Path Planning for Data Collection with Integrated Collision Avoidance
Authors:
Xueyuan Wang,
M. Cenk Gursoy,
Tugba Erpek,
Yalin E. Sagduyu
Abstract:
Unmanned aerial vehicles (UAVs) are expected to be an integral part of wireless networks, and determining collision-free trajectory in multi-UAV non-cooperative scenarios while collecting data from distributed Internet of Things (IoT) nodes is a challenging task. In this paper, we consider a path planning optimization problem to maximize the collected data from multiple IoT nodes under realistic c…
▽ More
Unmanned aerial vehicles (UAVs) are expected to be an integral part of wireless networks, and determining collision-free trajectory in multi-UAV non-cooperative scenarios while collecting data from distributed Internet of Things (IoT) nodes is a challenging task. In this paper, we consider a path planning optimization problem to maximize the collected data from multiple IoT nodes under realistic constraints. The considered multi-UAV non-cooperative scenarios involve random number of other UAVs in addition to the typical UAV, and UAVs do not communicate or share information among each other. We translate the problem into a Markov decision process (MDP) with parameterized states, permissible actions, and detailed reward functions. Dueling double deep Q-network (D3QN) is proposed to learn the decision making policy for the typical UAV, without any prior knowledge of the environment (e.g., channel propagation model and locations of the obstacles) and other UAVs (e.g., their missions, movements, and policies). The proposed algorithm can adapt to various missions in various scenarios, e.g., different numbers and positions of IoT nodes, different amount of data to be collected, and different numbers and positions of other UAVs.
Numerical results demonstrate that real-time navigation can be efficiently performed with high success rate, high data collection rate, and low collision rate.
△ Less
Submitted 11 December, 2023;
originally announced December 2023.
-
Anomaly Detection via Learning-Based Sequential Controlled Sensing
Authors:
Geethu Joseph,
Chen Zhong,
M. Cenk Gursoy,
Senem Velipasalar,
Pramod K. Varshney
Abstract:
In this paper, we address the problem of detecting anomalies among a given set of binary processes via learning-based controlled sensing. Each process is parameterized by a binary random variable indicating whether the process is anomalous. To identify the anomalies, the decision-making agent is allowed to observe a subset of the processes at each time instant. Also, probing each process has an as…
▽ More
In this paper, we address the problem of detecting anomalies among a given set of binary processes via learning-based controlled sensing. Each process is parameterized by a binary random variable indicating whether the process is anomalous. To identify the anomalies, the decision-making agent is allowed to observe a subset of the processes at each time instant. Also, probing each process has an associated cost. Our objective is to design a sequential selection policy that dynamically determines which processes to observe at each time with the goal to minimize the delay in making the decision and the total sensing cost. We cast this problem as a sequential hypothesis testing problem within the framework of Markov decision processes. This formulation utilizes both a Bayesian log-likelihood ratio-based reward and an entropy-based reward. The problem is then solved using two approaches: 1) a deep reinforcement learning-based approach where we design both deep Q-learning and policy gradient actor-critic algorithms; and 2) a deep active inference-based approach. Using numerical experiments, we demonstrate the efficacy of our algorithms and show that our algorithms adapt to any unknown statistical dependence pattern of the processes.
△ Less
Submitted 30 November, 2023;
originally announced December 2023.
-
Dynamic Channel Access via Meta-Reinforcement Learning
Authors:
Ziyang Lu,
M. Cenk Gursoy
Abstract:
In this paper, we address the channel access problem in a dynamic wireless environment via meta-reinforcement learning. Spectrum is a scarce resource in wireless communications, especially with the dramatic increase in the number of devices in networks. Recently, inspired by the success of deep reinforcement learning (DRL), extensive studies have been conducted in addressing wireless resource allo…
▽ More
In this paper, we address the channel access problem in a dynamic wireless environment via meta-reinforcement learning. Spectrum is a scarce resource in wireless communications, especially with the dramatic increase in the number of devices in networks. Recently, inspired by the success of deep reinforcement learning (DRL), extensive studies have been conducted in addressing wireless resource allocation problems via DRL. However, training DRL algorithms usually requires a massive amount of data collected from the environment for each specific task and the well-trained model may fail if there is a small variation in the environment. In this work, in order to address these challenges, we propose a meta-DRL framework that incorporates the method of Model-Agnostic Meta-Learning (MAML). In the proposed framework, we train a common initialization for similar channel selection tasks. From the initialization, we show that only a few gradient descents are required for adapting to different tasks drawn from the same distribution. We demonstrate the performance improvements via simulation results.
△ Less
Submitted 24 December, 2021;
originally announced January 2022.
-
Temporal Detection of Anomalies via Actor-Critic Based Controlled Sensing
Authors:
Geethu Joseph,
M. Cenk Gursoy,
Pramod K. Varshney
Abstract:
We address the problem of monitoring a set of binary stochastic processes and generating an alert when the number of anomalies among them exceeds a threshold. For this, the decision-maker selects and probes a subset of the processes to obtain noisy estimates of their states (normal or anomalous). Based on the received observations, the decisionmaker first determines whether to declare that the num…
▽ More
We address the problem of monitoring a set of binary stochastic processes and generating an alert when the number of anomalies among them exceeds a threshold. For this, the decision-maker selects and probes a subset of the processes to obtain noisy estimates of their states (normal or anomalous). Based on the received observations, the decisionmaker first determines whether to declare that the number of anomalies has exceeded the threshold or to continue taking observations. When the decision is to continue, it then decides whether to collect observations at the next time instant or defer it to a later time. If it chooses to collect observations, it further determines the subset of processes to be probed. To devise this three-step sequential decision-making process, we use a Bayesian formulation wherein we learn the posterior probability on the states of the processes. Using the posterior probability, we construct a Markov decision process and solve it using deep actor-critic reinforcement learning. Via numerical experiments, we demonstrate the superior performance of our algorithm compared to the traditional model-based algorithms.
△ Less
Submitted 16 June, 2023; v1 submitted 3 January, 2022;
originally announced January 2022.
-
Dynamic Channel Access and Power Control in Wireless Interference Networks via Multi-Agent Deep Reinforcement Learning
Authors:
Ziyang Lu,
Chen Zhong,
M. Cenk Gursoy
Abstract:
Due to the scarcity in the wireless spectrum and limited energy resources especially in mobile applications, efficient resource allocation strategies are critical in wireless networks. Motivated by the recent advances in deep reinforcement learning (DRL), we address multi-agent DRL-based joint dynamic channel access and power control in a wireless interference network. We first propose a multi-age…
▽ More
Due to the scarcity in the wireless spectrum and limited energy resources especially in mobile applications, efficient resource allocation strategies are critical in wireless networks. Motivated by the recent advances in deep reinforcement learning (DRL), we address multi-agent DRL-based joint dynamic channel access and power control in a wireless interference network. We first propose a multi-agent DRL algorithm with centralized training (DRL-CT) to tackle the joint resource allocation problem. In this case, the training is performed at the central unit (CU) and after training, the users make autonomous decisions on their transmission strategies with only local information. We demonstrate that with limited information exchange and faster convergence, DRL-CT algorithm can achieve 90% of the performance achieved by the combination of weighted minimum mean square error (WMMSE) algorithm for power control and exhaustive search for dynamic channel access. In the second part of this paper, we consider distributed multi-agent DRL scenario in which each user conducts its own training and makes its decisions individually, acting as a DRL agent. Finally, as a compromise between centralized and fully distributed scenarios, we consider federated DRL (FDRL) to approach the performance of DRL-CT with the use of a central unit in training while limiting the information exchange and preserving privacy of the users in the wireless system. Via simulation results, we show that proposed learning frameworks lead to efficient adaptive channel access and power control policies in dynamic environments.
△ Less
Submitted 24 December, 2021;
originally announced December 2021.
-
Joint Activity Detection and Channel Estimation in Cell-Free Massive MIMO Networks with Massive Connectivity
Authors:
Mangqing Guo,
M. Cenk Gursoy
Abstract:
Cell-free massive MIMO is one of the key technologies for future wireless communications, in which users are simultaneously and jointly served by all access points (APs). In this paper, we investigate the minimum mean square error (MMSE) estimation of effective channel coefficients in cell-free massive MIMO systems with massive connectivity. To facilitate the theoretical analysis, only single meas…
▽ More
Cell-free massive MIMO is one of the key technologies for future wireless communications, in which users are simultaneously and jointly served by all access points (APs). In this paper, we investigate the minimum mean square error (MMSE) estimation of effective channel coefficients in cell-free massive MIMO systems with massive connectivity. To facilitate the theoretical analysis, only single measurement vector (SMV) based MMSE estimation is considered in this paper, i.e., the MMSE estimation is performed based on the received pilot signals at each AP separately. Inspired by the decoupling principle of replica symmetric postulated MMSE estimation of sparse signal vectors with independent and identically distributed (i.i.d.) non-zero components, we develop the corresponding decoupling principle for the SMV based MMSE estimation of sparse signal vectors with independent and non-identically distributed (i.n.i.d.) non-zero components, which plays a key role in the theoretical analysis of SMV based MMSE estimation of the effective channel coefficients in cell-free massive MIMO systems with massive connectivity. Subsequently, based on the obtained decoupling principle of MMSE estimation, likelihood ratio test and the optimal fusion rule, we perform user activity detection based on the received pilot signals at only one AP, or cooperation among the entire set of APs for centralized or distributed detection. Via theoretical analysis, we show that the error probabilities of both centralized and distributed detection tend to zero when the number of APs tends to infinity while the asymptotic ratio between the number of users and pilots is kept constant. We also investigate the asymptotic behavior of oracle estimation in cell-free massive MIMO systems with massive connectivity via random matrix theory.
△ Less
Submitted 24 December, 2021;
originally announced December 2021.
-
A Predictive Online Transient Stability Assessment with Hierarchical Generative Adversarial Networks
Authors:
Rui Ma,
Sara Eftekharnejad,
Chen Zhong,
Mustafa Cenk Gursoy
Abstract:
Online transient stability assessment (TSA) is essential for secure and stable power system operations. The growing number of Phasor Measurement Units (PMUs) brings about massive sources of data that can enhance online TSA. However, conventional data-driven methods require large amounts of transient data to correctly assess the transient stability state of a system. In this paper, a new data-drive…
▽ More
Online transient stability assessment (TSA) is essential for secure and stable power system operations. The growing number of Phasor Measurement Units (PMUs) brings about massive sources of data that can enhance online TSA. However, conventional data-driven methods require large amounts of transient data to correctly assess the transient stability state of a system. In this paper, a new data-driven TSA approach is developed for TSA with fewer data compared to the conventional methods. The data reduction is enabled by learning the dynamic behaviors of the historical transient data using generative and adversarial networks (GAN). This knowledge is used online to predict the voltage time series data after a transient event. A classifier embedded in the generative network deploys the predicted post-contingency data to determine the stability of the system following a fault. The developed GAN-based TSA approach preserves the spatial and temporal correlations that exist in multivariate PMU time series data. Hence, in comparison with the state-of-the-art TSA methods, it achieves a higher assessment accuracy using only one sample of the measured data and a shorter response time. Case studies conducted on the IEEE 118-bus system demonstrate the superior performance of the GAN-based method compared to the conventional data-driven techniques.
△ Less
Submitted 14 December, 2021;
originally announced December 2021.
-
Scalable and Decentralized Algorithms for Anomaly Detection via Learning-Based Controlled Sensing
Authors:
Geethu Joseph,
Chen Zhong,
M. Cenk Gursoy,
Senem Velipasalar,
Pramod K. Varshney
Abstract:
We address the problem of sequentially selecting and observing processes from a given set to find the anomalies among them. The decision-maker observes a subset of the processes at any given time instant and obtains a noisy binary indicator of whether or not the corresponding process is anomalous. In this setting, we develop an anomaly detection algorithm that chooses the processes to be observed…
▽ More
We address the problem of sequentially selecting and observing processes from a given set to find the anomalies among them. The decision-maker observes a subset of the processes at any given time instant and obtains a noisy binary indicator of whether or not the corresponding process is anomalous. In this setting, we develop an anomaly detection algorithm that chooses the processes to be observed at a given time instant, decides when to stop taking observations, and declares the decision on anomalous processes. The objective of the detection algorithm is to identify the anomalies with an accuracy exceeding the desired value while minimizing the delay in decision making. We devise a centralized algorithm where the processes are jointly selected by a common agent as well as a decentralized algorithm where the decision of whether to select a process is made independently for each process. Our algorithms rely on a Markov decision process defined using the marginal probability of each process being normal or anomalous, conditioned on the observations. We implement the detection algorithms using the deep actor-critic reinforcement learning framework. Unlike prior work on this topic that has exponential complexity in the number of processes, our algorithms have computational and memory requirements that are both polynomial in the number of processes. We demonstrate the efficacy of these algorithms using numerical experiments by comparing them with state-of-the-art methods.
△ Less
Submitted 8 December, 2021;
originally announced December 2021.
-
Higher Order Derivative-Based Receiver Pre-processing for Molecular Communications
Authors:
Mustafa Can Gursoy,
Urbashi Mitra
Abstract:
While molecular communication via diffusion experiences significant inter-symbol interference (ISI), recent work suggests that ISI can be mitigated via time differentiation pre-processing which achieves pulse narrowing. Herein, the approach is generalized to higher order differentiation. The fundamental trade-off between ISI mitigation and noise amplification is characterized, showing the existenc…
▽ More
While molecular communication via diffusion experiences significant inter-symbol interference (ISI), recent work suggests that ISI can be mitigated via time differentiation pre-processing which achieves pulse narrowing. Herein, the approach is generalized to higher order differentiation. The fundamental trade-off between ISI mitigation and noise amplification is characterized, showing the existence of an optimal derivative order that minimizes the bit error rate (BER). Theoretical analyses of the BER and a signal-to-interference-plus-noise ratio are provided, the derivative order optimization problem is posed and solved for threshold-based detectors. For more complex detectors which exploit a window memory, it is shown that derivative pre-processing can strongly reduce the size of the needed window. Extensive numerical results confirm the accuracy of theoretical derivations, the gains in performance via derivative pre-processing over other methods and the impact of the optimal derivative order. Derivative pre-processing offers a low complexity/high-performance method for reducing ISI at the expense of increased transmission power to reduce noise amplification.
△ Less
Submitted 17 August, 2021;
originally announced August 2021.
-
A Scalable Algorithm for Anomaly Detection via Learning-Based Controlled Sensing
Authors:
Geethu Joseph,
M. Cenk Gursoy,
Pramod K. Varshney
Abstract:
We address the problem of sequentially selecting and observing processes from a given set to find the anomalies among them. The decision-maker observes one process at a time and obtains a noisy binary indicator of whether or not the corresponding process is anomalous. In this setting, we develop an anomaly detection algorithm that chooses the process to be observed at a given time instant, decides…
▽ More
We address the problem of sequentially selecting and observing processes from a given set to find the anomalies among them. The decision-maker observes one process at a time and obtains a noisy binary indicator of whether or not the corresponding process is anomalous. In this setting, we develop an anomaly detection algorithm that chooses the process to be observed at a given time instant, decides when to stop taking observations, and makes a decision regarding the anomalous processes. The objective of the detection algorithm is to arrive at a decision with an accuracy exceeding a desired value while minimizing the delay in decision making. Our algorithm relies on a Markov decision process defined using the marginal probability of each process being normal or anomalous, conditioned on the observations. We implement the detection algorithm using the deep actor-critic reinforcement learning framework. Unlike prior work on this topic that has exponential complexity in the number of processes, our algorithm has computational and memory requirements that are both polynomial in the number of processes. We demonstrate the efficacy of our algorithm using numerical experiments by comparing it with the state-of-the-art methods.
△ Less
Submitted 12 May, 2021;
originally announced May 2021.
-
Anomaly Detection via Controlled Sensing and Deep Active Inference
Authors:
Geethu Joseph,
Chen Zhong,
M. Cenk Gursoy,
Senem Velipasalar,
Pramod K. Varshney
Abstract:
In this paper, we address the anomaly detection problem where the objective is to find the anomalous processes among a given set of processes. To this end, the decision-making agent probes a subset of processes at every time instant and obtains a potentially erroneous estimate of the binary variable which indicates whether or not the corresponding process is anomalous. The agent continues to probe…
▽ More
In this paper, we address the anomaly detection problem where the objective is to find the anomalous processes among a given set of processes. To this end, the decision-making agent probes a subset of processes at every time instant and obtains a potentially erroneous estimate of the binary variable which indicates whether or not the corresponding process is anomalous. The agent continues to probe the processes until it obtains a sufficient number of measurements to reliably identify the anomalous processes. In this context, we develop a sequential selection algorithm that decides which processes to be probed at every instant to detect the anomalies with an accuracy exceeding a desired value while minimizing the delay in making the decision and the total number of measurements taken. Our algorithm is based on active inference which is a general framework to make sequential decisions in order to maximize the notion of free energy. We define the free energy using the objectives of the selection policy and implement the active inference framework using a deep neural network approximation. Using numerical experiments, we compare our algorithm with the state-of-the-art method based on deep actor-critic reinforcement learning and demonstrate the superior performance of our algorithm.
△ Less
Submitted 12 May, 2021;
originally announced May 2021.
-
Uplink Coverage in Heterogeneous mmWave Cellular Networks with Clustered Users
Authors:
Xueyuan Wang,
M. Cenk Gursoy
Abstract:
A K-tier heterogeneous mmWave uplink cellular network with clustered user equipments (UEs) is considered in this paper. In particular, UEs are assumed to be clustered around small-cell base stations (BSs) according to a Gaussian distribution, leading to the Thomas cluster process based modeling. Specific and practical line-of-sight (LOS) and non-line-of-sight (NLOS) models are adopted with differe…
▽ More
A K-tier heterogeneous mmWave uplink cellular network with clustered user equipments (UEs) is considered in this paper. In particular, UEs are assumed to be clustered around small-cell base stations (BSs) according to a Gaussian distribution, leading to the Thomas cluster process based modeling. Specific and practical line-of-sight (LOS) and non-line-of-sight (NLOS) models are adopted with different parameters for different tiers. The probability density functions (PDFs) and complementary cumulative distribution functions (CCDFs) of different distances from UEs to BSs are characterized. Coupled association strategy and largest long-term averaged biased received power criterion are considered, and general expressions for association probabilities are provided. Following the identification of the association probabilities, the Laplace transforms of the inter-cell interference and the intra-cluster interference are characterized. Using tools from stochastic geometry, general expressions of the SINR coverage probability are provided. As extensions, fractional power control is incorporated into the analysis, tractable closed-form expressions are provided for special cases, and average ergodic spectral efficiency is analyzed. Via numerical and simulation results, analytical characterizations are confirmed and the impact of key system and network parameters on the performance is identified.
△ Less
Submitted 15 April, 2021; v1 submitted 3 April, 2021;
originally announced April 2021.
-
Towards High Data-Rate Diffusive Molecular Communications: Performance Enhancement Strategies
Authors:
Mustafa Can Gursoy,
Masoumeh Nasiri-Kenari,
Urbashi Mitra
Abstract:
Diffusive molecular communications (DiMC) have recently gained attention as a candidate for nano- to micro- and macro-scale communications due to its simplicity and energy efficiency. As signal propagation is solely enabled by Brownian motion mechanics, DiMC faces severe inter-symbol interference (ISI), which limits reliable and high data-rate communications. Herein, recent literature on DiMC perf…
▽ More
Diffusive molecular communications (DiMC) have recently gained attention as a candidate for nano- to micro- and macro-scale communications due to its simplicity and energy efficiency. As signal propagation is solely enabled by Brownian motion mechanics, DiMC faces severe inter-symbol interference (ISI), which limits reliable and high data-rate communications. Herein, recent literature on DiMC performance enhancement strategies is surveyed; key research directions are identified. Signaling design and associated design constraints are presented. Classical and novel transceiver designs are reviewed with an emphasis on methods for ISI mitigation and performance-complexity tradeoffs. Key parameter estimation strategies such as synchronization and channel estimation are considered in conjunction with asynchronous and timing error robust receiver methods. Finally, source and channel coding in the context of DiMC is presented.
△ Less
Submitted 8 January, 2021;
originally announced January 2021.
-
Multi-Agent Double Deep Q-Learning for Beamforming in mmWave MIMO Networks
Authors:
Xueyuan Wang,
M. Cenk Gursoy
Abstract:
Beamforming is one of the key techniques in millimeter wave (mmWave) multi-input multi-output (MIMO) communications. Designing appropriate beamforming not only improves the quality and strength of the received signal, but also can help reduce the interference, consequently enhancing the data rate. In this paper, we propose a distributed multi-agent double deep Q-learning algorithm for beamforming…
▽ More
Beamforming is one of the key techniques in millimeter wave (mmWave) multi-input multi-output (MIMO) communications. Designing appropriate beamforming not only improves the quality and strength of the received signal, but also can help reduce the interference, consequently enhancing the data rate. In this paper, we propose a distributed multi-agent double deep Q-learning algorithm for beamforming in mmWave MIMO networks, where multiple base stations (BSs) can automatically and dynamically adjust their beams to serve multiple highly-mobile user equipments (UEs). In the analysis, largest received power association criterion is considered for UEs, and a realistic channel model is taken into account. Simulation results demonstrate that the proposed learning-based algorithm can achieve comparable performance with respect to exhaustive search while operating at much lower complexity.
△ Less
Submitted 13 August, 2020;
originally announced August 2020.
-
Energy Harvesting in Unmanned Aerial Vehicle Networks with 3D Antenna Radiation Patterns
Authors:
Esma Turgut,
M. Cenk Gursoy,
Ismail Guvenc
Abstract:
In this paper, an analytical framework is provided to analyze the energy coverage performance of unmanned aerial vehicle (UAV) energy harvesting networks with clustered user equipments (UEs). Locations of UEs are modeled as a Poisson Cluster Process (PCP), and UAVs are assumed to be located at a certain height above the center of user clusters. Hence, user-centric UAV deployments are addressed. Tw…
▽ More
In this paper, an analytical framework is provided to analyze the energy coverage performance of unmanned aerial vehicle (UAV) energy harvesting networks with clustered user equipments (UEs). Locations of UEs are modeled as a Poisson Cluster Process (PCP), and UAVs are assumed to be located at a certain height above the center of user clusters. Hence, user-centric UAV deployments are addressed. Two different models are considered for the line-of-sight (LOS) probability function to compare their effects on the network performance. Moreover, antennas with doughnut-shaped radiation patterns are employed at both UAVs and UEs, and the impact of practical 3D antenna radiation patterns on the network performance is also investigated. Initially, the path loss of each tier is statistically described by deriving the complementary cumulative distribution function and probability density function. Following this, association probabilities with each tier are determined, and energy coverage probability of the UAV network is characterized in terms of key system and network parameters for UAV deployments both at a single height level and more generally at multiple heights. Through numerical results, we have shown that cluster size and UAV height play crucial roles on the energy coverage performance. Furthermore, energy coverage probability is significantly affected by the antenna orientation and number of UAVs in the network.
△ Less
Submitted 16 July, 2020;
originally announced July 2020.
-
Adversarial jamming attacks and defense strategies via adaptive deep reinforcement learning
Authors:
Feng Wang,
Chen Zhong,
M. Cenk Gursoy,
Senem Velipasalar
Abstract:
As the applications of deep reinforcement learning (DRL) in wireless communications grow, sensitivity of DRL based wireless communication strategies against adversarial attacks has started to draw increasing attention. In order to address such sensitivity and alleviate the resulting security concerns, we in this paper consider a victim user that performs DRL-based dynamic channel access, and an at…
▽ More
As the applications of deep reinforcement learning (DRL) in wireless communications grow, sensitivity of DRL based wireless communication strategies against adversarial attacks has started to draw increasing attention. In order to address such sensitivity and alleviate the resulting security concerns, we in this paper consider a victim user that performs DRL-based dynamic channel access, and an attacker that executes DRLbased jamming attacks to disrupt the victim. Hence, both the victim and attacker are DRL agents and can interact with each other, retrain their models, and adapt to opponents' policies. In this setting, we initially develop an adversarial jamming attack policy that aims at minimizing the accuracy of victim's decision making on dynamic channel access. Subsequently, we devise defense strategies against such an attacker, and propose three defense strategies, namely diversified defense with proportional-integral-derivative (PID) control, diversified defense with an imitation attacker, and defense via orthogonal policies. We design these strategies to maximize the attacked victim's accuracy and evaluate their performances.
△ Less
Submitted 12 July, 2020;
originally announced July 2020.
-
Anomaly Detection Under Controlled Sensing Using Actor-Critic Reinforcement Learning
Authors:
Geethu Joseph,
M. Cenk Gursoy,
Pramod K. Varshney
Abstract:
We consider the problem of detecting anomalies among a given set of processes using their noisy binary sensor measurements. The noiseless sensor measurement corresponding to a normal process is 0, and the measurement is 1 if the process is anomalous. The decision-making algorithm is assumed to have no knowledge of the number of anomalous processes. The algorithm is allowed to choose a subset of th…
▽ More
We consider the problem of detecting anomalies among a given set of processes using their noisy binary sensor measurements. The noiseless sensor measurement corresponding to a normal process is 0, and the measurement is 1 if the process is anomalous. The decision-making algorithm is assumed to have no knowledge of the number of anomalous processes. The algorithm is allowed to choose a subset of the sensors at each time instant until the confidence level on the decision exceeds the desired value. Our objective is to design a sequential sensor selection policy that dynamically determines which processes to observe at each time and when to terminate the detection algorithm. The selection policy is designed such that the anomalous processes are detected with the desired confidence level while incurring minimum cost which comprises the delay in detection and the cost of sensing. We cast this problem as a sequential hypothesis testing problem within the framework of Markov decision processes, and solve it using the actor-critic deep reinforcement learning algorithm. This deep neural network-based algorithm offers a low complexity solution with good detection accuracy. We also study the effect of statistical dependence between the processes on the algorithm performance. Through numerical experiments, we show that our algorithm is able to adapt to any unknown statistical dependence pattern of the processes.
△ Less
Submitted 26 May, 2020;
originally announced June 2020.
-
Mission-Aware Spatio-Temporal Deep Learning Model for UAS Instantaneous Density Prediction
Authors:
Ziyi Zhao,
Zhao **,
Wentian Bai,
Wentan Bai,
Carlos Caicedo,
M. Cenk Gursoy,
Qinru Qiu
Abstract:
The number of daily sUAS operations in uncontrolled low altitude airspace is expected to reach into the millions in a few years. Therefore, UAS density prediction has become an emerging and challenging problem. In this paper, a deep learning-based UAS instantaneous density prediction model is presented. The model takes two types of data as input: 1) the historical density generated from the histor…
▽ More
The number of daily sUAS operations in uncontrolled low altitude airspace is expected to reach into the millions in a few years. Therefore, UAS density prediction has become an emerging and challenging problem. In this paper, a deep learning-based UAS instantaneous density prediction model is presented. The model takes two types of data as input: 1) the historical density generated from the historical data, and 2) the future sUAS mission information. The architecture of our model contains four components: Historical Density Formulation module, UAS Mission Translation module, Mission Feature Extraction module, and Density Map Projection module. The training and testing data are generated by a python based simulator which is inspired by the multi-agent air traffic resource usage simulator (MATRUS) framework. The quality of prediction is measured by the correlation score and the Area Under the Receiver Operating Characteristics (AUROC) between the predicted value and simulated value. The experimental results demonstrate outstanding performance of the deep learning-based UAS density predictor. Compared to the baseline models, for simplified traffic scenario where no-fly zones and safe distance among sUASs are not considered, our model improves the prediction accuracy by more than 15.2% and its correlation score reaches 0.947. In a more realistic scenario, where the no-fly zone avoidance and the safe distance among sUASs are maintained using A* routing algorithm, our model can still achieve 0.823 correlation score. Meanwhile, the AUROC can reach 0.951 for the hot spot prediction.
△ Less
Submitted 21 March, 2020;
originally announced March 2020.
-
Simulation of Real-time Routing for UAS traffic Management with Communication and Airspace Safety Considerations
Authors:
Zhao **,
Ziyi Zhao,
Chen Luo,
Franco Basti,
Adrian Solomon,
M. Cenk Gursoy,
Carlos Caicedo,
Qinru Qiu
Abstract:
Small Unmanned Aircraft Systems (sUAS) will be an important component of the smart city and intelligent transportation environments of the near future. The demand for sUAS related applications, such as commercial delivery and land surveying, is expected to grow rapidly in next few years. In general, sUAS traffic routing and management functions are needed to coordinate the launching of sUAS from d…
▽ More
Small Unmanned Aircraft Systems (sUAS) will be an important component of the smart city and intelligent transportation environments of the near future. The demand for sUAS related applications, such as commercial delivery and land surveying, is expected to grow rapidly in next few years. In general, sUAS traffic routing and management functions are needed to coordinate the launching of sUAS from different launch sites and determine their trajectories to avoid conflict while considering several other constraints such as expected arrival time, minimum flight energy, and availability of communication resources. However, as the airborne sUAS density grows in a certain area, it is difficult to foresee the potential airspace and communications resource conflicts and make immediate decisions to avoid them. To address this challenge, we present a temporal and spatial routing algorithm and simulation platform for sUAS trajectory management in a high density urban area that plans sUAS movements in a spatial and temporal maze taking into account obstacles that are either static or dynamic in time. The routing allows the sUAS to avoid static no-fly areas (i.e. static obstacles) or other in-flight sUAS and areas that have congested communication resources (i.e. dynamic obstacles). The algorithm is evaluated using an agent-based simulation platform. The simulation results show that the proposed algorithm outperforms other route management algorithms in many areas, especially in processing speed and memory efficiency. Detailed comparisons are provided for the sUAS flight time, the overall throughput, conflict rate and communication resource utilization. The results demonstrate that our proposed algorithm can be used to address the airspace and communication resource utilization needs for a next generation smart city and smart transportation.
△ Less
Submitted 26 February, 2020;
originally announced February 2020.
-
Optimal Resource Allocation in Ground Wireless Networks Supporting Unmanned Aerial Vehicle Transmissions
Authors:
Yulin Hu,
Guodong Sun,
Guohua Zhang,
M. Cenk Gursoy,
Anke Schmeink
Abstract:
We consider a fully-loaded ground wireless network supporting unmanned aerial vehicle (UAV) transmission services. To enable the overload transmissions to a ground user (GU) and a UAV, two transmission schemes are employed, namely non-orthogonal multiple access (NOMA) and relaying, depending on whether or not the GU and UAV are served simultaneously. Under the assumption of the system operating wi…
▽ More
We consider a fully-loaded ground wireless network supporting unmanned aerial vehicle (UAV) transmission services. To enable the overload transmissions to a ground user (GU) and a UAV, two transmission schemes are employed, namely non-orthogonal multiple access (NOMA) and relaying, depending on whether or not the GU and UAV are served simultaneously. Under the assumption of the system operating with infinite blocklength (IBL) codes, the IBL throughputs of both the GU and the UAV are derived under the two schemes. More importantly, we also consider the scenario in which data packets are transmitted via finite blocklength (FBL) codes, i.e., data transmission to both the UAV and the GU is performed under low-latency and high reliability constraints. In this setting, the FBL throughputs are characterized again considering the two schemes of NOMA and relaying. Following the IBL and FBL throughput characterizations, optimal resource allocation designs are subsequently proposed to maximize the UAV throughput while guaranteeing the throughput of the cellular user.Moreover, we prove that the relaying scheme is able to provide transmission service to the UAV while improving the GU's performance, and that the relaying scheme potentially offers a higher throughput to the UAV in the FBL regime than in the IBL regime. On the other hand, the NOMA scheme provides a higher UAV throughput (than relaying) by slightly sacrificing the GU's performance.
△ Less
Submitted 19 January, 2020;
originally announced January 2020.
-
Coverage Analysis for Energy-Harvesting UAV-assisted mmWave Cellular Networks
Authors:
Xueyuan Wang,
M. Cenk Gursoy
Abstract:
In this paper, we jointly consider the downlink simultaneous wireless information and power transfer (SWIPT) and uplink information transmission in unmanned aerial vehicle (UAV)-assisted millimeter wave (mmWave) cellular networks, in which the user equipment (UE) locations are modeled using Poisson cluster processes (e.g., Thomas cluster processes or Matérn cluster processes). Distinguishing featu…
▽ More
In this paper, we jointly consider the downlink simultaneous wireless information and power transfer (SWIPT) and uplink information transmission in unmanned aerial vehicle (UAV)-assisted millimeter wave (mmWave) cellular networks, in which the user equipment (UE) locations are modeled using Poisson cluster processes (e.g., Thomas cluster processes or Matérn cluster processes). Distinguishing features of mmWave communications, such as different path loss models for line-of-sight (LOS) and non-LOS (NLOS) links and directional transmissions are taken into account. In the downlink phase, the association probability, and energy coverages of different tier UAVs and ground base stations (GBSs) are investigated. Moreover, we define a successful transmission probability to jointly present the energy and signal-to-interference-plus-noise ratio (SINR) coverages and provide general expressions. In the uplink phase, we consider the scenario that each UAV receives information from its own cluster member UEs. We determine the Laplace transform of the interference components and characterize the uplink SINR coverage. In addition, we formulate the average uplink throughput, with the goal to identify the optimal time division multiplexing between the donwlink and uplink phases. Through numerical results we investigate the impact of key system parameters on the performance. We show that the network performance is improved when the cluster size becomes smaller. In addition, we analyze the optimal height of UAVs, optimal power splitting value and optimal time division multiplexing that maximizes the network performance.
△ Less
Submitted 22 October, 2019;
originally announced October 2019.
-
RSS-Based Detection of Drones in the Presence of RF Interferers
Authors:
Priyanka Sinha,
Yavuz Yapici,
Ismail Guvenc,
Esma Turgut,
M. Cenk Gursoy
Abstract:
Drones will have extensive use cases across various commercial, government, and military sectors, ranging from delivery of consumer goods to search and rescue operations. To maintain the safety and security of people and infrastructure, it becomes critically important to quickly and accurately detect non-cooperating drones. In this paper we formulate a received signal strength (RSS) based detector…
▽ More
Drones will have extensive use cases across various commercial, government, and military sectors, ranging from delivery of consumer goods to search and rescue operations. To maintain the safety and security of people and infrastructure, it becomes critically important to quickly and accurately detect non-cooperating drones. In this paper we formulate a received signal strength (RSS) based detector, leveraging the existing wireless infrastructures that might already be serving other devices. Thus the detector can detect the presence of a drone signal buried in radio frequency (RF) interference and thermal noise, in a mixed line-of-sight (LOS) and non-LOS (NLOS) environment. We develop analytical expressions for the probability of false alarm and the probability of detection of a drone, which quantify the impact of aggregate interference and air-to-ground (A2G) propagation characteristics on the detection performance of individual sensors. We also provide analytical expressions for the average network probability of detection, which capture the impact of sensor density on a network's detection coverage. Finally, we find the critical sensor density that maximizes the average network probability of detection for a given requirement of the probability of false alarm.
△ Less
Submitted 3 June, 2019; v1 submitted 9 May, 2019;
originally announced May 2019.
-
A Simulation Framework for Fast Design Space Exploration of Unmanned Air System Traffic Management Policies
Authors:
Ziyi Zhao,
Chen Luo,
** Zhao,
Qinru Qiu,
M. Cenk Gursoy,
Carlos Caicedo,
Franco Basti
Abstract:
The number of daily small Unmanned Aircraft Systems (sUAS) operations in uncontrolled low altitude airspace is expected to reach into the millions. UAS Traffic Management (UTM) is an emerging concept aiming at the safe and efficient management of such very dense traffic, but few studies are addressing the policies to accommodate such demand and the required ground infrastructure in suburban or urb…
▽ More
The number of daily small Unmanned Aircraft Systems (sUAS) operations in uncontrolled low altitude airspace is expected to reach into the millions. UAS Traffic Management (UTM) is an emerging concept aiming at the safe and efficient management of such very dense traffic, but few studies are addressing the policies to accommodate such demand and the required ground infrastructure in suburban or urban environments. Searching for the optimal air traffic management policy is a combinatorial optimization problem with intractable complexity when the number of sUAS and the constraints increases. As the demands on the airspace increase and traffic patterns get complicated, it is difficult to forecast the potential low altitude airspace hotspots and the corresponding ground resource requirements. This work presents a Multi-agent Air Traffic and Resource Usage Simulation (MATRUS) framework that aims for fast evaluation of different air traffic management policies and the relationship between policy, environment and resulting traffic patterns. It can also be used as a tool to decide the resource distribution and launch site location in the planning of a next-generation smart city. As a case study, detailed comparisons are provided for the sUAS flight time, conflict ratio, cellular communication resource usage, for a managed (centrally coordinated) and unmanaged (free flight) traffic scenario.
△ Less
Submitted 14 February, 2019; v1 submitted 11 February, 2019;
originally announced February 2019.
-
Deep Learning Based Power Control for Quality-Driven Wireless Video Transmissions
Authors:
Chuang Ye,
M. Cenk Gursoy,
Senem Velipasalar
Abstract:
In this paper, wireless video transmission to multiple users under total transmission power and minimum required video quality constraints is studied. In order to provide the desired performance levels to the end-users in real-time video transmissions while using the energy resources efficiently, we assume that power control is employed. Due to the presence of interference, determining the optimal…
▽ More
In this paper, wireless video transmission to multiple users under total transmission power and minimum required video quality constraints is studied. In order to provide the desired performance levels to the end-users in real-time video transmissions while using the energy resources efficiently, we assume that power control is employed. Due to the presence of interference, determining the optimal power control is a non-convex problem but can be solved via monotonic optimization framework. However, monotonic optimization is an iterative algorithm and can often entail considerable computational complexity, making it not suitable for real-time applications. To address this, we propose a learning-based approach that treats the input and output of a resource allocation algorithm as an unknown nonlinear map** and a deep neural network (DNN) is employed to learn this map**. This learned map** via DNN can provide the optimal power level quickly for given channel conditions.
△ Less
Submitted 16 October, 2018;
originally announced October 2018.
-
Learning-Based Delay-Aware Caching in Wireless D2D Caching Networks
Authors:
Yi Li,
Chen Zhong,
M. Cenk Gursoy,
Senem Velipasalar
Abstract:
Recently, wireless caching techniques have been studied to satisfy lower delay requirements and offload traffic from peak periods. By storing parts of the popular files at the mobile users, users can locate some of their requested files in their own caches or the caches at their neighbors. In the latter case, when a user receives files from its neighbors, device-to-device(D2D) communication is per…
▽ More
Recently, wireless caching techniques have been studied to satisfy lower delay requirements and offload traffic from peak periods. By storing parts of the popular files at the mobile users, users can locate some of their requested files in their own caches or the caches at their neighbors. In the latter case, when a user receives files from its neighbors, device-to-device(D2D) communication is performed. D2D communication underlaid with cellular networks is also a new paradigm for the upcoming wireless systems. By allowing a pair of adjacent D2D users to communicate directly, D2D communication can achieve higher throughput, better energy efficiency and lower traffic delay. In this work, we propose an efficient learning-based caching algorithm operating together with a non-parametric estimator to minimize the average transmission delay in D2D-enabled cellular networks. It is assumed that the system does not have any prior information regarding the popularity of the files, and the non-parametric estimator is aimed at learning the intensity function of the file requests. An algorithm is devised to determine the best <file,user> pairs that provide the best delay improvement in each loop to form a caching policy with very low transmission delay and high throughput. This algorithm is also extended to address a more general scenario, in which the distributions of fading coefficients and values of system parameters potentially change over time. Via numerical results, the superiority of the proposed algorithm is verified by comparing it with a naive algorithm, in which all users simply cache their favorite files, and by comparing with a probabilistic algorithm, in which the users cache a file with a probability that is proportional to its popularity.
△ Less
Submitted 26 September, 2018;
originally announced September 2018.
