-
Generative AI Empowered LiDAR Point Cloud Generation with Multimodal Transformer
Authors:
Mohammad Farzanullah,
Han Zhang,
Akram Bin Sediq,
Ali Afana,
Melike Erol-Kantarci
Abstract:
Integrated sensing and communications is a key enabler for the 6G wireless communication systems. The multiple sensing modalities will allow the base station to have a more accurate representation of the environment, leading to context-aware communications. Some widely equipped sensors such as cameras and RADAR sensors can provide some environmental perceptions. However, they are not enough to gen…
▽ More
Integrated sensing and communications is a key enabler for the 6G wireless communication systems. The multiple sensing modalities will allow the base station to have a more accurate representation of the environment, leading to context-aware communications. Some widely equipped sensors such as cameras and RADAR sensors can provide some environmental perceptions. However, they are not enough to generate precise environmental representations, especially in adverse weather conditions. On the other hand, the LiDAR sensors provide more accurate representations, however, their widespread adoption is hindered by their high cost. This paper proposes a novel approach to enhance the wireless communication systems by synthesizing LiDAR point clouds from images and RADAR data. Specifically, it uses a multimodal transformer architecture and pre-trained encoding models to enable an accurate LiDAR generation. The proposed framework is evaluated on the DeepSense 6G dataset, which is a real-world dataset curated for context-aware wireless applications. Our results demonstrate the efficacy of the proposed approach in accurately generating LiDAR point clouds. We achieve a modified mean squared error of 10.3931. Visual examination of the images indicates that our model can successfully capture the majority of structures present in the LiDAR point cloud for diverse environments. This will enable the base stations to achieve more precise environmental sensing. By integrating LiDAR synthesis with existing sensing modalities, our method can enhance the performance of various wireless applications, including beam and blockage prediction.
△ Less
Submitted 20 May, 2024;
originally announced June 2024.
-
DT-DDNN: A Physical Layer Security Attack Detector in 5G RF Domain for CAVs
Authors:
Ghazal Asemian,
Mohammadreza Amini,
Burak Kantarci,
Melike Erol-Kantarci
Abstract:
The Synchronization Signal Block (SSB) is a fundamental component of the 5G New Radio (NR) air interface, crucial for the initial access procedure of Connected and Automated Vehicles (CAVs), and serves several key purposes in the network's operation. However, due to the predictable nature of SSB transmission, including the Primary and Secondary Synchronization Signals (PSS and SSS), jamming attack…
▽ More
The Synchronization Signal Block (SSB) is a fundamental component of the 5G New Radio (NR) air interface, crucial for the initial access procedure of Connected and Automated Vehicles (CAVs), and serves several key purposes in the network's operation. However, due to the predictable nature of SSB transmission, including the Primary and Secondary Synchronization Signals (PSS and SSS), jamming attacks are critical threats. These attacks, which can be executed without requiring high power or complex equipment, pose substantial risks to the 5G network, particularly as a result of the unencrypted transmission of control signals. Leveraging RF domain knowledge, this work presents a novel deep learning-based technique for detecting jammers in CAV networks. Unlike the existing jamming detection algorithms that mostly rely on network parameters, we introduce a double-threshold deep learning jamming detector by focusing on the SSB. The detection method is focused on RF domain features and improves the robustness of the network without requiring integration with the pre-existing network infrastructure. By integrating a preprocessing block to extract PSS correlation and energy per null resource elements (EPNRE) characteristics, our method distinguishes between normal and jammed received signals with high precision. Additionally, by incorporating of Discrete Wavelet Transform (DWT), the efficacy of training and detection are optimized. A double-threshold double Deep Neural Network (DT-DDNN) is also introduced to the architecture complemented by a deep cascade learning model to increase the sensitivity of the model to variations of signal-to-jamming noise ratio (SJNR). Results show that the proposed method achieves 96.4% detection rate in extra low jamming power, i.e., SJNR between 15 to 30 dB. Further, performance of DT-DDNN is validated by analyzing real 5G signals obtained from a practical testbed.
△ Less
Submitted 11 March, 2024; v1 submitted 4 March, 2024;
originally announced March 2024.
-
Federated Learning with Dual Attention for Robust Modulation Classification under Attacks
Authors:
Han Zhang,
Medhat Elsayed,
Majid Bavand,
Raimundas Gaigalas,
Yigit Ozcan,
Melike Erol-Kantarci
Abstract:
Federated learning (FL) allows distributed participants to train machine learning models in a decentralized manner. It can be used for radio signal classification with multiple receivers due to its benefits in terms of privacy and scalability. However, the existing FL algorithms usually suffer from slow and unstable convergence and are vulnerable to poisoning attacks from malicious participants. I…
▽ More
Federated learning (FL) allows distributed participants to train machine learning models in a decentralized manner. It can be used for radio signal classification with multiple receivers due to its benefits in terms of privacy and scalability. However, the existing FL algorithms usually suffer from slow and unstable convergence and are vulnerable to poisoning attacks from malicious participants. In this work, we aim to design a versatile FL framework that simultaneously promotes the performance of the model both in a secure system and under attack. To this end, we leverage attention mechanisms as a defense against attacks in FL and propose a robust FL algorithm by integrating the attention mechanisms into the global model aggregation step. To be more specific, two attention models are combined to calculate the amount of attention cast on each participant. It will then be used to determine the weights of local models during the global aggregation. The proposed algorithm is verified on a real-world dataset and it outperforms existing algorithms, both in secure systems and in systems under data poisoning attacks.
△ Less
Submitted 19 January, 2024;
originally announced January 2024.
-
Empowering the 6G Cellular Architecture with Open RAN
Authors:
Michele Polese,
Mischa Dohler,
Falko Dressler,
Melike Erol-Kantarci,
Rittwik Jana,
Raymond Knopp,
Tommaso Melodia
Abstract:
Innovation and standardization in 5G have brought advancements to every facet of the cellular architecture. This ranges from the introduction of new frequency bands and signaling technologies for the radio access network (RAN), to a core network underpinned by micro-services and network function virtualization (NFV). However, like any emerging technology, the pace of real-world deployments does no…
▽ More
Innovation and standardization in 5G have brought advancements to every facet of the cellular architecture. This ranges from the introduction of new frequency bands and signaling technologies for the radio access network (RAN), to a core network underpinned by micro-services and network function virtualization (NFV). However, like any emerging technology, the pace of real-world deployments does not instantly match the pace of innovation. To address this discrepancy, one of the key aspects under continuous development is the RAN with the aim of making it more open, adaptive, functional, and easy to manage. In this paper, we highlight the transformative potential of embracing novel cellular architectures by transitioning from conventional systems to the progressive principles of Open RAN. This promises to make 6G networks more agile, cost-effective, energy-efficient, and resilient. It opens up a plethora of novel use cases, ranging from ubiquitous support for autonomous devices to cost-effective expansions in regions previously underserved. The principles of Open RAN encompass: (i) a disaggregated architecture with modular and standardized interfaces; (ii) cloudification, programmability and orchestration; and (iii) AI-enabled data-centric closed-loop control and automation. We first discuss the transformative role Open RAN principles have played in the 5G era. Then, we adopt a system-level approach and describe how these Open RAN principles will support 6G RAN and architecture innovation. We qualitatively discuss potential performance gains that Open RAN principles yield for specific 6G use cases. For each principle, we outline the steps that research, development and standardization communities ought to take to make Open RAN principles central to next-generation cellular network designs.
△ Less
Submitted 5 December, 2023;
originally announced December 2023.
-
Heuristic Algorithms for RIS-assisted Wireless Networks: Exploring Heuristic-aided Machine Learning
Authors:
Hao Zhou,
Melike Erol-Kantarci,
Yuanwei Liu,
H. Vincent Poor
Abstract:
Reconfigurable intelligent surfaces (RISs) are a promising technology to enable smart radio environments. However, integrating RISs into wireless networks also leads to substantial complexity for network management. This work investigates heuristic algorithms and applications to optimize RIS-aided wireless networks, including greedy algorithms, meta-heuristic algorithms, and matching theory. Moreo…
▽ More
Reconfigurable intelligent surfaces (RISs) are a promising technology to enable smart radio environments. However, integrating RISs into wireless networks also leads to substantial complexity for network management. This work investigates heuristic algorithms and applications to optimize RIS-aided wireless networks, including greedy algorithms, meta-heuristic algorithms, and matching theory. Moreover, we combine heuristic algorithms with machine learning (ML), and propose three heuristic-aided ML algorithms, namely heuristic deep reinforcement learning (DRL), heuristic-aided supervised learning, and heuristic hierarchical learning. Finally, a case study shows that heuristic DRL can achieve higher data rates and faster convergence than conventional deep Q-networks (DQN). This work provides a new perspective for optimizing RIS-aided wireless networks by taking advantage of heuristic algorithms and ML.
△ Less
Submitted 5 November, 2023; v1 submitted 26 June, 2023;
originally announced July 2023.
-
Smart Home Energy Management: VAE-GAN synthetic dataset generator and Q-learning
Authors:
Mina Razghandi,
Hao Zhou,
Melike Erol-Kantarci,
Damla Turgut
Abstract:
Recent years have noticed an increasing interest among academia and industry towards analyzing the electrical consumption of residential buildings and employing smart home energy management systems (HEMS) to reduce household energy consumption and costs. HEMS has been developed to simulate the statistical and functional properties of actual smart grids. Access to publicly available datasets is a m…
▽ More
Recent years have noticed an increasing interest among academia and industry towards analyzing the electrical consumption of residential buildings and employing smart home energy management systems (HEMS) to reduce household energy consumption and costs. HEMS has been developed to simulate the statistical and functional properties of actual smart grids. Access to publicly available datasets is a major challenge in this type of research. The potential of artificial HEMS applications will be further enhanced with the development of time series that represent different operating conditions of the synthetic systems. In this paper, we propose a novel variational auto-encoder-generative adversarial network (VAE-GAN) technique for generating time-series data on energy consumption in smart homes. We also explore how the generative model performs when combined with a Q-learning-based HEMS. We tested the online performance of Q-learning-based HEMS with real-world smart home data. To test the generated dataset, we measure the Kullback-Leibler (KL) divergence, maximum mean discrepancy (MMD), and the Wasserstein distance between the probability distributions of the real and synthetic data. Our experiments show that VAE-GAN-generated synthetic data closely matches the real data distribution. Finally, we show that the generated data allows for the training of a higher-performance Q-learning-based HEMS compared to datasets generated with baseline approaches.
△ Less
Submitted 14 May, 2023;
originally announced May 2023.
-
Cooperative Hierarchical Deep Reinforcement Learning based Joint Sleep, Power, and RIS Control for Energy-Efficient HetNet
Authors:
Hao Zhou,
Medhat Elsayed,
Majid Bavand,
Raimundas Gaigalas,
Steve Furr,
Melike Erol-Kantarci
Abstract:
Energy efficiency (EE) is one of the most important metrics for 5G and future 6G networks to reduce energy costs and control carbon footprint. Sleep control, as a cost-efficient approach, can significantly lower power consumption by switching off network devices selectively. Meanwhile, reconfigurable intelligent surface (RIS) has emerged as a promising technique to enhance the EE of 5G beyond and…
▽ More
Energy efficiency (EE) is one of the most important metrics for 5G and future 6G networks to reduce energy costs and control carbon footprint. Sleep control, as a cost-efficient approach, can significantly lower power consumption by switching off network devices selectively. Meanwhile, reconfigurable intelligent surface (RIS) has emerged as a promising technique to enhance the EE of 5G beyond and 6G networks. In this work, we jointly consider sleep and transmission power control for reconfigurable intelligent surface (RIS)-aided energy-efficient heterogeneous networks (Hetnets). In particular, we first propose a fractional programming (FP) method for RIS phase-shift control, which aims to maximize the sum-rate under given transmission power levels. Then, considering the timescale difference between sleep control and power control, we introduce a cooperative hierarchical deep reinforcement learning (Co-HDRL) algorithm, including a cross-entropy enabled meta-controller for sleep control, and correlated equilibrium-based sub-controllers for power control. Moreover, we proposed a surrogate optimization method as one baseline for RIS control, and conventional HDRL as another baseline for sleep and power control. Finally, simulations show that the RIS-assisted sleep control can achieve more than 16% lower energy consumption and 30% higher energy efficiency than baseline algorithms.
△ Less
Submitted 25 April, 2023;
originally announced April 2023.
-
On-Device Intelligence for 5G RAN: Knowledge Transfer and Federated Learning enabled UE-Centric Traffic Steering
Authors:
Han Zhang,
Hao Zhou,
Medhat Elsayed,
Majid Bavand,
Raimundas Gaigalas,
Yigit Ozcan,
Melike Erol-Kantarci
Abstract:
Traffic steering (TS) is a promising approach to support various service requirements and enhance transmission reliability by distributing network traffic loads to appropriate base stations (BSs). In conventional cell-centric TS strategies, BSs make TS decisions for all user equipment (UEs) in a centralized manner, which focuses more on the overall performance of the whole cell, disregarding speci…
▽ More
Traffic steering (TS) is a promising approach to support various service requirements and enhance transmission reliability by distributing network traffic loads to appropriate base stations (BSs). In conventional cell-centric TS strategies, BSs make TS decisions for all user equipment (UEs) in a centralized manner, which focuses more on the overall performance of the whole cell, disregarding specific requirements of individual UE. The flourishing machine learning technologies and evolving UE-centric 5G network architecture have prompted the emergence of new TS technologies. In this paper, we propose a knowledge transfer and federated learning-enabled UE-centric (KT-FLUC) TS framework for highly dynamic 5G radio access networks (RAN). Specifically, first, we propose an attention-weighted group federated learning scheme. It enables intelligent UEs to make TS decisions autonomously using local models and observations, and a global model is defined to coordinate local TS decisions and share experiences among UEs. Secondly, considering the individual UE's limited computation and energy resources, a growing and pruning-based model compression method is introduced, mitigating the computation burden of UEs and reducing the communication overhead of federated learning. In addition, we propose a Q-value-based knowledge transfer method to initialize newcomer UEs, achieving a jump start for their training efficiency. Finally, the simulations show that our proposed KT-FLUC algorithm can effectively improve the service quality, achieving 65\% and 38\% lower delay and 52% and 57% higher throughput compared with cell-based TS and other UE-centric TS strategies, respectively.
△ Less
Submitted 28 November, 2023; v1 submitted 21 April, 2023;
originally announced April 2023.
-
A Survey on Model-based, Heuristic, and Machine Learning Optimization Approaches in RIS-aided Wireless Networks
Authors:
Hao Zhou,
Melike Erol-Kantarci,
Yuanwei Liu,
H. Vincent Poor
Abstract:
Reconfigurable intelligent surfaces (RISs) have received considerable attention as a key enabler for envisioned 6G networks, for the purpose of improving the network capacity, coverage, efficiency, and security with low energy consumption and low hardware cost. However, integrating RISs into the existing infrastructure greatly increases the network management complexity, especially for controlling…
▽ More
Reconfigurable intelligent surfaces (RISs) have received considerable attention as a key enabler for envisioned 6G networks, for the purpose of improving the network capacity, coverage, efficiency, and security with low energy consumption and low hardware cost. However, integrating RISs into the existing infrastructure greatly increases the network management complexity, especially for controlling a significant number of RIS elements. To unleash the full potential of RISs, efficient optimization approaches are of great importance. This work provides a comprehensive survey on optimization techniques for RIS-aided wireless communications, including model-based, heuristic, and machine learning (ML) algorithms. In particular, we first summarize the problem formulations in the literature with diverse objectives and constraints, e.g., sum-rate maximization, power minimization, and imperfect channel state information constraints. Then, we introduce model-based algorithms that have been used in the literature, such as alternating optimization, the majorization-minimization method, and successive convex approximation. Next, heuristic optimization is discussed, which applies heuristic rules for obtaining low-complexity solutions. Moreover, we present state-of-the-art ML algorithms and applications towards RISs, i.e., supervised and unsupervised learning, reinforcement learning, federated learning, graph learning, transfer learning, and hierarchical learning-based approaches. Model-based, heuristic, and ML approaches are compared in terms of stability, robustness, optimality and so on, providing a systematic understanding of these techniques. Finally, we highlight RIS-aided applications towards 6G networks and identify future challenges.
△ Less
Submitted 28 November, 2023; v1 submitted 24 March, 2023;
originally announced March 2023.
-
Beam Selection for Energy-Efficient mmWave Network Using Advantage Actor Critic Learning
Authors:
Ycaro Dantas,
Pedro Enrique Iturria-Rivera,
Hao Zhou,
Majid Bavand,
Medhat Elsayed,
Raimundas Gaigalas,
Melike Erol-Kantarci
Abstract:
The growing adoption of mmWave frequency bands to realize the full potential of 5G, turns beamforming into a key enabler for current and next-generation wireless technologies. Many mmWave networks rely on beam selection with Grid-of-Beams (GoB) approach to handle user-beam association. In beam selection with GoB, users select the appropriate beam from a set of pre-defined beams and the overhead du…
▽ More
The growing adoption of mmWave frequency bands to realize the full potential of 5G, turns beamforming into a key enabler for current and next-generation wireless technologies. Many mmWave networks rely on beam selection with Grid-of-Beams (GoB) approach to handle user-beam association. In beam selection with GoB, users select the appropriate beam from a set of pre-defined beams and the overhead during the beam selection process is a common challenge in this area. In this paper, we propose an Advantage Actor Critic (A2C) learning-based framework to improve the GoB and the beam selection process, as well as optimize transmission power in a mmWave network. The proposed beam selection technique allows performance improvement while considering transmission power improves Energy Efficiency (EE) and ensures the coverage is maintained in the network. We further investigate how the proposed algorithm can be deployed in a Service Management and Orchestration (SMO) platform. Our simulations show that A2C-based joint optimization of beam selection and transmission power is more effective than using Equally Spaced Beams (ESB) and fixed power strategy, or optimization of beam selection and transmission power disjointly. Compared to the ESB and fixed transmission power strategy, the proposed approach achieves more than twice the average EE in the scenarios under test and is closer to the maximum theoretical EE.
△ Less
Submitted 31 January, 2023;
originally announced February 2023.
-
Hierarchical Reinforcement Learning for RIS-Assisted Energy-Efficient RAN
Authors:
Hao Zhou,
Long Kong,
Medhat Elsayed,
Majid Bavand,
Raimundas Gaigalas,
Steve Furr,
Melike Erol-Kantarci
Abstract:
Reconfigurable intelligent surface (RIS) is emerging as a promising technology to boost the energy efficiency (EE) of 5G beyond and 6G networks. Inspired by this potential, in this paper, we investigate the RIS-assisted energy-efficient radio access networks (RAN). In particular, we combine RIS with sleep control techniques, and develop a hierarchical reinforcement learning (HRL) algorithm for net…
▽ More
Reconfigurable intelligent surface (RIS) is emerging as a promising technology to boost the energy efficiency (EE) of 5G beyond and 6G networks. Inspired by this potential, in this paper, we investigate the RIS-assisted energy-efficient radio access networks (RAN). In particular, we combine RIS with sleep control techniques, and develop a hierarchical reinforcement learning (HRL) algorithm for network management. In HRL, the meta-controller decides the on/off status of the small base stations (SBSs) in heterogeneous networks, while the sub-controller can change the transmission power levels of SBSs to save energy. The simulations show that the RIS-assisted sleep control can achieve significantly lower energy consumption, higher throughput, and more than doubled energy efficiency than no-RIS conditions.
△ Less
Submitted 6 January, 2023;
originally announced January 2023.
-
The Internet of Senses: Building on Semantic Communications and Edge Intelligence
Authors:
Roghayeh Joda,
Medhat Elsayed,
Hatem Abou-zeid,
Ramy Atawia,
Akram Bin Sediq,
Gary Boudreau,
Melike Erol-Kantarci,
Lajos Hanzo
Abstract:
The Internet of Senses (IoS) holds the promise of flawless telepresence-style communication for all human `receptors' and therefore blurs the difference of virtual and real environments. We commence by highlighting the compelling use cases empowered by the IoS and also the key network requirements. We then elaborate on how the emerging semantic communications and Artificial Intelligence (AI)/Machi…
▽ More
The Internet of Senses (IoS) holds the promise of flawless telepresence-style communication for all human `receptors' and therefore blurs the difference of virtual and real environments. We commence by highlighting the compelling use cases empowered by the IoS and also the key network requirements. We then elaborate on how the emerging semantic communications and Artificial Intelligence (AI)/Machine Learning (ML) paradigms along with 6G technologies may satisfy the requirements of IoS use cases. On one hand, semantic communications can be applied for extracting meaningful and significant information and hence efficiently exploit the resources and for harnessing a priori information at the receiver to satisfy IoS requirements. On the other hand, AI/ML facilitates frugal network resource management by making use of the enormous amount of data generated in IoS edge nodes and devices, as well as by optimizing the IoS performance via intelligent agents. However, the intelligent agents deployed at the edge are not completely aware of each others' decisions and the environments of each other, hence they operate in a partially rather than fully observable environment. Therefore, we present a case study of Partially Observable Markov Decision Processes (POMDP) for improving the User Equipment (UE) throughput and energy consumption, as they are imperative for IoS use cases, using Reinforcement Learning for astutely activating and deactivating the component carriers in carrier aggregation. Finally, we outline the challenges and open issues of IoS implementations and employing semantic communications, edge intelligence as well as learning under partial observability in the IoS context.
△ Less
Submitted 20 December, 2022;
originally announced December 2022.
-
Knowledge Transfer and Reuse: A Case Study of AI-enabled Resource Management in RAN Slicing
Authors:
Hao Zhou,
Melike Erol-Kantarci,
Vincent Poor
Abstract:
An efficient resource management scheme is critical to enable network slicing in 5G networks and in envisioned 6G networks, and artificial intelligence (AI) techniques offer promising solutions. Considering the rapidly emerging new machine learning techniques, such as graph learning, federated learning, and transfer learning, a timely survey is needed to provide an overview of resource management…
▽ More
An efficient resource management scheme is critical to enable network slicing in 5G networks and in envisioned 6G networks, and artificial intelligence (AI) techniques offer promising solutions. Considering the rapidly emerging new machine learning techniques, such as graph learning, federated learning, and transfer learning, a timely survey is needed to provide an overview of resource management and network slicing techniques of AI-enabled wireless networks. This article provides such a survey along with an application of knowledge transfer in radio access network (RAN) slicing. In particular, we firs provide some background on resource management and network slicing, and review relevant state-of-the-art AI and machine learning (ML) techniques and their applications. Then, we introduce our AI-enabled knowledge transfer and reuse-based resource management (AKRM) scheme, where we apply transfer learning to improve system performance. Compared with most existing works, which focus on the training of standalone agents from scratch, the main difference of AKRM lies in its knowledge transfer and reuse capability between different tasks. Our paper aims to be a roadmap for researchers to use knowledge transfer schemes in AI-enabled wireless networks, and we provide a case study over the resource allocation problem in RAN slicing.
△ Less
Submitted 18 December, 2022;
originally announced December 2022.
-
Joint Sensing and Communications for Deep Reinforcement Learning-based Beam Management in 6G
Authors:
Yujie Yao,
Hao Zhou,
Melike Erol-Kantarci
Abstract:
User location is a piece of critical information for network management and control. However, location uncertainty is unavoidable in certain settings leading to localization errors. In this paper, we consider the user location uncertainty in the mmWave networks, and investigate joint vision-aided sensing and communications using deep reinforcement learning-based beam management for future 6G netwo…
▽ More
User location is a piece of critical information for network management and control. However, location uncertainty is unavoidable in certain settings leading to localization errors. In this paper, we consider the user location uncertainty in the mmWave networks, and investigate joint vision-aided sensing and communications using deep reinforcement learning-based beam management for future 6G networks. In particular, we first extract pixel characteristic-based features from satellite images to improve localization accuracy. Then we propose a UK-medoids based method for user clustering with location uncertainty, and the clustering results are consequently used for the beam management. Finally, we apply the DRL algorithm for intra-beam radio resource allocation. The simulations first show that our proposed vision-aided method can substantially reduce the localization error. The proposed UK-medoids and DRL based scheme (UKM-DRL) is compared with two other schemes: K-means based clustering and DRL based resource allocation (K-DRL) and UK-means based clustering and DRL based resource allocation (UK-DRL). The proposed method has 17.2% higher throughput and 7.7% lower delay than UK-DRL, and more than doubled throughput and 55.8% lower delay than K-DRL.
△ Less
Submitted 3 August, 2022;
originally announced August 2022.
-
Federated Deep Reinforcement Learning for Resource Allocation in O-RAN Slicing
Authors:
Han Zhang,
Hao Zhou,
Melike Erol-Kantarci
Abstract:
Recently, open radio access network (O-RAN) has become a promising technology to provide an open environment for network vendors and operators. Coordinating the x-applications (xAPPs) is critical to increase flexibility and guarantee high overall network performance in O-RAN. Meanwhile, federated reinforcement learning has been proposed as a promising technique to enhance the collaboration among d…
▽ More
Recently, open radio access network (O-RAN) has become a promising technology to provide an open environment for network vendors and operators. Coordinating the x-applications (xAPPs) is critical to increase flexibility and guarantee high overall network performance in O-RAN. Meanwhile, federated reinforcement learning has been proposed as a promising technique to enhance the collaboration among distributed reinforcement learning agents and improve learning efficiency. In this paper, we propose a federated deep reinforcement learning algorithm to coordinate multiple independent xAPPs in O-RAN for network slicing. We design two xAPPs, namely a power control xAPP and a slice-based resource allocation xAPP, and we use a federated learning model to coordinate two xAPP agents to enhance learning efficiency and improve network performance. Compared with conventional deep reinforcement learning, our proposed algorithm can achieve 11% higher throughput for enhanced mobile broadband (eMBB) slices and 33% lower delay for ultra-reliable low-latency communication (URLLC) slices.
△ Less
Submitted 2 August, 2022;
originally announced August 2022.
-
Deep Reinforcement Learning-based Radio Resource Allocation and Beam Management under Location Uncertainty in 5G mmWave Networks
Authors:
Yujie Yao,
Hao Zhou,
Melike Erol-Kantarci
Abstract:
Millimeter Wave (mmWave) is an important part of 5G new radio (NR), in which highly directional beams are adapted to compensate for the substantial propagation loss based on UE locations. However, the location information may have some errors such as GPS errors. In any case, some uncertainty, and localization error is unavoidable in most settings. Applying these distorted locations for clustering…
▽ More
Millimeter Wave (mmWave) is an important part of 5G new radio (NR), in which highly directional beams are adapted to compensate for the substantial propagation loss based on UE locations. However, the location information may have some errors such as GPS errors. In any case, some uncertainty, and localization error is unavoidable in most settings. Applying these distorted locations for clustering will increase the error of beam management. Meanwhile, the traffic demand may change dynamically in the wireless environment. Therefore, a scheme that can handle both the uncertainty of localization and dynamic radio resource allocation is needed. In this paper, we propose a UK-means-based clustering and deep reinforcement learning-based resource allocation algorithm (UK-DRL) for radio resource allocation and beam management in 5G mmWave networks. We first apply UK-means as the clustering algorithm to mitigate the localization uncertainty, then deep reinforcement learning (DRL) is adopted to dynamically allocate radio resources. Finally, we compare the UK-DRL with K-means-based clustering and DRL-based resource allocation algorithm (K-DRL), the simulations show that our proposed UK-DRL-based method achieves 150% higher throughput and 61.5% lower delay compared with K-DRL when traffic load is 4Mbps.
△ Less
Submitted 22 April, 2022;
originally announced April 2022.
-
On the Impacts of Phase Shifting Design and Eavesdrop** Uncertainty on Secrecy Metrics of RIS-aided Systems
Authors:
Long Kong,
Steven Kisseleff,
Symeon Chatzinotas,
Björn Ottersten,
Melike Erol-Kantarci
Abstract:
This paper investigates the secrecy outage probability (SOP), the lower bound of SOP, and the probability of non-zero secrecy capacity (PNZ) of reconfigurable intelligent surface (RIS)-assisted systems from an information-theoretic perspective. In particular, we consider the impacts of eavesdroppers' location uncertainty and the phase adjustment uncertainty, namely imperfect coherent phase shiftin…
▽ More
This paper investigates the secrecy outage probability (SOP), the lower bound of SOP, and the probability of non-zero secrecy capacity (PNZ) of reconfigurable intelligent surface (RIS)-assisted systems from an information-theoretic perspective. In particular, we consider the impacts of eavesdroppers' location uncertainty and the phase adjustment uncertainty, namely imperfect coherent phase shifting and discrete phase shifting on RIS. More specifically, analytical and simulation results are presented to show that (i) the SOP gain due to the increase of the RIS reflecting elements number gradually decreases; and (ii) both phase shifting designs demonstrate the same PNZ secrecy performance, in other words, the random discrete phase shifting outperforms the imperfect coherent phase shifting design with reduced complexity.
△ Less
Submitted 4 April, 2022; v1 submitted 11 February, 2022;
originally announced February 2022.
-
Variational Autoencoder Generative Adversarial Network for Synthetic Data Generation in Smart Home
Authors:
Mina Razghandi,
Hao Zhou,
Melike Erol-Kantarci,
Damla Turgut
Abstract:
Data is the fuel of data science and machine learning techniques for smart grid applications, similar to many other fields. However, the availability of data can be an issue due to privacy concerns, data size, data quality, and so on. To this end, in this paper, we propose a Variational AutoEncoder Generative Adversarial Network (VAE-GAN) as a smart grid data generative model which is capable of l…
▽ More
Data is the fuel of data science and machine learning techniques for smart grid applications, similar to many other fields. However, the availability of data can be an issue due to privacy concerns, data size, data quality, and so on. To this end, in this paper, we propose a Variational AutoEncoder Generative Adversarial Network (VAE-GAN) as a smart grid data generative model which is capable of learning various types of data distributions and generating plausible samples from the same distribution without performing any prior analysis on the data before the training phase.We compared the Kullback-Leibler (KL) divergence, maximum mean discrepancy (MMD), and Wasserstein distance between the synthetic data (electrical load and PV production) distribution generated by the proposed model, vanilla GAN network, and the real data distribution, to evaluate the performance of our model. Furthermore, we used five key statistical parameters to describe the smart grid data distribution and compared them between synthetic data generated by both models and real data. Experiments indicate that the proposed synthetic data generative model outperforms the vanilla GAN network. The distribution of VAE-GAN synthetic data is the most comparable to that of real data.
△ Less
Submitted 18 January, 2022;
originally announced January 2022.
-
Effective Rate of RIS-aided Networks with Location and Phase Estimation Uncertainty
Authors:
Long Kong,
Steven Kisseleff,
Symeon Chatzinotas,
Björn Ottersten,
Melike Erol-Kantarci
Abstract:
Reconfigurable Intelligent Surfaces (RIS) are planar structures connected to electronic circuitry, which can be employed to steer the electromagnetic signals in a controlled manner. Through this, the signal quality and the effective data rate can be substantially improved. While the benefits of RIS-assisted wireless communications have been investigated for various scenarios, some aspects of the n…
▽ More
Reconfigurable Intelligent Surfaces (RIS) are planar structures connected to electronic circuitry, which can be employed to steer the electromagnetic signals in a controlled manner. Through this, the signal quality and the effective data rate can be substantially improved. While the benefits of RIS-assisted wireless communications have been investigated for various scenarios, some aspects of the network design, such as coverage, optimal placement of RIS, etc., often require complex optimization and numerical simulations, since the achievable effective rate is difficult to predict. This problem becomes even more difficult in the presence of phase estimation errors or location uncertainty, which can lead to substantial performance degradation if neglected. Considering randomly distributed receivers within a ring-shaped RIS-assisted wireless network, this paper mainly investigates the effective rate by taking into account the above-mentioned impairments. Furthermore, exact closed-form expressions for the effective rate are derived in terms of Meijer's $G$-function, which (i) reveals that the location and phase estimation uncertainty should be well considered in the deployment of RIS in wireless networks; and (ii) facilitates future network design and performance prediction.
△ Less
Submitted 17 December, 2021; v1 submitted 16 December, 2021;
originally announced December 2021.
-
Multi-agent Bayesian Deep Reinforcement Learning for Microgrid Energy Management under Communication Failures
Authors:
Hao Zhou,
Atakan Aral,
Ivona Brandic,
Melike Erol-Kantarci
Abstract:
Microgrids (MGs) are important players for the future transactive energy systems where a number of intelligent Internet of Things (IoT) devices interact for energy management in the smart grid. Although there have been many works on MG energy management, most studies assume a perfect communication environment, where communication failures are not considered. In this paper, we consider the MG as a…
▽ More
Microgrids (MGs) are important players for the future transactive energy systems where a number of intelligent Internet of Things (IoT) devices interact for energy management in the smart grid. Although there have been many works on MG energy management, most studies assume a perfect communication environment, where communication failures are not considered. In this paper, we consider the MG as a multi-agent environment with IoT devices in which AI agents exchange information with their peers for collaboration. However, the collaboration information may be lost due to communication failures or packet loss. Such events may affect the operation of the whole MG. To this end, we propose a multi-agent Bayesian deep reinforcement learning (BA-DRL) method for MG energy management under communication failures. We first define a multi-agent partially observable Markov decision process (MA-POMDP) to describe agents under communication failures, in which each agent can update its beliefs on the actions of its peers. Then, we apply a double deep Q-learning (DDQN) architecture for Q-value estimation in BA-DRL, and propose a belief-based correlated equilibrium for the joint-action selection of multi-agent BA-DRL. Finally, the simulation results show that BA-DRL is robust to both power supply uncertainty and communication failure uncertainty. BA-DRL has 4.1% and 10.3% higher reward than Nash Deep Q-learning (Nash-DQN) and alternating direction method of multipliers (ADMM) respectively under 1% communication failure probability.
△ Less
Submitted 21 November, 2021;
originally announced November 2021.
-
Knowledge Transfer based Radio and Computation Resource Allocation for 5G RAN Slicing
Authors:
Hao Zhou,
Melike Erol-Kantarci
Abstract:
To implement network slicing in 5G, resource allocation is a key function to allocate limited network resources such as radio and computation resources to multiple slices. However, the joint resource allocation also leads to a higher complexity in the network management. In this work, we propose a knowledge transfer based resource allocation (KTRA) method to jointly allocate radio and computation…
▽ More
To implement network slicing in 5G, resource allocation is a key function to allocate limited network resources such as radio and computation resources to multiple slices. However, the joint resource allocation also leads to a higher complexity in the network management. In this work, we propose a knowledge transfer based resource allocation (KTRA) method to jointly allocate radio and computation resources for 5G RAN slicing. Compared with existing works, the main difference is that the proposed KTRA method has a knowledge transfer capability. It is designed to use the prior knowledge of similar tasks to improve performance of the target task, e.g., faster convergence speed or higher average reward. The proposed KTRA is compared with Qlearning based resource allocation (QLRA), and KTRA method presents a 18.4% lower URLLC delay and a 30.1% higher eMBB throughput as well as a faster convergence speed.
△ Less
Submitted 23 October, 2021;
originally announced October 2021.
-
Smart Home Energy Management: Sequence-to-Sequence Load Forecasting and Q-Learning
Authors:
Mina Razghandi,
Hao Zhou,
Melike Erol-Kantarci,
Damla Turgut
Abstract:
A smart home energy management system (HEMS) can contribute towards reducing the energy costs of customers; however, HEMS suffers from uncertainty in both energy generation and consumption patterns. In this paper, we propose a sequence to sequence (Seq2Seq) learning-based supply and load prediction along with reinforcement learning-based HEMS control. We investigate how the prediction method affec…
▽ More
A smart home energy management system (HEMS) can contribute towards reducing the energy costs of customers; however, HEMS suffers from uncertainty in both energy generation and consumption patterns. In this paper, we propose a sequence to sequence (Seq2Seq) learning-based supply and load prediction along with reinforcement learning-based HEMS control. We investigate how the prediction method affects the HEMS operation. First, we use Seq2Seq learning to predict photovoltaic (PV) power and home devices' load. We then apply Q-learning for offline optimization of HEMS based on the prediction results. Finally, we test the online performance of the trained Q-learning scheme with actual PV and load data. The Seq2Seq learning is compared with VARMA, SVR, and LSTM in both prediction and operation levels. The simulation results show that Seq2Seq performs better with a lower prediction error and online operation performance.
△ Less
Submitted 25 September, 2021;
originally announced September 2021.
-
Learning from Peers: Deep Transfer Reinforcement Learning for Joint Radio and Cache Resource Allocation in 5G RAN Slicing
Authors:
Hao Zhou,
Melike Erol-Kantarci,
Vincent Poor
Abstract:
Network slicing is a critical technique for 5G communications that covers radio access network (RAN), edge, transport and core slicing.The evolving network architecture requires the orchestration of multiple network resources such as radio and cache resources. In recent years, machine learning (ML) techniques have been widely applied for network management. However, most existing works do not take…
▽ More
Network slicing is a critical technique for 5G communications that covers radio access network (RAN), edge, transport and core slicing.The evolving network architecture requires the orchestration of multiple network resources such as radio and cache resources. In recent years, machine learning (ML) techniques have been widely applied for network management. However, most existing works do not take advantage of the knowledge transfer capability in ML. In this paper, we propose a deep transfer reinforcement learning (DTRL) scheme for joint radio and cache resource allocation to serve 5G RAN slicing. We first define a hierarchical architecture for joint resource allocation. Then we propose two DTRL algorithms: Q-value-based deep transfer reinforcement learning (QDTRL) and action selection-based deep transfer reinforcement learning (ADTRL). In the proposed schemes, learner agents utilize expert agents' knowledge to improve their performance on current tasks. The proposed algorithms are compared with both the model-free exploration bonus deep Q-learning (EB-DQN) and the model-based priority proportional fairness and time-to-live (PPF-TTL) algorithms. Compared with EB-DQN, our proposed DTRL-based method presents 21.4% lower delay for Ultra Reliable Low Latency Communications (URLLC) slice and 22.4% higher throughput for enhanced Mobile Broad Band (eMBB) slice, while achieving significantly faster convergence than EB-DQN. Moreover, 40.8% lower URLLC delay and 59.8% higher eMBB throughput are observed with respect to PPF-TTL.
△ Less
Submitted 31 August, 2022; v1 submitted 16 September, 2021;
originally announced September 2021.
-
RAN Resource Slicing in 5G Using Multi-Agent Correlated Q-Learning
Authors:
Hao Zhou,
Medhat Elsayed,
Melike Erol-Kantarci
Abstract:
5G is regarded as a revolutionary mobile network, which is expected to satisfy a vast number of novel services, ranging from remote health care to smart cities. However, heterogeneous Quality of Service (QoS) requirements of different services and limited spectrum make the radio resource allocation a challenging problem in 5G. In this paper, we propose a multi-agent reinforcement learning (MARL) m…
▽ More
5G is regarded as a revolutionary mobile network, which is expected to satisfy a vast number of novel services, ranging from remote health care to smart cities. However, heterogeneous Quality of Service (QoS) requirements of different services and limited spectrum make the radio resource allocation a challenging problem in 5G. In this paper, we propose a multi-agent reinforcement learning (MARL) method for radio resource slicing in 5G. We model each slice as an intelligent agent that competes for limited radio resources, and the correlated Q-learning is applied for inter-slice resource block (RB) allocation. The proposed correlated Q-learning based interslice RB allocation (COQRA) scheme is compared with Nash Q-learning (NQL), Latency-Reliability-Throughput Q-learning (LRTQ) methods, and the priority proportional fairness (PPF) algorithm. Our simulation results show that the proposed COQRA achieves 32.4% lower latency and 6.3% higher throughput when compared with LRTQ, and 5.8% lower latency and 5.9% higher throughput than NQL. Significantly higher throughput and lower packet drop rate (PDR) is observed in comparison to PPF.
△ Less
Submitted 23 June, 2021;
originally announced July 2021.
-
Short-Term Load Forecasting for Smart HomeAppliances with Sequence to Sequence Learning
Authors:
Mina Razghandi,
Hao Zhou,
Melike Erol-Kantarci,
Damla Turgut
Abstract:
Appliance-level load forecasting plays a critical role in residential energy management, besides having significant importance for ancillary services performed by the utilities. In this paper, we propose to use an LSTM-based sequence-to-sequence (seq2seq) learning model that can capture the load profiles of appliances. We use a real dataset collected fromfour residential buildings and compare our…
▽ More
Appliance-level load forecasting plays a critical role in residential energy management, besides having significant importance for ancillary services performed by the utilities. In this paper, we propose to use an LSTM-based sequence-to-sequence (seq2seq) learning model that can capture the load profiles of appliances. We use a real dataset collected fromfour residential buildings and compare our proposed schemewith three other techniques, namely VARMA, Dilated One Dimensional Convolutional Neural Network, and an LSTM model.The results show that the proposed LSTM-based seq2seq model outperforms other techniques in terms of prediction error in most cases.
△ Less
Submitted 25 June, 2021;
originally announced June 2021.
-
Age of Information Aware VNF Scheduling in Industrial IoT Using Deep Reinforcement Learning
Authors:
Mohammad Akbari,
Mohammad Reza Abedi,
Roghayeh Joda,
Mohsen Pourghasemian,
Nader Mokari,
Melike Erol-Kantarci
Abstract:
In delay-sensitive industrial internet of things (IIoT) applications, the age of information (AoI) is employed to characterize the freshness of information. Meanwhile, the emerging network function virtualization provides flexibility and agility for service providers to deliver a given network service using a sequence of virtual network functions (VNFs). However, suitable VNF placement and schedul…
▽ More
In delay-sensitive industrial internet of things (IIoT) applications, the age of information (AoI) is employed to characterize the freshness of information. Meanwhile, the emerging network function virtualization provides flexibility and agility for service providers to deliver a given network service using a sequence of virtual network functions (VNFs). However, suitable VNF placement and scheduling in these schemes is NP-hard and finding a globally optimal solution by traditional approaches is complex. Recently, deep reinforcement learning (DRL) has appeared as a viable way to solve such problems. In this paper, we first utilize single agent low-complex compound action actor-critic RL to cover both discrete and continuous actions and jointly minimize VNF cost and AoI in terms of network resources under end-to end Quality of Service constraints. To surmount the single-agent capacity limitation for learning, we then extend our solution to a multi-agent DRL scheme in which agents collaborate with each other. Simulation results demonstrate that single-agent schemes significantly outperform the greedy algorithm in terms of average network cost and AoI. Moreover, multi-agent solution decreases the average cost by dividing the tasks between the agents. However, it needs more iterations to be learned due to the requirement on the agents collaboration.
△ Less
Submitted 10 May, 2021;
originally announced May 2021.
-
Decentralized Microgrid Energy Management: A Multi-agent Correlated Q-learning Approach
Authors:
Hao Zhou,
Melike Erol-Kantarci
Abstract:
Microgrids (MG) are anticipated to be important players in the future smart grid. For proper operation of MGs an Energy Management System (EMS) is essential. The EMS of an MG could be rather complicated when renewable energy resources (RER), energy storage system (ESS) and demand side management (DSM) need to be orchestrated. Furthermore, these systems may belong to different entities and competit…
▽ More
Microgrids (MG) are anticipated to be important players in the future smart grid. For proper operation of MGs an Energy Management System (EMS) is essential. The EMS of an MG could be rather complicated when renewable energy resources (RER), energy storage system (ESS) and demand side management (DSM) need to be orchestrated. Furthermore, these systems may belong to different entities and competition may exist between them. Nash equilibrium is most commonly used for coordination of such entities however the convergence and existence of Nash equilibrium can not always be guaranteed. To this end, we use the correlated equilibrium to coordinate agents, whose convergence can be guaranteed. In this paper, we build an energy trading model based on mid-market rate, and propose a correlated Q-learning (CEQ) algorithm to maximize the revenue of each agent. Our results show that CEQ is able to balance the revenue of agents without harming total benefit. In addition, compared with Q-learning without correlation, CEQ could save 19.3% cost for the DSM agent and 44.2% more benefits for the ESS agent.
△ Less
Submitted 6 March, 2021;
originally announced March 2021.
-
Correlated Deep Q-learning based Microgrid Energy Management
Authors:
Hao Zhou,
Melike Erol-Kantarci
Abstract:
Microgrid (MG) energy management is an important part of MG operation. Various entities are generally involved in the energy management of an MG, e.g., energy storage system (ESS), renewable energy resources (RER) and the load of users, and it is crucial to coordinate these entities. Considering the significant potential of machine learning techniques, this paper proposes a correlated deep Q-learn…
▽ More
Microgrid (MG) energy management is an important part of MG operation. Various entities are generally involved in the energy management of an MG, e.g., energy storage system (ESS), renewable energy resources (RER) and the load of users, and it is crucial to coordinate these entities. Considering the significant potential of machine learning techniques, this paper proposes a correlated deep Q-learning (CDQN) based technique for the MG energy management. Each electrical entity is modeled as an agent which has a neural network to predict its own Q-values, after which the correlated Q-equilibrium is used to coordinate the operation among agents. In this paper, the Long Short Term Memory networks (LSTM) based deep Q-learning algorithm is introduced and the correlated equilibrium is proposed to coordinate agents. The simulation result shows 40.9% and 9.62% higher profit for ESS agent and photovoltaic (PV) agent, respectively.
△ Less
Submitted 6 March, 2021;
originally announced March 2021.
