-
LLM-Based Intent Processing and Network Optimization Using Attention-Based Hierarchical Reinforcement Learning
Authors:
Md Arafat Habib,
Pedro Enrique Iturria Rivera,
Yigit Ozcan,
Medhat Elsayed,
Majid Bavand,
Raimundus Gaigalas,
Melike Erol-Kantarci
Abstract:
Intent-based network automation is a promising tool to enable easier network management however certain challenges need to be effectively addressed. These are: 1) processing intents, i.e., identification of logic and necessary parameters to fulfill an intent, 2) validating an intent to align it with current network status, and 3) satisfying intents via network optimizing functions like xApps and r…
▽ More
Intent-based network automation is a promising tool to enable easier network management however certain challenges need to be effectively addressed. These are: 1) processing intents, i.e., identification of logic and necessary parameters to fulfill an intent, 2) validating an intent to align it with current network status, and 3) satisfying intents via network optimizing functions like xApps and rApps in O-RAN. This paper addresses these points via a three-fold strategy to introduce intent-based automation for O-RAN. First, intents are processed via a lightweight Large Language Model (LLM). Secondly, once an intent is processed, it is validated against future incoming traffic volume profiles (high or low). Finally, a series of network optimization applications (rApps and xApps) have been developed. With their machine learning-based functionalities, they can improve certain key performance indicators such as throughput, delay, and energy efficiency. In this final stage, using an attention-based hierarchical reinforcement learning algorithm, these applications are optimally initiated to satisfy the intent of an operator. Our simulations show that the proposed method can achieve at least 12% increase in throughput, 17.1% increase in energy efficiency, and 26.5% decrease in network delay compared to the baseline algorithms.
△ Less
Submitted 10 June, 2024;
originally announced June 2024.
-
Extended Reality (XR) Codec Adaptation in 5G using Multi-Agent Reinforcement Learning with Attention Action Selection
Authors:
Pedro Enrique Iturria-Rivera,
Raimundas Gaigalas,
Medhat Elsayed,
Majid Bavand,
Yigit Ozcan,
Melike Erol-Kantarci
Abstract:
Extended Reality (XR) services will revolutionize applications over 5th and 6th generation wireless networks by providing seamless virtual and augmented reality experiences. These applications impose significant challenges on network infrastructure, which can be addressed by machine learning algorithms due to their adaptability. This paper presents a Multi- Agent Reinforcement Learning (MARL) solu…
▽ More
Extended Reality (XR) services will revolutionize applications over 5th and 6th generation wireless networks by providing seamless virtual and augmented reality experiences. These applications impose significant challenges on network infrastructure, which can be addressed by machine learning algorithms due to their adaptability. This paper presents a Multi- Agent Reinforcement Learning (MARL) solution for optimizing codec parameters of XR traffic, comparing it to the Adjust Packet Size (APS) algorithm. Our cooperative multi-agent system uses an Optimistic Mixture of Q-Values (oQMIX) approach for handling Cloud Gaming (CG), Augmented Reality (AR), and Virtual Reality (VR) traffic. Enhancements include an attention mechanism and slate-Markov Decision Process (MDP) for improved action selection. Simulations show our solution outperforms APS with average gains of 30.1%, 15.6%, 16.5% 50.3% in XR index, jitter, delay, and Packet Loss Ratio (PLR), respectively. APS tends to increase throughput but also packet losses, whereas oQMIX reduces PLR, delay, and jitter while maintaining goodput.
△ Less
Submitted 24 May, 2024;
originally announced May 2024.
-
Transformer-Based Wireless Traffic Prediction and Network Optimization in O-RAN
Authors:
Md Arafat Habib,
Pedro Enrique Iturria-Rivera,
Yigit Ozcan,
Medhat Elsayed,
Majid Bavand,
Raimundus Gaigalas,
Melike Erol-Kantarci
Abstract:
This paper introduces an innovative method for predicting wireless network traffic in concise temporal intervals for Open Radio Access Networks (O-RAN) using a transformer architecture, which is the machine learning model behind generative AI tools. Depending on the anticipated traffic, the system either launches a reinforcement learning-based traffic steering xApp or a cell slee** rApp to enhan…
▽ More
This paper introduces an innovative method for predicting wireless network traffic in concise temporal intervals for Open Radio Access Networks (O-RAN) using a transformer architecture, which is the machine learning model behind generative AI tools. Depending on the anticipated traffic, the system either launches a reinforcement learning-based traffic steering xApp or a cell slee** rApp to enhance performance metrics like throughput or energy efficiency. Our simulation results demonstrate that the proposed traffic prediction-based network optimization mechanism matches the performance of standalone RAN applications (rApps/ xApps) that are always on during the whole simulation time while offering on-demand activation. This feature is particularly advantageous during instances of abrupt fluctuations in traffic volume. Rather than persistently operating specific applications irrespective of the actual incoming traffic conditions, the proposed prediction-based method increases the average energy efficiency by 39.7% compared to the "Always on Traffic Steering xApp" and achieves 10.1% increase in throughput compared to the "Always on Cell Slee** rApp". The simulation has been conducted over 24 hours, emulating a whole day traffic pattern for a dense urban area.
△ Less
Submitted 16 March, 2024;
originally announced March 2024.
-
Federated Learning with Dual Attention for Robust Modulation Classification under Attacks
Authors:
Han Zhang,
Medhat Elsayed,
Majid Bavand,
Raimundas Gaigalas,
Yigit Ozcan,
Melike Erol-Kantarci
Abstract:
Federated learning (FL) allows distributed participants to train machine learning models in a decentralized manner. It can be used for radio signal classification with multiple receivers due to its benefits in terms of privacy and scalability. However, the existing FL algorithms usually suffer from slow and unstable convergence and are vulnerable to poisoning attacks from malicious participants. I…
▽ More
Federated learning (FL) allows distributed participants to train machine learning models in a decentralized manner. It can be used for radio signal classification with multiple receivers due to its benefits in terms of privacy and scalability. However, the existing FL algorithms usually suffer from slow and unstable convergence and are vulnerable to poisoning attacks from malicious participants. In this work, we aim to design a versatile FL framework that simultaneously promotes the performance of the model both in a secure system and under attack. To this end, we leverage attention mechanisms as a defense against attacks in FL and propose a robust FL algorithm by integrating the attention mechanisms into the global model aggregation step. To be more specific, two attention models are combined to calculate the amount of attention cast on each participant. It will then be used to determine the weights of local models during the global aggregation. The proposed algorithm is verified on a real-world dataset and it outperforms existing algorithms, both in secure systems and in systems under data poisoning attacks.
△ Less
Submitted 19 January, 2024;
originally announced January 2024.
-
Distributed Attacks over Federated Reinforcement Learning-enabled Cell Sleep Control
Authors:
Han Zhang,
Hao Zhou,
Medhat Elsayed,
Majid Bavand,
Raimundas Gaigalas,
Yigit Ozcan,
Melike Erol-Kantarci
Abstract:
Federated learning (FL) is particularly useful in wireless networks due to its distributed implementation and privacy-preserving features. However, as a distributed learning system, FL can be vulnerable to malicious attacks from both internal and external sources. Our work aims to investigate the attack models in a FL-enabled wireless networks. Specifically, we consider a cell sleep control scenar…
▽ More
Federated learning (FL) is particularly useful in wireless networks due to its distributed implementation and privacy-preserving features. However, as a distributed learning system, FL can be vulnerable to malicious attacks from both internal and external sources. Our work aims to investigate the attack models in a FL-enabled wireless networks. Specifically, we consider a cell sleep control scenario, and apply federated reinforcement learning to improve energy-efficiency. We design three attacks, namely free rider attacks, Byzantine data poisoning attacks and backdoor attacks. The simulation results show that the designed attacks can degrade the network performance and lead to lower energy-efficiency. Moreover, we also explore possible ways to mitigate the above attacks. We design a defense model called refined-Krum to defend against attacks by enabling a secure aggregation on the global server. The proposed refined- Krum scheme outperforms the existing Krum scheme and can effectively prevent wireless networks from malicious attacks, improving the system energy-efficiency performance.
△ Less
Submitted 27 November, 2023;
originally announced November 2023.
-
Intent-driven Intelligent Control and Orchestration in O-RAN Via Hierarchical Reinforcement Learning
Authors:
Md Arafat Habib,
Hao Zhou,
Pedro Enrique Iturria-Rivera,
Medhat Elsayed,
Majid Bavand,
Raimundas Gaigalas,
Yigit Ozcan,
Melike Erol-Kantarci
Abstract:
rApps and xApps need to be controlled and orchestrated well in the open radio access network (O-RAN) so that they can deliver a guaranteed network performance in a complex multi-vendor environment. This paper proposes a novel intent-driven intelligent control and orchestration scheme based on hierarchical reinforcement learning (HRL). The proposed scheme can orchestrate multiple rApps or xApps acc…
▽ More
rApps and xApps need to be controlled and orchestrated well in the open radio access network (O-RAN) so that they can deliver a guaranteed network performance in a complex multi-vendor environment. This paper proposes a novel intent-driven intelligent control and orchestration scheme based on hierarchical reinforcement learning (HRL). The proposed scheme can orchestrate multiple rApps or xApps according to the operator's intent of optimizing certain key performance indicators (KPIs), such as throughput, energy efficiency, and latency. Specifically, we propose a bi-level architecture with a meta-controller and a controller. The meta-controller provides the target performance in terms of KPIs, while the controller performs xApp orchestration at the lower level. Our simulation results show that the proposed HRL-based intent-driven xApp orchestration mechanism achieves 7.5% and 21.4% increase in average system throughput with respect to two baselines, i.e., a single xApp baseline and a non-machine learning-based algorithm, respectively. Similarly, 17.3% and 37.9% increase in energy efficiency are observed in comparison to the same baselines.
△ Less
Submitted 5 July, 2023;
originally announced July 2023.
-
On-Device Intelligence for 5G RAN: Knowledge Transfer and Federated Learning enabled UE-Centric Traffic Steering
Authors:
Han Zhang,
Hao Zhou,
Medhat Elsayed,
Majid Bavand,
Raimundas Gaigalas,
Yigit Ozcan,
Melike Erol-Kantarci
Abstract:
Traffic steering (TS) is a promising approach to support various service requirements and enhance transmission reliability by distributing network traffic loads to appropriate base stations (BSs). In conventional cell-centric TS strategies, BSs make TS decisions for all user equipment (UEs) in a centralized manner, which focuses more on the overall performance of the whole cell, disregarding speci…
▽ More
Traffic steering (TS) is a promising approach to support various service requirements and enhance transmission reliability by distributing network traffic loads to appropriate base stations (BSs). In conventional cell-centric TS strategies, BSs make TS decisions for all user equipment (UEs) in a centralized manner, which focuses more on the overall performance of the whole cell, disregarding specific requirements of individual UE. The flourishing machine learning technologies and evolving UE-centric 5G network architecture have prompted the emergence of new TS technologies. In this paper, we propose a knowledge transfer and federated learning-enabled UE-centric (KT-FLUC) TS framework for highly dynamic 5G radio access networks (RAN). Specifically, first, we propose an attention-weighted group federated learning scheme. It enables intelligent UEs to make TS decisions autonomously using local models and observations, and a global model is defined to coordinate local TS decisions and share experiences among UEs. Secondly, considering the individual UE's limited computation and energy resources, a growing and pruning-based model compression method is introduced, mitigating the computation burden of UEs and reducing the communication overhead of federated learning. In addition, we propose a Q-value-based knowledge transfer method to initialize newcomer UEs, achieving a jump start for their training efficiency. Finally, the simulations show that our proposed KT-FLUC algorithm can effectively improve the service quality, achieving 65\% and 38\% lower delay and 52% and 57% higher throughput compared with cell-based TS and other UE-centric TS strategies, respectively.
△ Less
Submitted 28 November, 2023; v1 submitted 21 April, 2023;
originally announced April 2023.
-
Hierarchical Reinforcement Learning Based Traffic Steering in Multi-RAT 5G Deployments
Authors:
Md Arafat Habib,
Hao Zhou,
Pedro Enrique Iturria-Rivera,
Medhat Elsayed,
Majid Bavand,
Raimundas Gaigalas,
Yigit Ozcan,
Melike Erol-Kantarci
Abstract:
In 5G non-standalone mode, an intelligent traffic steering mechanism can vastly aid in ensuring smooth user experience by selecting the best radio access technology (RAT) from a multi-RAT environment for a specific traffic flow. In this paper, we propose a novel load-aware traffic steering algorithm based on hierarchical reinforcement learning (HRL) while satisfying diverse QoS requirements of dif…
▽ More
In 5G non-standalone mode, an intelligent traffic steering mechanism can vastly aid in ensuring smooth user experience by selecting the best radio access technology (RAT) from a multi-RAT environment for a specific traffic flow. In this paper, we propose a novel load-aware traffic steering algorithm based on hierarchical reinforcement learning (HRL) while satisfying diverse QoS requirements of different traffic types. HRL can significantly increase system performance using a bi-level architecture having a meta-controller and a controller. In our proposed method, the meta-controller provides an appropriate threshold for load balancing, while the controller performs traffic admission to an appropriate RAT in the lower level. Simulation results show that HRL outperforms a Deep Q-Learning (DQN) and a threshold-based heuristic baseline with 8.49%, 12.52% higher average system throughput and 27.74%, 39.13% lower network delay, respectively.
△ Less
Submitted 18 January, 2023;
originally announced January 2023.
-
Reconfigurable Intelligent Surfaces for the Connectivity of Autonomous Vehicles
Authors:
Y. Ugur Ozcan,
Ozgur Ozdemir,
Gunes Karabulut Kurt
Abstract:
The use of real-time software-controlled reconfigurable intelligent surface (RIS) units is proposed to increase the reliability of vehicle-to-everything (V2X) communications. The optimum placement problem of the RIS units is formulated by considering their sizes and operating modes. The solution of the problem is given, where it is shown that the placement of the RIS depends on the locations of th…
▽ More
The use of real-time software-controlled reconfigurable intelligent surface (RIS) units is proposed to increase the reliability of vehicle-to-everything (V2X) communications. The optimum placement problem of the RIS units is formulated by considering their sizes and operating modes. The solution of the problem is given, where it is shown that the placement of the RIS depends on the locations of the transmitter and the receiver. The proposed RIS-supported highway deployment can combat the high path loss experienced by the use of higher frequency bands, including the millimeter-wave and the terahertz bands, that are expected to be used in the next-generation wireless networks, enabling the use of the existing base station deployment plans to remain operational, while providing reliable and energy-efficient connectivity for autonomous vehicles.
△ Less
Submitted 20 July, 2020;
originally announced July 2020.
-
Map** Firms' Locations in Technological Space: A Topological Analysis of Patent Statistics
Authors:
Emerson G. Escolar,
Yasuaki Hiraoka,
Mitsuru Igami,
Yasin Ozcan
Abstract:
Where do firms innovate? Map** their locations and directions in technological space is challenging due to its high dimensionality. We propose a new method to characterize firms' inventive activities via topological data analysis (TDA) that represents high-dimensional data in a shape graph. Applying this method to 333 major firms' patents in 1976--2005 reveals substantial heterogeneity: some fir…
▽ More
Where do firms innovate? Map** their locations and directions in technological space is challenging due to its high dimensionality. We propose a new method to characterize firms' inventive activities via topological data analysis (TDA) that represents high-dimensional data in a shape graph. Applying this method to 333 major firms' patents in 1976--2005 reveals substantial heterogeneity: some firms remain undifferentiated; others develop unique portfolios. Firms with unique trajectories, which we define and measure graph-theoretically as "flares" in the Mapper graph, perform better. This association is statistically and economically significant, and continues to hold after we control for portfolio size, firm survivorship, industry classification, and firm fixed effects. By contrast, existing techniques -- such as principal component analysis (PCA) and Jaffe's (1989) clustering method -- struggle to track these firm-level dynamics.
△ Less
Submitted 31 March, 2022; v1 submitted 31 August, 2019;
originally announced September 2019.
