-
Online Frequency Scheduling by Learning Parallel Actions
Authors:
Anastasios Giovanidis,
Mathieu Leconte,
Sabrine Aroua,
Tor Kvernvik,
David Sandberg
Abstract:
Radio Resource Management is a challenging topic in future 6G networks where novel applications create strong competition among the users for the available resources. In this work we consider the frequency scheduling problem in a multi-user MIMO system. Frequency resources need to be assigned to a set of users while allowing for concurrent transmissions in the same sub-band. Traditional methods ar…
▽ More
Radio Resource Management is a challenging topic in future 6G networks where novel applications create strong competition among the users for the available resources. In this work we consider the frequency scheduling problem in a multi-user MIMO system. Frequency resources need to be assigned to a set of users while allowing for concurrent transmissions in the same sub-band. Traditional methods are insufficient to cope with all the involved constraints and uncertainties, whereas reinforcement learning can directly learn near-optimal solutions for such complex environments. However, the scheduling problem has an enormous action space accounting for all the combinations of users and sub-bands, so out-of-the-box algorithms cannot be used directly. In this work, we propose a scheduler based on action-branching over sub-bands, which is a deep Q-learning architecture with parallel decision capabilities. The sub-bands learn correlated but local decision policies and altogether they optimize a global reward. To improve the scaling of the architecture with the number of sub-bands, we propose variations (Unibranch, Graph Neural Network-based) that reduce the number of parameters to learn. The parallel decision making of the proposed architecture allows to meet short inference time requirements in real systems. Furthermore, the deep Q-learning approach permits online fine-tuning after deployment to bridge the sim-to-real gap. The proposed architectures are evaluated against relevant baselines from the literature showing competitive performance and possibilities of online adaptation to evolving environments.
△ Less
Submitted 7 June, 2024;
originally announced June 2024.
-
Mobility, traffic and radio channel prediction: 5G and beyond applications
Authors:
Henrik Rydén,
Alex Palaios,
László Hévizi,
David Sandberg,
Tor Kvernvik,
Hamed Farhadi
Abstract:
Machine learning (ML) is an important component for enabling automation in Radio Access Networks (RANs). The work on applying ML for RAN has been under development for many years and is now also drawing attention in 3GPP and Open-RAN standardization fora. A key component of multiple features, also highlighted in the recent 3GPP specification work, is the use of mobility, traffic and radio channel…
▽ More
Machine learning (ML) is an important component for enabling automation in Radio Access Networks (RANs). The work on applying ML for RAN has been under development for many years and is now also drawing attention in 3GPP and Open-RAN standardization fora. A key component of multiple features, also highlighted in the recent 3GPP specification work, is the use of mobility, traffic and radio channel prediction. These types of predictions form the intelligence enablers to leverage the potentials for ML for RAN, both for current and future wireless networks. This paper provides an overview with evaluation results of current applications that utilize such intelligence enablers, we then discuss how those enablers likely will be a cornerstone for emerging 6G use cases such as wireless energy transmission.
△ Less
Submitted 15 March, 2022;
originally announced March 2022.
-
Learning Robust Scheduling with Search and Attention
Authors:
David Sandberg,
Tor Kvernvik,
Francesco Davide Calabrese
Abstract:
Allocating physical layer resources to users based on channel quality, buffer size, requirements and constraints represents one of the central optimization problems in the management of radio resources. The solution space grows combinatorially with the cardinality of each dimension making it hard to find optimal solutions using an exhaustive search or even classical optimization algorithms given t…
▽ More
Allocating physical layer resources to users based on channel quality, buffer size, requirements and constraints represents one of the central optimization problems in the management of radio resources. The solution space grows combinatorially with the cardinality of each dimension making it hard to find optimal solutions using an exhaustive search or even classical optimization algorithms given the stringent time requirements. This problem is even more pronounced in MU-MIMO scheduling where the scheduler can assign multiple users to the same time-frequency physical resources. Traditional approaches thus resort to designing heuristics that trade optimality in favor of feasibility of execution. In this work we treat the MU-MIMO scheduling problem as a tree-structured combinatorial problem and, borrowing from the recent successes of AlphaGo Zero, we investigate the feasibility of searching for the best performing solutions using a combination of Monte Carlo Tree Search and Reinforcement Learning. To cater to the nature of the problem at hand, like the lack of an intrinsic ordering of the users as well as the importance of dependencies between combinations of users, we make fundamental modifications to the neural network architecture by introducing the self-attention mechanism. We then demonstrate that the resulting approach is not only feasible but vastly outperforms state-of-the-art heuristic-based scheduling approaches in the presence of measurement uncertainties and finite buffers.
△ Less
Submitted 15 November, 2021;
originally announced November 2021.
