Showing 1–7 of 7 results for author: Esteves, J J A

DRL-based Slice Placement under Realistic Network Load Conditions

Authors: José Jurandir Alves Esteves, Amina Boubendir, Fabrice Guillemin, Pierre Sens

Abstract: We propose to demonstrate a network slice placement optimization solution based on Deep Reinforcement Learning (DRL), referred to as Heuristically-controlled DRL, which uses a heuristic to control the DRL algorithm convergence. The solution is adapted to realistic networks with large scale and under non-stationary traffic conditions (namely, the network load). We demonstrate the applicability of t… ▽ More We propose to demonstrate a network slice placement optimization solution based on Deep Reinforcement Learning (DRL), referred to as Heuristically-controlled DRL, which uses a heuristic to control the DRL algorithm convergence. The solution is adapted to realistic networks with large scale and under non-stationary traffic conditions (namely, the network load). We demonstrate the applicability of the proposed solution and its higher and stable performance over a non-controlled DRL-based solution. Demonstration scenarios include full online learning with multiple volatile network slice placement request arrivals. △ Less

Submitted 27 September, 2021; originally announced September 2021.

Comments: arXiv admin note: substantial text overlap with arXiv:2010.08295

On the Robustness of Controlled Deep Reinforcement Learning for Slice Placement

Authors: Jose Jurandir Alves Esteves, Amina Boubendir, Fabrice Guillemin, Pierre Sens

Abstract: The evaluation of the impact of using Machine Learning in the management of softwarized networks is considered in multiple research works. Beyond that, we propose to evaluate the robustness of online learning for optimal network slice placement. A major assumption to this study is to consider that slice request arrivals are non-stationary. In this context, we simulate unpredictable network load va… ▽ More The evaluation of the impact of using Machine Learning in the management of softwarized networks is considered in multiple research works. Beyond that, we propose to evaluate the robustness of online learning for optimal network slice placement. A major assumption to this study is to consider that slice request arrivals are non-stationary. In this context, we simulate unpredictable network load variations and compare two Deep Reinforcement Learning (DRL) algorithms: a pure DRL-based algorithm and a heuristically controlled DRL as a hybrid DRL-heuristic algorithm, to assess the impact of these unpredictable changes of traffic load on the algorithms performance. We conduct extensive simulations of a large-scale operator infrastructure. The evaluation results show that the proposed hybrid DRL-heuristic approach is more robust and reliable in case of unpredictable network load changes than pure DRL as it reduces the performance degradation. These results are follow-ups for a series of recent research we have performed showing that the proposed hybrid DRL-heuristic approach is efficient and more adapted to real network scenarios than pure DRL. △ Less

Submitted 5 August, 2021; originally announced August 2021.

Comments: arXiv admin note: substantial text overlap with arXiv:2105.06741, arXiv:2108.02495

DRL-based Slice Placement Under Non-Stationary Conditions

Authors: Jose Jurandir Alves Esteves, Amina Boubendir, Fabrice Guillemin, Pierre Sens

Abstract: We consider online learning for optimal network slice placement under the assumption that slice requests arrive according to a non-stationary Poisson process. We propose a framework based on Deep Reinforcement Learning (DRL) combined with a heuristic to design algorithms. We specifically design two pure-DRL algorithms and two families of hybrid DRL-heuristic algorithms. To validate their performan… ▽ More We consider online learning for optimal network slice placement under the assumption that slice requests arrive according to a non-stationary Poisson process. We propose a framework based on Deep Reinforcement Learning (DRL) combined with a heuristic to design algorithms. We specifically design two pure-DRL algorithms and two families of hybrid DRL-heuristic algorithms. To validate their performance, we perform extensive simulations in the context of a large-scale operator infrastructure. The evaluation results show that the proposed hybrid DRL-heuristic algorithms require three orders of magnitude of learning episodes less than pure-DRL to achieve convergence. This result indicates that the proposed hybrid DRL-heuristic approach is more reliable than pure-DRL in a real non-stationary network scenario. △ Less

Submitted 5 August, 2021; originally announced August 2021.

Comments: arXiv admin note: text overlap with arXiv:2105.06741

Controlled Deep Reinforcement Learning for Optimized Slice Placement

Authors: Jose Jurandir Alves Esteves, Amina Boubendir, Fabrice Guillemin, Pierre Sens

Abstract: We present a hybrid ML-heuristic approach that we name "Heuristically Assisted Deep Reinforcement Learning (HA-DRL)" to solve the problem of Network Slice Placement Optimization. The proposed approach leverages recent works on Deep Reinforcement Learning (DRL) for slice placement and Virtual Network Embedding (VNE) and uses a heuristic function to optimize the exploration of the action space by gi… ▽ More We present a hybrid ML-heuristic approach that we name "Heuristically Assisted Deep Reinforcement Learning (HA-DRL)" to solve the problem of Network Slice Placement Optimization. The proposed approach leverages recent works on Deep Reinforcement Learning (DRL) for slice placement and Virtual Network Embedding (VNE) and uses a heuristic function to optimize the exploration of the action space by giving priority to reliable actions indicated by an efficient heuristic algorithm. The evaluation results show that the proposed HA-DRL algorithm can accelerate the learning of an efficient slice placement policy improving slice acceptance ratio when compared with state-of-the-art approaches that are based only on reinforcement learning. △ Less

Submitted 3 August, 2021; originally announced August 2021.

Comments: arXiv admin note: substantial text overlap with arXiv:2105.06741

A Heuristically Assisted Deep Reinforcement Learning Approach for Network Slice Placement

Authors: Jose Jurandir Alves Esteves, Amina Boubendir, Fabrice Guillemin, Pierre Sens

Abstract: Network Slice placement with the problem of allocation of resources from a virtualized substrate network is an optimization problem which can be formulated as a multiobjective Integer Linear Programming (ILP) problem. However, to cope with the complexity of such a continuous task and seeking for optimality and automation, the use of Machine Learning (ML) techniques appear as a promising approach.… ▽ More Network Slice placement with the problem of allocation of resources from a virtualized substrate network is an optimization problem which can be formulated as a multiobjective Integer Linear Programming (ILP) problem. However, to cope with the complexity of such a continuous task and seeking for optimality and automation, the use of Machine Learning (ML) techniques appear as a promising approach. We introduce a hybrid placement solution based on Deep Reinforcement Learning (DRL) and a dedicated optimization heuristic based on the Power of Two Choices principle. The DRL algorithm uses the so-called Asynchronous Advantage Actor Critic (A3C) algorithm for fast learning, and Graph Convolutional Networks (GCN) to automate feature extraction from the physical substrate network. The proposed Heuristically-Assisted DRL (HA-DRL) allows to accelerate the learning process and gain in resource usage when compared against other state-of-the-art approaches as the evaluation results evidence. △ Less

Submitted 14 May, 2021; originally announced May 2021.

Edge-enabled Optimized Network Slicing in Large Scale Networks

Authors: Jose Jurandir Alves Esteves, Amina Boubendir, Fabice Guillemin, Pierre Sens

Abstract: In this demo paper, we consider the network slice placement optimization problem and give some insights into a fast heuristic algorithm tailored to placement in large scale networks. We consider an online optimization scenario with multiple and volatile network slice request arrivals and we showcase the applicability of the proposed Edge-enabled network slice placement solution through a Proof-of-… ▽ More In this demo paper, we consider the network slice placement optimization problem and give some insights into a fast heuristic algorithm tailored to placement in large scale networks. We consider an online optimization scenario with multiple and volatile network slice request arrivals and we showcase the applicability of the proposed Edge-enabled network slice placement solution through a Proof-of-Concept illustrated by large scale networks scenarios. △ Less

Submitted 16 October, 2020; originally announced October 2020.

Journal ref: NoF 2020

Heuristic for Edge-enabled Network Slicing Optimization using the Power of Two Choices

Authors: Jose Jurandir Alves Esteves, Amina Boubendir, Fabrice Guillemin, Pierre Sens

Abstract: We propose an online heuristic algorithm for the problem of network slice placement optimization. The solution is adapted to support placement on large scale networks and integrates Edge-specific and URLLC constraints. We rely on an approach called the Power of Two Choices to build the heuristic. The evaluation results show the good performance of the heuristic that solves the problem in few secon… ▽ More We propose an online heuristic algorithm for the problem of network slice placement optimization. The solution is adapted to support placement on large scale networks and integrates Edge-specific and URLLC constraints. We rely on an approach called the Power of Two Choices to build the heuristic. The evaluation results show the good performance of the heuristic that solves the problem in few seconds under a large scale scenario. The heuristic also improves the acceptance ratio of network slice placement requests when compared against a deterministic online Integer Linear Programming (ILP) solution. △ Less

Submitted 16 October, 2020; originally announced October 2020.

Journal ref: CNSM 2020