Showing 1–19 of 19 results for author: Barrachina-Munoz, S

X-GRL: An Empirical Assessment of Explainable GNN-DRL in B5G/6G Networks

Authors: Farhad Rezazadeh, Sergio Barrachina-MuNoz, Engin Zeydan, Houbing Song, K. P. Subbalakshmi, Josep Mangues-Bafalluy

Abstract: The rapid development of artificial intelligence (AI) techniques has triggered a revolution in beyond fifth-generation (B5G) and upcoming sixth-generation (6G) mobile networks. Despite these advances, efficient resource allocation in dynamic and complex networks remains a major challenge. This paper presents an experimental implementation of deep reinforcement learning (DRL) enhanced with graph ne… ▽ More The rapid development of artificial intelligence (AI) techniques has triggered a revolution in beyond fifth-generation (B5G) and upcoming sixth-generation (6G) mobile networks. Despite these advances, efficient resource allocation in dynamic and complex networks remains a major challenge. This paper presents an experimental implementation of deep reinforcement learning (DRL) enhanced with graph neural networks (GNNs) on a real 5G testbed. The method addresses the explainability of GNNs by evaluating the importance of each edge in determining the model's output. The custom sampling functions feed the data into the proposed GNN-driven Monte Carlo policy gradient (REINFORCE) agent to optimize the gNodeB (gNB) radio resources according to the specific traffic demands. The demo demonstrates real-time visualization of network parameters and superior performance compared to benchmarks. △ Less

Submitted 15 November, 2023; originally announced November 2023.

Comments: 3 pages, 8 figures

A Multi-Agent Deep Reinforcement Learning Approach for RAN Resource Allocation in O-RAN

Authors: Farhad Rezazadeh, Lanfranco Zanzi, Francesco Devoti, Sergio Barrachina-Munoz, Engin Zeydan, Xavier Costa-Pérez, Josep Mangues-Bafalluy

Abstract: Artificial intelligence (AI) and Machine Learning (ML) are considered as key enablers for realizing the full potential of fifth-generation (5G) and beyond mobile networks, particularly in the context of resource management and orchestration. In this demonstration, we consider a fully-fledged 5G mobile network and develop a multi-agent deep reinforcement learning (DRL) framework for RAN resource al… ▽ More Artificial intelligence (AI) and Machine Learning (ML) are considered as key enablers for realizing the full potential of fifth-generation (5G) and beyond mobile networks, particularly in the context of resource management and orchestration. In this demonstration, we consider a fully-fledged 5G mobile network and develop a multi-agent deep reinforcement learning (DRL) framework for RAN resource allocation. By leveraging local monitoring information generated by a shared gNodeB instance (gNB), each DRL agent aims to optimally allocate radio resources concerning service-specific traffic demands belonging to heterogeneous running services. We perform experiments on the deployed testbed in real-time, showing that DRL-based agents can allocate radio resources fairly while improving the overall efficiency of resource utilization and minimizing the risk of over provisioning. △ Less

Submitted 3 July, 2023; originally announced July 2023.

Comments: 2 pages, 3 figures

Mobile Edge Vertical Applications Using ETSI MEC APIs and Sandbox

Authors: Rasoul Nikbakht, Michail Dalgitsis, Sergio Barrachina-Muñoz, Sarang Kahvazadeh

Abstract: MEC Sandbox is an excellent tool that simulates wireless networks and deploys ETSI Multi-access Edge Computing (MEC) APIs on top of the simulated wireless network. In this demo, we consume these APIs using a decision engine (DE) to scale a video-on-demand (VoD) application located on the network edge, assuming that the average number of users is a good proxy of the demand. Specifically, the develo… ▽ More MEC Sandbox is an excellent tool that simulates wireless networks and deploys ETSI Multi-access Edge Computing (MEC) APIs on top of the simulated wireless network. In this demo, we consume these APIs using a decision engine (DE) to scale a video-on-demand (VoD) application located on the network edge, assuming that the average number of users is a good proxy of the demand. Specifically, the developed DE uses the ETSI MEC Location API and retrieves the number of users in a given zone. The DE then takes actions at the microservice scaling level and executes them through a custom-made Kubernetes-based OpenAPI. △ Less

Submitted 25 November, 2022; originally announced November 2022.

Comments: A one-page version of this paper is accepted in the 2022 IEEE Conference on Standards for Communications and Networking(CSCN)-Demo submissions

Intent-Based Orchestration for Application Relocation in a 5G Cloud-native Platform

Authors: Sergio Barrachina-Muñoz, Jorge Baranda, Miquel Payaró, Josep Mangues-Bafalluy

Abstract: The need of mobile network operators for cost-effectiveness is driving 5G and beyond networks towards highly flexible and agile deployments to adapt to dynamic and resource-constrained scenarios while meeting a myriad of user network stakeholders' requirements. In this setting, we consider that zero-touch orchestration schemes based on cloud-native deployments equipped with end-to-end monitoring c… ▽ More The need of mobile network operators for cost-effectiveness is driving 5G and beyond networks towards highly flexible and agile deployments to adapt to dynamic and resource-constrained scenarios while meeting a myriad of user network stakeholders' requirements. In this setting, we consider that zero-touch orchestration schemes based on cloud-native deployments equipped with end-to-end monitoring capabilities provide the necessary technology mix to be a solution candidate. This demonstration, built on top of an end-to-end cloud-native 5G experimental platform with over-the-air transmissions, shows how dynamic orchestration can relocate container-based end-user applications to fulfil intent-based requirements. Accordingly, we provide an experimental validation to showcase how the platform enables the desired flexible and agile 5G deployments. △ Less

Submitted 21 November, 2022; originally announced November 2022.

Cloud-native 5G experimental platform with over-the-air transmissions and end-to-end monitoring

Authors: Sergio Barrachina-Muñoz, Miquel Payaró, Josep Mangues-Bafalluy

Abstract: 5G represents a revolutionary shift with respect to previous generations given its design centered on network softwarization. Within such a change of paradigm, cloud-native solutions are widely regarded as the future of vertical application development because of their enhanced flexibility and adaptability to complex and dynamic scenarios. In this context, we present an experimental framework with… ▽ More 5G represents a revolutionary shift with respect to previous generations given its design centered on network softwarization. Within such a change of paradigm, cloud-native solutions are widely regarded as the future of vertical application development because of their enhanced flexibility and adaptability to complex and dynamic scenarios. In this context, we present an experimental framework with over-the-air transmissions that tackles two critical aspects for enhancing the lifecycle management of 5G and beyond networks: cloud-native deployments of 5G core network functions (NFs) and end-to-end monitoring. First, we deploy Open5GS and Prometheus-based monitoring as containerized network functions (CNFs) in a Kubernetes cluster spanning a multi-tier network with a multi-access edge computing (MEC) host. We then demonstrate the end-to-end monitoring system by showcasing via Grafana dashboards both infrastructure resources and radio metrics of two scenarios; one devoted to user plane function (UPF) re-selection and the other to user mobility. △ Less

Submitted 25 July, 2022; originally announced July 2022.

End-to-End Latency Analysis and Optimal Block Size of Proof-of-Work Blockchain Applications

Authors: Francesc Wilhelmi, Sergio Barrachina-Muñoz, Paolo Dini

Abstract: Due to the increasing interest in blockchain technology for fostering secure, auditable, decentralized applications, a set of challenges associated with this technology need to be addressed. In this letter, we focus on the delay associated with Proof-of-Work (PoW)-based blockchain networks, whereby participants validate the new information to be appended to a distributed ledger via consensus to co… ▽ More Due to the increasing interest in blockchain technology for fostering secure, auditable, decentralized applications, a set of challenges associated with this technology need to be addressed. In this letter, we focus on the delay associated with Proof-of-Work (PoW)-based blockchain networks, whereby participants validate the new information to be appended to a distributed ledger via consensus to confirm transactions. We propose a novel end-to-end latency model based on batch-service queuing theory that characterizes timers and forks for the first time. Furthermore, we derive an estimation of optimum block size analytically. Endorsed by simulation results, we show that the optimal block size approximation is a consistent method that leads to close-to-optimal performance by significantly reducing the overheads associated with blockchain applications. △ Less

Submitted 3 February, 2022; originally announced February 2022.

Stateless Reinforcement Learning for Multi-Agent Systems: the Case of Spectrum Allocation in Dynamic Channel Bonding WLANs

Authors: Sergio Barrachina-Muñoz, Alessandro Chiumento, Boris Bellalta

Abstract: Spectrum allocation in the form of primary channel and bandwidth selection is a key factor for dynamic channel bonding (DCB) wireless local area networks (WLANs). To cope with varying environments, where networks change their configurations on their own, the wireless community is looking towards solutions aided by machine learning (ML), and especially reinforcement learning (RL) given its trial-an… ▽ More Spectrum allocation in the form of primary channel and bandwidth selection is a key factor for dynamic channel bonding (DCB) wireless local area networks (WLANs). To cope with varying environments, where networks change their configurations on their own, the wireless community is looking towards solutions aided by machine learning (ML), and especially reinforcement learning (RL) given its trial-and-error approach. However, strong assumptions are normally made to let complex RL models converge to near-optimal solutions. Our goal with this paper is two-fold: justify in a comprehensible way why RL should be the approach for wireless networks problems like decentralized spectrum allocation, and call into question whether the use of complex RL algorithms helps the quest of rapid learning in realistic scenarios. We derive that stateless RL in the form of lightweight multi-armed-bandits (MABs) is an efficient solution for rapid adaptation avoiding the definition of extensive or meaningless RL states. △ Less

Submitted 10 June, 2021; originally announced June 2021.

Wi-Fi All-Channel Analyzer

Authors: Sergio Barrachina-Muñoz, Boris Bellalta, Edward Knightly

Abstract: In this paper, we present WACA, the first system to simultaneously measure the energy in all 24 Wi-Fi channels that allow channel bonding at 5 GHz with microsecond scale granularity. With WACA, we perform a first-of-its-kind measurement campaign in areas including urban hotspots, residential neighborhoods, universities, and a sold-out stadium with 98,000 fans and 12,000 simultaneous Wi-Fi connecti… ▽ More In this paper, we present WACA, the first system to simultaneously measure the energy in all 24 Wi-Fi channels that allow channel bonding at 5 GHz with microsecond scale granularity. With WACA, we perform a first-of-its-kind measurement campaign in areas including urban hotspots, residential neighborhoods, universities, and a sold-out stadium with 98,000 fans and 12,000 simultaneous Wi-Fi connections. The gathered dataset is a unique asset to find insights otherwise not possible in the context of multi-channel technologies like Wi-Fi. To show its potential, we compare the performance of contiguous and non-contiguous channel bonding using a trace-driven framework. We show that while non-contiguous outperforms contiguous channel bonding's throughput, occasionally bigger by a factor of 5, their average throughputs are similar. △ Less

Submitted 27 August, 2020; originally announced August 2020.

A Flexible Machine Learning-Aware Architecture for Future WLANs

Authors: Francesc Wilhelmi, Sergio Barrachina-Muñoz, Boris Bellalta, Cristina Cano, Anders Jonsson, Vishnu Ram

Abstract: Lots of hopes have been placed on Machine Learning (ML) as a key enabler of future wireless networks. By taking advantage of large volumes of data, ML is expected to deal with the ever-increasing complexity of networking problems. Unfortunately, current networks are not yet prepared to support the ensuing requirements of ML-based applications in terms of data collection, processing, and output dis… ▽ More Lots of hopes have been placed on Machine Learning (ML) as a key enabler of future wireless networks. By taking advantage of large volumes of data, ML is expected to deal with the ever-increasing complexity of networking problems. Unfortunately, current networks are not yet prepared to support the ensuing requirements of ML-based applications in terms of data collection, processing, and output distribution. This article points out the architectural requirements that are needed to pervasively include ML as part of future wireless networks operation. Specifically, we look into Wireless Local Area Networks (WLANs), which, due to their nature can be found in multiple forms, ranging from cloud-based to edge-computing-like deployments. In particular, we propose to adopt the International Telecommunications Union (ITU) unified architecture for 5G and beyond. Based on ITU's architecture, we provide insights on the main requirements and the major challenges of introducing ML to the multiple modalities of WLANs. Finally, we showcase the superiority of the architecture through an ML-enabled use case for future networks. △ Less

Submitted 17 February, 2020; v1 submitted 8 October, 2019; originally announced October 2019.

Online Primary Channel Selection for Dynamic Channel Bonding in High-Density WLANs

Authors: Sergio Barrachina-Muñoz, Francesc Wilhelmi, Boris Bellalta

Abstract: In order to dynamically adapt the transmission bandwidth in wireless local area networks (WLANs), dynamic channel bonding (DCB) was introduced in IEEE 802.11n. It has been extended since then, and it is expected to be a key element in IEEE 802.11ax and future amendments such as IEEE 802.11be. While DCB is proven to be a compelling mechanism by itself, its performance is deeply tied to the primary… ▽ More In order to dynamically adapt the transmission bandwidth in wireless local area networks (WLANs), dynamic channel bonding (DCB) was introduced in IEEE 802.11n. It has been extended since then, and it is expected to be a key element in IEEE 802.11ax and future amendments such as IEEE 802.11be. While DCB is proven to be a compelling mechanism by itself, its performance is deeply tied to the primary channel selection, especially in high-density (HD) deployments, where multiple nodes contend for the spectrum. Traditionally, this primary channel selection relied on picking the most free one without any further consideration. In this paper, in contrast, we propose dynamic-wise (DyWi), a light-weight, decentralized, online primary channel selection algorithm for DCB that maximizes the expected WLAN throughput by considering not only the occupancy of the target primary channel but also the activity of the secondary channels. Even when assuming important delay costs due to primary switching, simulation results show a significant improvement both in terms of average delay and throughput. △ Less

Submitted 1 July, 2019; originally announced July 2019.

On the Performance of the Spatial Reuse Operation in IEEE 802.11ax WLANs

Authors: Francesc Wilhelmi, Sergio Barrachina-Muñoz, Boris Bellalta

Abstract: The Spatial Reuse (SR) operation included in the IEEE 802.11ax-2020 (11ax) amendment aims at increasing the number of parallel transmissions in an Overlap** Basic Service Set (OBSS). However, many unknowns exist about the performance gains that can be achieved through SR. In this paper, we provide a brief introduction to the SR operation described in the IEEE 802.11ax (draft D4.0). Then, a simul… ▽ More The Spatial Reuse (SR) operation included in the IEEE 802.11ax-2020 (11ax) amendment aims at increasing the number of parallel transmissions in an Overlap** Basic Service Set (OBSS). However, many unknowns exist about the performance gains that can be achieved through SR. In this paper, we provide a brief introduction to the SR operation described in the IEEE 802.11ax (draft D4.0). Then, a simulation-based implementation is provided in order to explore the performance gains of the SR operation. Our results show the potential of using SR in different scenarios covering multiple network densities and traffic loads. In particular, we observe significant improvements on the channel utilization when applying SR with respect to the default configuration, thus allowing to increase the throughput and reduce the delay. Interestingly, the highest improvements provided by the SR operation are observed in the most pessimistic situations in terms of network density and traffic load. △ Less

Submitted 17 September, 2019; v1 submitted 19 June, 2019; originally announced June 2019.

Komondor: a Wireless Network Simulator for Next-Generation High-Density WLANs

Authors: Sergio Barrachina-Muñoz, Francesc Wilhelmi, Ioannis Selinis, Boris Bellalta

Abstract: Komondor is a wireless network simulator for next-generation wireless local area networks (WLANs). The simulator has been conceived as an accessible (ready-to-use) open source tool for research on wireless networks and academia. An important advantage of Komondor over other well-known wireless simulators lies in its high event processing rate, which is furnished by the simplification of the core o… ▽ More Komondor is a wireless network simulator for next-generation wireless local area networks (WLANs). The simulator has been conceived as an accessible (ready-to-use) open source tool for research on wireless networks and academia. An important advantage of Komondor over other well-known wireless simulators lies in its high event processing rate, which is furnished by the simplification of the core operation. This allows outperforming the execution time of other simulators like ns-3, thus supporting large-scale scenarios with a huge number of nodes. In this paper, we provide insights into the Komondor simulator and overview its main features, development stages and use cases. The operation of Komondor is validated in a variety of scenarios against different tools: the ns-3 simulator and two analytical tools based on Continuous Time Markov Networks (CTMNs) and the Bianchi's DCF model. Results show that Komondor captures the IEEE 802.11 operation very similarly to ns-3. Finally, we discuss the potential of Komondor for simulating complex environments -- even with machine learning support -- in next-generation WLANs by easily develo** new user-defined modules of code △ Less

Submitted 14 February, 2019; v1 submitted 29 November, 2018; originally announced November 2018.

Potential and Pitfalls of Multi-Armed Bandits for Decentralized Spatial Reuse in WLANs

Authors: Francesc Wilhelmi, Sergio Barrachina-Muñoz, Cristina Cano, Boris Bellalta, Anders Jonsson, Gergely Neu

Abstract: Spatial Reuse (SR) has recently gained attention to maximize the performance of IEEE 802.11 Wireless Local Area Networks (WLANs). Decentralized mechanisms are expected to be key in the development of SR solutions for next-generation WLANs, since many deployments are characterized by being uncoordinated by nature. However, the potential of decentralized mechanisms is limited by the significant lack… ▽ More Spatial Reuse (SR) has recently gained attention to maximize the performance of IEEE 802.11 Wireless Local Area Networks (WLANs). Decentralized mechanisms are expected to be key in the development of SR solutions for next-generation WLANs, since many deployments are characterized by being uncoordinated by nature. However, the potential of decentralized mechanisms is limited by the significant lack of knowledge with respect to the overall wireless environment. To shed some light on this subject, we show the main considerations and possibilities of applying online learning to address the SR problem in uncoordinated WLANs. In particular, we provide a solution based on Multi-Armed Bandits (MABs) whereby independent WLANs dynamically adjust their frequency channel, transmit power and sensitivity threshold. To that purpose, we provide two different strategies, which refer to selfish and environment-aware learning. While the former stands for pure individual behavior, the second one considers the performance experienced by surrounding networks, thus taking into account the impact of individual actions on the environment. Through these two strategies we delve into practical issues of applying MABs in wireless networks, such as convergence guarantees or adversarial effects. Our simulation results illustrate the potential of the proposed solutions for enabling SR in future WLANs. We show that substantial improvements on network performance can be achieved regarding throughput and fairness. △ Less

Submitted 14 December, 2018; v1 submitted 28 May, 2018; originally announced May 2018.

Combining Software Defined Networks and Machine Learning to enable Self Organizing WLANs

Authors: Álvaro López-Raventós, Francesc Wilhelmi, Sergio Barrachina-Muñoz, Boris Bellalta

Abstract: Next generation of wireless local area networks (WLANs) will operate in dense, chaotic and highly dynamic scenarios that in a significant number of cases may result in a low user experience due to uncontrolled high interference levels. Flexible network architectures, such as the software-defined networking (SDN) paradigm, will provide WLANs with new capabilities to deal with users' demands, while… ▽ More Next generation of wireless local area networks (WLANs) will operate in dense, chaotic and highly dynamic scenarios that in a significant number of cases may result in a low user experience due to uncontrolled high interference levels. Flexible network architectures, such as the software-defined networking (SDN) paradigm, will provide WLANs with new capabilities to deal with users' demands, while achieving greater levels of efficiency and flexibility in those complex scenarios. On top of SDN, the use of machine learning (ML) techniques may improve network resource usage and management by identifying feasible configurations through learning. ML techniques can drive WLANs to reach optimal working points by means of parameter adjustment, in order to cope with different network requirements and policies, as well as with the dynamic conditions. In this paper we overview the work done in SDN for WLANs, as well as the pioneering works considering ML for WLAN optimization. Finally, in order to demonstrate the potential of ML techniques in combination with SDN to improve the network operation, we evaluate different use cases for intelligent-based spatial reuse and dynamic channel bonding operation in WLANs using Multi-Armed Bandits. △ Less

Submitted 12 September, 2019; v1 submitted 16 April, 2018; originally announced April 2018.

Towards Energy Efficient LPWANs through Learning-based Multi-hop Routing

Authors: Sergio Barrachina-Muñoz, Toni Adame, Albert Bel, Boris Bellalta

Abstract: Low-power wide area networks (LPWANs) have been identified as one of the top emerging wireless technologies due to their autonomy and wide range of applications. Yet, the limited energy resources of battery-powered sensor nodes is a top constraint, especially in single-hop topologies, where nodes located far from the base station must conduct uplink (UL) communications in high power levels. On thi… ▽ More Low-power wide area networks (LPWANs) have been identified as one of the top emerging wireless technologies due to their autonomy and wide range of applications. Yet, the limited energy resources of battery-powered sensor nodes is a top constraint, especially in single-hop topologies, where nodes located far from the base station must conduct uplink (UL) communications in high power levels. On this point, multi-hop routings in the UL are starting to gain attention due to their capability of reducing energy consumption by enabling transmissions to closer hops. Nonetheless, a priori identifying energy efficient multi-hop routings is not trivial due to the unpredictable factors affecting the communication links in large LPWAN areas. In this paper, we propose epsilon multi-hop (EMH), a simple reinforcement learning (RL) algorithm based on epsilon-greedy to enable reliable and low consumption LPWAN multi-hop topologies. Results from a real testbed show that multi-hop topologies based on EMH achieve significant energy savings with respect to the default single-hop approach, which are accentuated as the network operation progresses. △ Less

Submitted 12 December, 2018; v1 submitted 29 March, 2018; originally announced March 2018.

To overlap or not to overlap: Enabling Channel Bonding in High-Density WLANs

Authors: Sergio Barrachina-Muñoz, Francesc Wilhelmi, Boris Bellalta

Abstract: Wireless local area networks (WLANs) are the most popular kind of wireless Internet connection because of their simplicity of deployment and operation. As a result, the number of devices accessing the Internet through WLANs such as laptops, smartphones, or wearables, is increasing drastically at the same time that applications' throughput requirements do. To cope with these challenges, channel bon… ▽ More Wireless local area networks (WLANs) are the most popular kind of wireless Internet connection because of their simplicity of deployment and operation. As a result, the number of devices accessing the Internet through WLANs such as laptops, smartphones, or wearables, is increasing drastically at the same time that applications' throughput requirements do. To cope with these challenges, channel bonding (CB) techniques are used for enabling higher data rates by transmitting in wider channels, thus increasing spectrum efficiency. However, important issues like higher potential co-channel and adjacent channel interference arise when bonding channels. This may harm the performance of the carrier sense multiple access (CSMA) protocol because of recurrent backoff freezing while making nodes more sensitive to hidden node effects. In this paper, we address the following point at issue: is it convenient for high-density (HD) WLANs to use wider channels and potentially overlap in the spectrum? First, we highlight key aspects of DCB in toy scenarios through a continuous time Markov network (CTMN) model. Then, by means of extensive simulations covering a wide range of traffic loads and access point (AP) densities, we show that dynamic channel bonding (DCB) - which adapts the channel bandwidth on a per-packet transmission - significantly outperforms traditional single-channel on average. Nevertheless, results also corroborate that DCB is more prone to generate unfair situations where WLANs may starve. Contrary to most of the current thoughts pushing towards non-overlap** channels in HD deployments, we highlight the benefits of allocating channels as wider as possible to WLANs altogether with implementing adaptive access policies to cope with the unfairness situations that may appear. △ Less

Submitted 29 December, 2018; v1 submitted 24 March, 2018; originally announced March 2018.

Comments: arXiv admin note: text overlap with arXiv:1801.00594

Dynamic Channel Bonding in Spatially Distributed High-Density WLANs

Authors: Sergio Barrachina-Muñoz, Francesc Wilhelmi, Boris Bellalta

Abstract: In this paper, we discuss the effects on throughput and fairness of dynamic channel bonding (DCB) in spatially distributed high-density wireless local area networks (WLANs). First, we present an analytical framework based on continuous-time Markov networks (CTMNs) for depicting the behavior of different DCB policies in spatially distributed scenarios, where nodes are not required to be within the… ▽ More In this paper, we discuss the effects on throughput and fairness of dynamic channel bonding (DCB) in spatially distributed high-density wireless local area networks (WLANs). First, we present an analytical framework based on continuous-time Markov networks (CTMNs) for depicting the behavior of different DCB policies in spatially distributed scenarios, where nodes are not required to be within the carrier sense range of each other. Then, we assess the performance of DCB in high-density IEEE 802.11ac/ax WLANs by means of simulations. We show that there may be critical interrelations among nodes in the spatial domain - even if they are located outside the carrier sense range of each other - in a chain reaction manner. Results also reveal that, while always selecting the widest available channel normally maximizes the individual long-term throughput, it often generates unfair situations where other WLANs starve. Moreover, we show that there are scenarios where DCB with stochastic channel width selection improves the latter approach both in terms of individual throughput and fairness. It follows that there is not a unique optimal DCB policy for every case. Instead, smarter bandwidth adaptation is required in the challenging scenarios of next-generation WLANs. △ Less

Submitted 6 February, 2020; v1 submitted 2 January, 2018; originally announced January 2018.

Collaborative Spatial Reuse in Wireless Networks via Selfish Multi-Armed Bandits

Authors: Francesc Wilhelmi, Cristina Cano, Gergely Neu, Boris Bellalta, Anders Jonsson, Sergio Barrachina-Muñoz

Abstract: Next-generation wireless deployments are characterized by being dense and uncoordinated, which often leads to inefficient use of resources and poor performance. To solve this, we envision the utilization of completely decentralized mechanisms to enable Spatial Reuse (SR). In particular, we focus on dynamic channel selection and Transmission Power Control (TPC). We rely on Reinforcement Learning (R… ▽ More Next-generation wireless deployments are characterized by being dense and uncoordinated, which often leads to inefficient use of resources and poor performance. To solve this, we envision the utilization of completely decentralized mechanisms to enable Spatial Reuse (SR). In particular, we focus on dynamic channel selection and Transmission Power Control (TPC). We rely on Reinforcement Learning (RL), and more specifically on Multi-Armed Bandits (MABs), to allow networks to learn their best configuration. In this work, we study the exploration-exploitation trade-off by means of the $\varepsilon$-greedy, EXP3, UCB and Thompson sampling action-selection, and compare their performance. In addition, we study the implications of selecting actions simultaneously in an adversarial setting (i.e., concurrently), and compare it with a sequential approach. Our results show that optimal proportional fairness can be achieved, even when no information about neighboring networks is available to the learners and Wireless Networks (WNs) operate selfishly. However, there is high temporal variability in the throughput experienced by the individual networks, specially for $\varepsilon$-greedy and EXP3. These strategies, contrary to UCB and Thompson sampling, base their operation on the absolute experienced reward, rather than on its distribution. We identify the cause of this variability to be the adversarial setting of our setup in which the set of most played actions provide intermittent good/poor performance depending on the neighboring decisions. We also show that learning sequentially, even if using a selfish strategy, contributes to minimize this variability. The sequential approach is therefore shown to effectively deal with the challenges posed by the adversarial settings that are typically found in decentralized WNs. △ Less

Submitted 13 November, 2018; v1 submitted 31 October, 2017; originally announced October 2017.

Learning Optimal Routing for the Uplink in LPWANs Using Similarity-enhanced epsilon-greedy

Authors: Sergio Barrachina-Muñoz, Boris Bellalta

Abstract: Despite being a relatively new communication technology, Low-Power Wide Area Networks (LPWANs) have shown their suitability to empower a major part of Internet of Things applications. Nonetheless, most LPWAN solutions are built on star topology (or single-hop) networks, often causing lifetime shortening in stations located far from the gateway. In this respect, recent studies show that multi-hop r… ▽ More Despite being a relatively new communication technology, Low-Power Wide Area Networks (LPWANs) have shown their suitability to empower a major part of Internet of Things applications. Nonetheless, most LPWAN solutions are built on star topology (or single-hop) networks, often causing lifetime shortening in stations located far from the gateway. In this respect, recent studies show that multi-hop routing for uplink communications can reduce LPWANs' energy consumption significantly. However, it is a troublesome task to identify such energetically optimal routings through trial-and-error brute-force approaches because of time and, especially, energy consumption constraints. In this work we show the benefits of facing this exploration/exploitation problem by running centralized variations of the multi-arm bandit's epsilon-greedy, a well-known online decision-making method that combines best known action selection and knowledge expansion. Important energy savings are achieved when proper randomness parameters are set, which are often improved when conveniently applying similarity, a concept introduced in this work that allows harnessing the gathered knowledge by sporadically selecting unexplored routing combinations akin to the best known one. △ Less

Submitted 22 May, 2017; originally announced May 2017.

Comments: 5 pages, 6 figures. arXiv admin note: text overlap with arXiv:1611.08703