-
Pragmatic Communication for Remote Control of Finite-State Markov Processes
Authors:
Pietro Talli,
Edoardo David Santi,
Federico Chiariotti,
Touraj Soleymani,
Federico Mason,
Andrea Zanella,
Deniz Gündüz
Abstract:
Pragmatic or goal-oriented communication can optimize communication decisions beyond the reliable transmission of data, instead aiming at directly affecting application performance with the minimum channel utilization. In this paper, we develop a general theoretical framework for the remote control of finite-state Markov processes, using pragmatic communication over a costly zero-delay communicati…
▽ More
Pragmatic or goal-oriented communication can optimize communication decisions beyond the reliable transmission of data, instead aiming at directly affecting application performance with the minimum channel utilization. In this paper, we develop a general theoretical framework for the remote control of finite-state Markov processes, using pragmatic communication over a costly zero-delay communication channel. To that end, we model a cyber-physical system composed of an encoder, which observes and transmits the states of a process in real-time, and a decoder, which receives that information and controls the behavior of the process. The encoder and the decoder should cooperatively optimize the trade-off between the control performance (i.e., reward) and the communication cost (i.e., channel use). This scenario underscores a pragmatic (i.e., goal-oriented) communication problem, where the purpose is to convey only the data that is most valuable for the underlying task, taking into account the state of the decoder (hence, the pragmatic aspect). We investigate two different decision-making architectures: in pull-based remote control, the decoder is the only decision-maker, while in push-based remote control, the encoder and the decoder constitute two independent decision-makers, leading to a multi-agent scenario. We propose three algorithms to optimize our system (i.e., design the encoder and the decoder policies), discuss the optimality guarantees ofs the algorithms, and shed light on their computational complexity and fundamental limits.
△ Less
Submitted 17 May, 2024;
originally announced May 2024.
-
Push- and Pull-based Effective Communication in Cyber-Physical Systems
Authors:
Pietro Talli,
Federico Mason,
Federico Chiariotti,
Andrea Zanella
Abstract:
In Cyber Physical Systems (CPSs), two groups of actors interact toward the maximization of system performance: the sensors, observing and disseminating the system state, and the actuators, performing physical decisions based on the received information. While it is generally assumed that sensors periodically transmit updates, returning the feedback signal only when necessary, and consequently adap…
▽ More
In Cyber Physical Systems (CPSs), two groups of actors interact toward the maximization of system performance: the sensors, observing and disseminating the system state, and the actuators, performing physical decisions based on the received information. While it is generally assumed that sensors periodically transmit updates, returning the feedback signal only when necessary, and consequently adapting the physical decisions to the communication policy, can significantly improve the efficiency of the system. In particular, the choice between push-based communication, in which updates are initiated autonomously by the sensors, and pull-based communication, in which they are requested by the actuators, is a key design step. In this work, we propose an analytical model for optimizing push- and pull-based communication in CPSs, observing that the policy optimality coincides with Value of Information (VoI) maximization. Our results also highlight that, despite providing a better optimal solution, implementable push-based communication strategies may underperform even in relatively simple scenarios.
△ Less
Submitted 15 January, 2024;
originally announced January 2024.
-
Fast Context Adaptation in Cost-Aware Continual Learning
Authors:
Seyyidahmed Lahmer,
Federico Mason,
Federico Chiariotti,
Andrea Zanella
Abstract:
In the past few years, DRL has become a valuable solution to automatically learn efficient resource management strategies in complex networks with time-varying statistics. However, the increased complexity of 5G and Beyond networks requires correspondingly more complex learning agents and the learning process itself might end up competing with users for communication and computational resources. T…
▽ More
In the past few years, DRL has become a valuable solution to automatically learn efficient resource management strategies in complex networks with time-varying statistics. However, the increased complexity of 5G and Beyond networks requires correspondingly more complex learning agents and the learning process itself might end up competing with users for communication and computational resources. This creates friction: on the one hand, the learning process needs resources to quickly convergence to an effective strategy; on the other hand, the learning process needs to be efficient, i.e., take as few resources as possible from the user's data plane, so as not to throttle users' QoS. In this paper, we investigate this trade-off and propose a dynamic strategy to balance the resources assigned to the data plane and those reserved for learning. With the proposed approach, a learning agent can quickly converge to an efficient resource allocation strategy and adapt to changes in the environment as for the CL paradigm, while minimizing the impact on the users' QoS. Simulation results show that the proposed method outperforms static allocation methods with minimal learning overhead, almost reaching the performance of an ideal out-of-band CL solution.
△ Less
Submitted 6 June, 2023;
originally announced June 2023.
-
Multi-Agent Reinforcement Learning for Pragmatic Communication and Control
Authors:
Federico Mason,
Federico Chiariotti,
Andrea Zanella,
Petar Popovski
Abstract:
The automation of factories and manufacturing processes has been accelerating over the past few years, boosted by the Industry 4.0 paradigm, including diverse scenarios with mobile, flexible agents. Efficient coordination between mobile robots requires reliable wireless transmission in highly dynamic environments, often with strict timing requirements. Goal-oriented communication is a possible sol…
▽ More
The automation of factories and manufacturing processes has been accelerating over the past few years, boosted by the Industry 4.0 paradigm, including diverse scenarios with mobile, flexible agents. Efficient coordination between mobile robots requires reliable wireless transmission in highly dynamic environments, often with strict timing requirements. Goal-oriented communication is a possible solution for this problem: communication decisions should be optimized for the target control task, providing the information that is most relevant to decide which action to take. From the control perspective, networked control design takes the communication impairments into account in its optmization of physical actions. In this work, we propose a joint design that combines goal-oriented communication and networked control into a single optimization model, an extension of a multiagent POMDP which we call Cyber-Physical POMDP (CP-POMDP). The model is flexible enough to represent several swarm and cooperative scenarios, and we illustrate its potential with two simple reference scenarios with a single agent and a set of supporting sensors. Joint training of the communication and control systems can significantly improve the overall performance, particularly if communication is severely constrained, and can even lead to implicit coordination of communication actions.
△ Less
Submitted 28 February, 2023;
originally announced February 2023.
-
Towards Decentralized Predictive Quality of Service in Next-Generation Vehicular Networks
Authors:
Filippo Bragato,
Tommaso Lotta,
Gianmaria Ventura,
Matteo Drago,
Federico Mason,
Marco Giordani,
Michele Zorzi
Abstract:
To ensure safety in teleoperated driving scenarios, communication between vehicles and remote drivers must satisfy strict latency and reliability requirements. In this context, Predictive Quality of Service (PQoS) was investigated as a tool to predict unanticipated degradation of the Quality of Service (QoS), and allow the network to react accordingly. In this work, we design a reinforcement learn…
▽ More
To ensure safety in teleoperated driving scenarios, communication between vehicles and remote drivers must satisfy strict latency and reliability requirements. In this context, Predictive Quality of Service (PQoS) was investigated as a tool to predict unanticipated degradation of the Quality of Service (QoS), and allow the network to react accordingly. In this work, we design a reinforcement learning (RL) agent to implement PQoS in vehicular networks. To do so, based on data gathered at the Radio Access Network (RAN) and/or the end vehicles, as well as QoS predictions, our framework is able to identify the optimal level of compression to send automotive data under low latency and reliability constraints. We consider different learning schemes, including centralized, fully-distributed, and federated learning. We demonstrate via ns-3 simulations that, while centralized learning generally outperforms any other solution, decentralized learning, and especially federated learning, offers a good trade-off between convergence time and reliability, with positive implications in terms of privacy and complexity.
△ Less
Submitted 22 February, 2023;
originally announced February 2023.
-
Artificial Intelligence in Vehicular Wireless Networks: A Case Study Using ns-3
Authors:
Matteo Drago,
Tommaso Zugno,
Federico Mason,
Marco Giordani,
Mate Boban,
Michele Zorzi
Abstract:
Artificial intelligence (AI) techniques have emerged as a powerful approach to make wireless networks more efficient and adaptable. In this paper we present an ns-3 simulation framework, able to implement AI algorithms for the optimization of wireless networks. Our pipeline consists of: (i) a new geometry-based mobility-dependent channel model for V2X; (ii) all the layers of a 5G-NR-compliant prot…
▽ More
Artificial intelligence (AI) techniques have emerged as a powerful approach to make wireless networks more efficient and adaptable. In this paper we present an ns-3 simulation framework, able to implement AI algorithms for the optimization of wireless networks. Our pipeline consists of: (i) a new geometry-based mobility-dependent channel model for V2X; (ii) all the layers of a 5G-NR-compliant protocol stack, based on the ns3-mmwave module; (iii) a new application to simulate V2X data transmission, and (iv) a new intelligent entity for the control of the network via AI. Thanks to its flexible and modular design, researchers can use this tool to implement, train, and evaluate their own algorithms in a realistic and controlled environment. We test the behavior of our framework in a Predictive Quality of Service (PQoS) scenario, where AI functionalities are implemented using Reinforcement Learning (RL), and demonstrate that it promotes better network optimization compared to baseline solutions that do not implement AI.
△ Less
Submitted 10 March, 2022;
originally announced March 2022.
-
A Reinforcement Learning Framework for PQoS in a Teleoperated Driving Scenario
Authors:
Federico Mason,
Matteo Drago,
Tommaso Zugno,
Marco Giordani,
Mate Boban,
Michele Zorzi
Abstract:
In recent years, autonomous networks have been designed with Predictive Quality of Service (PQoS) in mind, as a means for applications operating in the industrial and/or automotive sectors to predict unanticipated Quality of Service (QoS) changes and react accordingly. In this context, Reinforcement Learning (RL) has come out as a promising approach to perform accurate predictions, and optimize th…
▽ More
In recent years, autonomous networks have been designed with Predictive Quality of Service (PQoS) in mind, as a means for applications operating in the industrial and/or automotive sectors to predict unanticipated Quality of Service (QoS) changes and react accordingly. In this context, Reinforcement Learning (RL) has come out as a promising approach to perform accurate predictions, and optimize the efficiency and adaptability of wireless networks. Along these lines, in this paper we propose the design of a new entity, implemented at the RAN-level that, with the support of an RL framework, implements PQoS functionalities. Specifically, we focus on the design of the reward function of the learning agent, able to convert QoS estimates into appropriate countermeasures if QoS requirements are not satisfied. We demonstrate via ns-3 simulations that our approach achieves the best trade-off in terms of QoS and Quality of Experience (QoE) performance of end users in a teleoperated-driving-like scenario, compared to other baseline solutions.
△ Less
Submitted 3 February, 2022;
originally announced February 2022.
-
No Free Lunch: Balancing Learning and Exploitation at the Network Edge
Authors:
Federico Mason,
Federico Chiariotti,
Andrea Zanella
Abstract:
Over the last few years, the DRL paradigm has been widely adopted for 5G and beyond network optimization because of its extreme adaptability to many different scenarios. However, collecting and processing learning data entail a significant cost in terms of communication and computational resources, which is often disregarded in the networking literature. In this work, we analyze the cost of learni…
▽ More
Over the last few years, the DRL paradigm has been widely adopted for 5G and beyond network optimization because of its extreme adaptability to many different scenarios. However, collecting and processing learning data entail a significant cost in terms of communication and computational resources, which is often disregarded in the networking literature. In this work, we analyze the cost of learning in a resource-constrained system, defining an optimization problem in which training a DRL agent makes it possible to improve the resource allocation strategy but also reduces the number of available resources. Our simulation results show that the cost of learning can be critical when evaluating DRL schemes on the network edge and that assuming a cost-free learning model can lead to significantly overestimating performance.
△ Less
Submitted 23 November, 2021;
originally announced November 2021.
-
Using Distributed Reinforcement Learning for Resource Orchestration in a Network Slicing Scenario
Authors:
Federico Mason,
Gianfranco Nencioni,
Andrea Zanella
Abstract:
The Network Slicing (NS) paradigm enables the partition of physical and virtual resources among multiple logical networks, possibly managed by different tenants. In such a scenario, network resources need to be dynamically allocated according to the slices' requirements. In this paper, we attack the above problem by exploiting a Deep Reinforcement Learning approach. Our framework is based on a dis…
▽ More
The Network Slicing (NS) paradigm enables the partition of physical and virtual resources among multiple logical networks, possibly managed by different tenants. In such a scenario, network resources need to be dynamically allocated according to the slices' requirements. In this paper, we attack the above problem by exploiting a Deep Reinforcement Learning approach. Our framework is based on a distributed architecture, where multiple agents cooperate towards a common goal. The agents' training is carried out following the Advantage Actor Critic algorithm, which allows to handle continuous action spaces. By means of extensive simulations, we show that our approach yields better performance than both a static allocation of system resources and an efficient empirical strategy. At the same time, the proposed system ensures high adaptability to different scenarios without the need for additional training.
△ Less
Submitted 17 May, 2021;
originally announced May 2021.
-
Distributed Reinforcement Learning for Flexible and Efficient UAV Swarm Control
Authors:
Federico Venturini,
Federico Mason,
Francesco Pase,
Federico Chiariotti,
Alberto Testolin,
Andrea Zanella,
Michele Zorzi
Abstract:
Over the past few years, the use of swarms of Unmanned Aerial Vehicles (UAVs) in monitoring and remote area surveillance applications has become widespread thanks to the price reduction and the increased capabilities of drones. The drones in the swarm need to cooperatively explore an unknown area, in order to identify and monitor interesting targets, while minimizing their movements. In this work,…
▽ More
Over the past few years, the use of swarms of Unmanned Aerial Vehicles (UAVs) in monitoring and remote area surveillance applications has become widespread thanks to the price reduction and the increased capabilities of drones. The drones in the swarm need to cooperatively explore an unknown area, in order to identify and monitor interesting targets, while minimizing their movements. In this work, we propose a distributed Reinforcement Learning (RL) approach that scales to larger swarms without modifications. The proposed framework relies on the possibility for the UAVs to exchange some information through a communication channel, in order to achieve context-awareness and implicitly coordinate the swarm's actions. Our experiments show that the proposed method can yield effective strategies, which are robust to communication channel impairments, and that can easily deal with non-uniform distributions of targets and obstacles. Moreover, when agents are trained in a specific scenario, they can adapt to a new one with minimal additional training. We also show that our approach achieves better performance compared to a computationally intensive look-ahead heuristic.
△ Less
Submitted 8 March, 2021;
originally announced March 2021.
-
Internet of Things for Elderly and Fragile People
Authors:
Andrea Zanella,
Federico Mason,
Patrik Pluchino,
Giulia Cisotto,
Valeria Orso,
Luciano Gamberini
Abstract:
This paper discusses the potential of the Internet of Things (IoT) paradigm in the context of assisted living for elderly and fragile people, in the light of the peculiar requirements of such users, both from a functional and a technological perspective. We stress some aspects that are often disregarded by the technical community, such as technology acceptability and usability, and we describe the…
▽ More
This paper discusses the potential of the Internet of Things (IoT) paradigm in the context of assisted living for elderly and fragile people, in the light of the peculiar requirements of such users, both from a functional and a technological perspective. We stress some aspects that are often disregarded by the technical community, such as technology acceptability and usability, and we describe the framework and the phases of the current co-design approaches that imply the active involvement of the final users in the system design process. Thereby, we identify a series of design practices to merge technical and fragile people's requirements. The discussion is backed up by the description of DOMHO, a prototypal IoT-based AAL system that embodies most of the concepts described in the paper, and that is being deployed and tested in a shelter house for elders, and in an apartment for the co-housing of individuals with disabilities. Finally, we discuss the potential and limits of the current approaches and present some open challenges and future research directions.
△ Less
Submitted 10 June, 2020;
originally announced June 2020.
-
An Adaptive Broadcasting Strategy for Efficient Dynamic Map** in Vehicular Networks
Authors:
Federico Mason,
Marco Giordani,
Federico Chiariotti,
Andrea Zanella,
Michele Zorzi
Abstract:
In this work, we face the issue of achieving an efficient dynamic map** in vehicular networking scenarios, i.e., to obtain an accurate estimate of the positions and trajectories of connected vehicles in a certain area. State of the art solutions are based on the periodic broadcasting of the position information of the network nodes, with an inter-transmission period set by a congestion control s…
▽ More
In this work, we face the issue of achieving an efficient dynamic map** in vehicular networking scenarios, i.e., to obtain an accurate estimate of the positions and trajectories of connected vehicles in a certain area. State of the art solutions are based on the periodic broadcasting of the position information of the network nodes, with an inter-transmission period set by a congestion control scheme. However, the movements and maneuvers of vehicles can often be erratic, making transmitted data inaccurate or downright misleading. To address this problem, we propose to adopt a dynamic transmission scheme based on the actual positioning error, sending new data when the estimate passes a preset error threshold. Furthermore, the proposed method adapts the error threshold to the operational context according to a congestion control algorithm that limits the collision probability among broadcast packet transmissions. This threshold-based strategy can reduce the network load by avoiding the transmission of redundant messages, and is shown to improve the overall positioning accuracy by more than 20% in realistic urban scenarios.
△ Less
Submitted 16 October, 2019;
originally announced October 2019.
-
Quality-Aware Broadcasting Strategies for Position Estimation in VANETs
Authors:
Federico Mason,
Marco Giordani,
Federico Chiariotti,
Andrea Zanella,
Michele Zorzi
Abstract:
The dissemination of vehicle position data all over the network is a fundamental task in Vehicular Ad Hoc Network (VANET) operations, as applications often need to know the position of other vehicles over a large area. In such cases, inter-vehicular communications should be exploited to satisfy application requirements, although congestion control mechanisms are required to minimize the packet col…
▽ More
The dissemination of vehicle position data all over the network is a fundamental task in Vehicular Ad Hoc Network (VANET) operations, as applications often need to know the position of other vehicles over a large area. In such cases, inter-vehicular communications should be exploited to satisfy application requirements, although congestion control mechanisms are required to minimize the packet collision probability. In this work, we face the issue of achieving accurate vehicle position estimation and prediction in a VANET scenario. State of the art solutions to the problem try to broadcast the positioning information periodically, so that vehicles can ensure that the information their neighbors have about them is never older than the inter-transmission period. However, the rate of decay of the information is not deterministic in complex urban scenarios: the movements and maneuvers of vehicles can often be erratic and unpredictable, making old positioning information inaccurate or downright misleading. To address this problem, we propose to use the Quality of Information (QoI) as the decision factor for broadcasting. We implement a threshold-based strategy to distribute position information whenever the positioning error passes a reference value, thereby shifting the objective of the network to limiting the actual positioning error and guaranteeing quality across the VANET. The threshold-based strategy can reduce the network load by avoiding the transmission of redundant messages, as well as improving the overall positioning accuracy by more than 20% in realistic urban scenarios.
△ Less
Submitted 8 April, 2019;
originally announced April 2019.
