-
Collaborative Semantic Occupancy Prediction with Hybrid Feature Fusion in Connected Automated Vehicles
Authors:
Rui Song,
Chenwei Liang,
Hu Cao,
Zhiran Yan,
Walter Zimmer,
Markus Gross,
Andreas Festag,
Alois Knoll
Abstract:
Collaborative perception in automated vehicles leverages the exchange of information between agents, aiming to elevate perception results. Previous camera-based collaborative 3D perception methods typically employ 3D bounding boxes or bird's eye views as representations of the environment. However, these approaches fall short in offering a comprehensive 3D environmental prediction. To bridge this…
▽ More
Collaborative perception in automated vehicles leverages the exchange of information between agents, aiming to elevate perception results. Previous camera-based collaborative 3D perception methods typically employ 3D bounding boxes or bird's eye views as representations of the environment. However, these approaches fall short in offering a comprehensive 3D environmental prediction. To bridge this gap, we introduce the first method for collaborative 3D semantic occupancy prediction. Particularly, it improves local 3D semantic occupancy predictions by hybrid fusion of (i) semantic and occupancy task features, and (ii) compressed orthogonal attention features shared between vehicles. Additionally, due to the lack of a collaborative perception dataset designed for semantic occupancy prediction, we augment a current collaborative perception dataset to include 3D collaborative semantic occupancy labels for a more robust evaluation. The experimental findings highlight that: (i) our collaborative semantic occupancy predictions excel above the results from single vehicles by over 30%, and (ii) models anchored on semantic occupancy outpace state-of-the-art collaborative 3D detection techniques in subsequent perception applications, showcasing enhanced accuracy and enriched semantic-awareness in road environments.
△ Less
Submitted 25 April, 2024; v1 submitted 12 February, 2024;
originally announced February 2024.
-
Multi-Channel Operation for the Release 2 of ETSI Cooperative Intelligent Transport Systems
Authors:
Alessandro Bazzi,
Miguel Sepulcre,
Quentin Delooz,
Andreas Festag,
Jonas Vogt,
Horst Wieker,
Friedbert Berens,
Paul Spaanderman
Abstract:
Vehicles and road infrastructure are starting to be equipped with vehicle-to-everything (V2X) communication solutions to increase road safety and provide new services to drivers and passengers. In Europe, the deployment is based on a set of Release 1 standards developed by ETSI to support basic use cases for cooperative intelligent transport systems (C-ITS). For them, the capacity of a single 10 M…
▽ More
Vehicles and road infrastructure are starting to be equipped with vehicle-to-everything (V2X) communication solutions to increase road safety and provide new services to drivers and passengers. In Europe, the deployment is based on a set of Release 1 standards developed by ETSI to support basic use cases for cooperative intelligent transport systems (C-ITS). For them, the capacity of a single 10 MHz channel in the ITS band at 5.9 GHz is considered sufficient. At the same time, the ITS stakeholders are working towards several advanced use cases, which imply a significant increment of data traffic and the need for multiple channels. To address this issue, ETSI has recently standardized a new multi-channel operation (MCO) concept for flexible, efficient, and future-proof use of multiple channels. This new concept is defined in a set of new specifications that represent the foundation for the future releases of C-ITS standards. The present paper provides a comprehensive review of the new set of specifications, describing the main entities extending the C-ITS architecture at the different layers of the protocol stack, In addition, the paper provides representative examples that describe how these MCO standards will be used in the future and discusses some of the main open issues arising. The review and analysis of this paper facilitate the understanding and motivation of the new set of Release 2 ETSI specifications for MCO and the identification of new research opportunities.
△ Less
Submitted 31 May, 2023;
originally announced May 2023.
-
V2X-Boosted Federated Learning for Cooperative Intelligent Transportation Systems with Contextual Client Selection
Authors:
Rui Song,
Lingjuan Lyu,
Wei Jiang,
Andreas Festag,
Alois Knoll
Abstract:
Machine learning (ML) has revolutionized transportation systems, enabling autonomous driving and smart traffic services. Federated learning (FL) overcomes privacy constraints by training ML models in distributed systems, exchanging model parameters instead of raw data. However, the dynamic states of connected vehicles affect the network connection quality and influence the FL performance. To tackl…
▽ More
Machine learning (ML) has revolutionized transportation systems, enabling autonomous driving and smart traffic services. Federated learning (FL) overcomes privacy constraints by training ML models in distributed systems, exchanging model parameters instead of raw data. However, the dynamic states of connected vehicles affect the network connection quality and influence the FL performance. To tackle this challenge, we propose a contextual client selection pipeline that uses Vehicle-to-Everything (V2X) messages to select clients based on the predicted communication latency. The pipeline includes: (i) fusing V2X messages, (ii) predicting future traffic topology, (iii) pre-clustering clients based on local data distribution similarity, and (iv) selecting clients with minimal latency for future model aggregation. Experiments show that our pipeline outperforms baselines on various datasets, particularly in non-iid settings.
△ Less
Submitted 19 May, 2023;
originally announced May 2023.
-
FedBEVT: Federated Learning Bird's Eye View Perception Transformer in Road Traffic Systems
Authors:
Rui Song,
Runsheng Xu,
Andreas Festag,
Jiaqi Ma,
Alois Knoll
Abstract:
Bird's eye view (BEV) perception is becoming increasingly important in the field of autonomous driving. It uses multi-view camera data to learn a transformer model that directly projects the perception of the road environment onto the BEV perspective. However, training a transformer model often requires a large amount of data, and as camera data for road traffic are often private, they are typical…
▽ More
Bird's eye view (BEV) perception is becoming increasingly important in the field of autonomous driving. It uses multi-view camera data to learn a transformer model that directly projects the perception of the road environment onto the BEV perspective. However, training a transformer model often requires a large amount of data, and as camera data for road traffic are often private, they are typically not shared. Federated learning offers a solution that enables clients to collaborate and train models without exchanging data but model parameters. In this paper, we introduce FedBEVT, a federated transformer learning approach for BEV perception. In order to address two common data heterogeneity issues in FedBEVT: (i) diverse sensor poses, and (ii) varying sensor numbers in perception systems, we propose two approaches -- Federated Learning with Camera-Attentive Personalization (FedCaP) and Adaptive Multi-Camera Masking (AMCM), respectively. To evaluate our method in real-world settings, we create a dataset consisting of four typical federated use cases. Our findings suggest that FedBEVT outperforms the baseline approaches in all four use cases, demonstrating the potential of our approach for improving BEV perception in autonomous driving.
△ Less
Submitted 8 September, 2023; v1 submitted 4 April, 2023;
originally announced April 2023.
-
ResFed: Communication Efficient Federated Learning by Transmitting Deep Compressed Residuals
Authors:
Rui Song,
Liguo Zhou,
Lingjuan Lyu,
Andreas Festag,
Alois Knoll
Abstract:
Federated learning enables cooperative training among massively distributed clients by sharing their learned local model parameters. However, with increasing model size, deploying federated learning requires a large communication bandwidth, which limits its deployment in wireless networks. To address this bottleneck, we introduce a residual-based federated learning framework (ResFed), where residu…
▽ More
Federated learning enables cooperative training among massively distributed clients by sharing their learned local model parameters. However, with increasing model size, deploying federated learning requires a large communication bandwidth, which limits its deployment in wireless networks. To address this bottleneck, we introduce a residual-based federated learning framework (ResFed), where residuals rather than model parameters are transmitted in communication networks for training. In particular, we integrate two pairs of shared predictors for the model prediction in both server-to-client and client-to-server communication. By employing a common prediction rule, both locally and globally updated models are always fully recoverable in clients and the server. We highlight that the residuals only indicate the quasi-update of a model in a single inter-round, and hence contain more dense information and have a lower entropy than the model, comparing to model weights and gradients. Based on this property, we further conduct lossy compression of the residuals by sparsification and quantization and encode them for efficient communication. The experimental evaluation shows that our ResFed needs remarkably less communication costs and achieves better accuracy by leveraging less sensitive residuals, compared to standard federated learning. For instance, to train a 4.08 MB CNN model on CIFAR-10 with 10 clients under non-independent and identically distributed (Non-IID) setting, our approach achieves a compression ratio over 700X in each communication round with minimum impact on the accuracy. To reach an accuracy of 70%, it saves around 99% of the total communication volume from 587.61 Mb to 6.79 Mb in up-streaming and to 4.61 Mb in down-streaming on average for all clients.
△ Less
Submitted 11 December, 2022;
originally announced December 2022.
-
Federated Learning via Decentralized Dataset Distillation in Resource-Constrained Edge Environments
Authors:
Rui Song,
Dai Liu,
Dave Zhenyu Chen,
Andreas Festag,
Carsten Trinitis,
Martin Schulz,
Alois Knoll
Abstract:
In federated learning, all networked clients contribute to the model training cooperatively. However, with model sizes increasing, even sharing the trained partial models often leads to severe communication bottlenecks in underlying networks, especially when communicated iteratively. In this paper, we introduce a federated learning framework FedD3 requiring only one-shot communication by integrati…
▽ More
In federated learning, all networked clients contribute to the model training cooperatively. However, with model sizes increasing, even sharing the trained partial models often leads to severe communication bottlenecks in underlying networks, especially when communicated iteratively. In this paper, we introduce a federated learning framework FedD3 requiring only one-shot communication by integrating dataset distillation instances. Instead of sharing model updates in other federated learning approaches, FedD3 allows the connected clients to distill the local datasets independently, and then aggregates those decentralized distilled datasets (e.g. a few unrecognizable images) from networks for model training. Our experimental results show that FedD3 significantly outperforms other federated learning frameworks in terms of needed communication volumes, while it provides the additional benefit to be able to balance the trade-off between accuracy and communication cost, depending on usage scenario or target dataset. For instance, for training an AlexNet model on CIFAR-10 with 10 clients under non-independent and identically distributed (Non-IID) setting, FedD3 can either increase the accuracy by over 71% with a similar communication volume, or save 98% of communication volume, while reaching the same accuracy, compared to other one-shot federated learning approaches.
△ Less
Submitted 19 May, 2023; v1 submitted 24 August, 2022;
originally announced August 2022.
-
Edge-Aided Sensor Data Sharing in Vehicular Communication Networks
Authors:
Rui Song,
Anupama Hegde,
Numan Senel,
Alois Knoll,
Andreas Festag
Abstract:
Sensor data sharing in vehicular networks can significantly improve the range and accuracy of environmental perception for connected automated vehicles. Different concepts and schemes for dissemination and fusion of sensor data have been developed. It is common to these schemes that measurement errors of the sensors impair the perception quality and can result in road traffic accidents. Specifical…
▽ More
Sensor data sharing in vehicular networks can significantly improve the range and accuracy of environmental perception for connected automated vehicles. Different concepts and schemes for dissemination and fusion of sensor data have been developed. It is common to these schemes that measurement errors of the sensors impair the perception quality and can result in road traffic accidents. Specifically, when the measurement error from the sensors (also referred as measurement noise) is unknown and time varying, the performance of the data fusion process is restricted, which represents a major challenge in the calibration of sensors. In this paper, we consider sensor data sharing and fusion in a vehicular network with both, vehicle-to-infrastructure and vehicle-to-vehicle communication. We propose a method, named Bidirectional Feedback Noise Estimation (BiFNoE), in which an edge server collects and caches sensor measurement data from vehicles. The edge estimates the noise and the targets alternately in double dynamic sliding time windows and enhances the distributed cooperative environment sensing at each vehicle with low communication costs. We evaluate the proposed algorithm and data dissemination strategy in an application scenario by simulation and show that the perception accuracy is on average improved by around 80 % with only 12 kbps uplink and 28 kbps downlink bandwidth.
△ Less
Submitted 17 June, 2022;
originally announced June 2022.
-
Federated Learning Framework Co** with Hierarchical Heterogeneity in Cooperative ITS
Authors:
Rui Song,
Liguo Zhou,
Venkatnarayanan Lakshminarasimhan,
Andreas Festag,
Alois Knoll
Abstract:
Deep learning is a key approach for the environment perception function of Cooperative Intelligent Transportation Systems (C-ITS) with autonomous vehicles and smart traffic infrastructure. In today's C-ITS, smart traffic participants are capable of timely generating and transmitting a large amount of data. However, these data can not be used for model training directly due to privacy constraints.…
▽ More
Deep learning is a key approach for the environment perception function of Cooperative Intelligent Transportation Systems (C-ITS) with autonomous vehicles and smart traffic infrastructure. In today's C-ITS, smart traffic participants are capable of timely generating and transmitting a large amount of data. However, these data can not be used for model training directly due to privacy constraints. In this paper, we introduce a federated learning framework co** with Hierarchical Heterogeneity (H2-Fed), which can notably enhance the conventional pre-trained deep learning model. The framework exploits data from connected public traffic agents in vehicular networks without affecting user data privacy. By coordinating existing traffic infrastructure, including roadside units and road traffic clouds, the model parameters are efficiently disseminated by vehicular communications and hierarchically aggregated. Considering the individual heterogeneity of data distribution, computational and communication capabilities across traffic agents and roadside units, we employ a novel method that addresses the heterogeneity of different aggregation layers of the framework architecture, i.e., aggregation in layers of roadside units and cloud. The experiment results indicate that our method can well balance the learning accuracy and stability according to the knowledge of heterogeneity in current communication networks. Comparing to other baseline approaches, the evaluation on federated datasets shows that our framework is more general and capable especially in application scenarios with low communication quality. Even when 90% of the agents are timely disconnected, the pre-trained deep learning model can still be forced to converge stably, and its accuracy can be enhanced from 68% to over 90% after convergence.
△ Less
Submitted 28 July, 2022; v1 submitted 1 April, 2022;
originally announced April 2022.
-
Artery-C -- An OMNeT++ Based Discrete Event Simulation Framework for Cellular V2X
Authors:
Anupama Hegde,
Andreas Festag
Abstract:
Cellular Vehicle-to-X (Cellular V2X) is a communication technology that aims to facilitate the communication among vehicles and with the roadside infrastructure. Introduced with LTE Release 14, Cellular V2X enables device-to-device communication to support road safety and traffic efficiency applications. We present Artery-C, a simulation framework for the performance evaluation of Cellular V2X pro…
▽ More
Cellular Vehicle-to-X (Cellular V2X) is a communication technology that aims to facilitate the communication among vehicles and with the roadside infrastructure. Introduced with LTE Release 14, Cellular V2X enables device-to-device communication to support road safety and traffic efficiency applications. We present Artery-C, a simulation framework for the performance evaluation of Cellular V2X protocols and V2X applications. Our simulator relies on the simulation framework SimuLTE and substantially extends it by implementing control and user planes. Besides the vehicle-to-network communication via the up-/downlink interface, it provides vehicle-to-vehicle and vehicle-infrastructure communication via the sidelink interface using the managed and the unmanaged mode of Cellular V2X (mode 3 and 4, respectively). The simulator also implements advanced features of 5G mobile networks, such as variable numerologies. For the transmission of of V2X messages, it adds a non-IP interface. Artery-C integrates seamlessly into the simulation framework Artery, which enables the simulation of standardized V2X messages at the facilities layer as well as the coupling to the mobility simulator SUMO. A specific feature of Artery-C is the support of dynamic switching between all modes of Cellular V2X. In order to demonstrate the capabilities of Artery-C, we evaluate V2X-based platooning as a representative use case and present results for mode 3, mode 4 and mode switching in a highway scenario.
△ Less
Submitted 12 September, 2020;
originally announced September 2020.
