-
ActiveAnno3D -- An Active Learning Framework for Multi-Modal 3D Object Detection
Authors:
Ahmed Ghita,
Bjørk Antoniussen,
Walter Zimmer,
Ross Greer,
Christian Creß,
Andreas Møgelmose,
Mohan M. Trivedi,
Alois C. Knoll
Abstract:
The curation of large-scale datasets is still costly and requires much time and resources. Data is often manually labeled, and the challenge of creating high-quality datasets remains. In this work, we fill the research gap using active learning for multi-modal 3D object detection. We propose ActiveAnno3D, an active learning framework to select data samples for labeling that are of maximum informat…
▽ More
The curation of large-scale datasets is still costly and requires much time and resources. Data is often manually labeled, and the challenge of creating high-quality datasets remains. In this work, we fill the research gap using active learning for multi-modal 3D object detection. We propose ActiveAnno3D, an active learning framework to select data samples for labeling that are of maximum informativeness for training. We explore various continuous training methods and integrate the most efficient method regarding computational demand and detection performance. Furthermore, we perform extensive experiments and ablation studies with BEVFusion and PV-RCNN on the nuScenes and TUM Traffic Intersection dataset. We show that we can achieve almost the same performance with PV-RCNN and the entropy-based query strategy when using only half of the training data (77.25 mAP compared to 83.50 mAP) of the TUM Traffic Intersection dataset. BEVFusion achieved an mAP of 64.31 when using half of the training data and 75.0 mAP when using the complete nuScenes dataset. We integrate our active learning framework into the proAnno labeling tool to enable AI-assisted data selection and labeling and minimize the labeling costs. Finally, we provide code, weights, and visualization results on our website: https://active3d-framework.github.io/active3d-framework.
△ Less
Submitted 5 February, 2024;
originally announced February 2024.
-
TUMTraf Event: Calibration and Fusion Resulting in a Dataset for Roadside Event-Based and RGB Cameras
Authors:
Christian Creß,
Walter Zimmer,
Nils Purschke,
Bach Ngoc Doan,
Sven Kirchner,
Venkatnarayanan Lakshminarasimhan,
Leah Strand,
Alois C. Knoll
Abstract:
Event-based cameras are predestined for Intelligent Transportation Systems (ITS). They provide very high temporal resolution and dynamic range, which can eliminate motion blur and improve detection performance at night. However, event-based images lack color and texture compared to images from a conventional RGB camera. Considering that, data fusion between event-based and conventional cameras can…
▽ More
Event-based cameras are predestined for Intelligent Transportation Systems (ITS). They provide very high temporal resolution and dynamic range, which can eliminate motion blur and improve detection performance at night. However, event-based images lack color and texture compared to images from a conventional RGB camera. Considering that, data fusion between event-based and conventional cameras can combine the strengths of both modalities. For this purpose, extrinsic calibration is necessary. To the best of our knowledge, no targetless calibration between event-based and RGB cameras can handle multiple moving objects, nor does data fusion optimized for the domain of roadside ITS exist. Furthermore, synchronized event-based and RGB camera datasets considering roadside perspective are not yet published. To fill these research gaps, based on our previous work, we extended our targetless calibration approach with clustering methods to handle multiple moving objects. Furthermore, we developed an early fusion, simple late fusion, and a novel spatiotemporal late fusion method. Lastly, we published the TUMTraf Event Dataset, which contains more than 4,111 synchronized event-based and RGB images with 50,496 labeled 2D boxes. During our extensive experiments, we verified the effectiveness of our calibration method with multiple moving objects. Furthermore, compared to a single RGB camera, we increased the detection performance of up to +9 % mAP in the day and up to +13 % mAP during the challenging night with our presented event-based sensor fusion methods. The TUMTraf Event Dataset is available at https://innovation-mobility.com/tumtraf-dataset.
△ Less
Submitted 9 March, 2024; v1 submitted 16 January, 2024;
originally announced January 2024.
-
A9 Intersection Dataset: All You Need for Urban 3D Camera-LiDAR Roadside Perception
Authors:
Walter Zimmer,
Christian Creß,
Huu Tung Nguyen,
Alois C. Knoll
Abstract:
Intelligent Transportation Systems (ITS) allow a drastic expansion of the visibility range and decrease occlusions for autonomous driving. To obtain accurate detections, detailed labeled sensor data for training is required. Unfortunately, high-quality 3D labels of LiDAR point clouds from the infrastructure perspective of an intersection are still rare. Therefore, we provide the A9 Intersection Da…
▽ More
Intelligent Transportation Systems (ITS) allow a drastic expansion of the visibility range and decrease occlusions for autonomous driving. To obtain accurate detections, detailed labeled sensor data for training is required. Unfortunately, high-quality 3D labels of LiDAR point clouds from the infrastructure perspective of an intersection are still rare. Therefore, we provide the A9 Intersection Dataset, which consists of labeled LiDAR point clouds and synchronized camera images. Here, we recorded the sensor output from two roadside cameras and LiDARs mounted on intersection gantry bridges. The point clouds were labeled in 3D by experienced annotators. Furthermore, we provide calibration data between all sensors, which allow the projection of the 3D labels into the camera images and an accurate data fusion. Our dataset consists of 4.8k images and point clouds with more than 57.4k manually labeled 3D boxes. With ten object classes, it has a high diversity of road users in complex driving maneuvers, such as left and right turns, overtaking, and U-turns. In experiments, we provided multiple baselines for the perception tasks. Overall, our dataset is a valuable contribution to the scientific community to perform complex 3D camera-LiDAR roadside perception tasks. Find data, code, and more information at https://a9-dataset.com.
△ Less
Submitted 15 June, 2023;
originally announced June 2023.
-
A9-Dataset: Multi-Sensor Infrastructure-Based Dataset for Mobility Research
Authors:
Christian Creß,
Walter Zimmer,
Leah Strand,
Venkatnarayanan Lakshminarasimhan,
Maximilian Fortkord,
Siyi Dai,
Alois Knoll
Abstract:
Data-intensive machine learning based techniques increasingly play a prominent role in the development of future mobility solutions - from driver assistance and automation functions in vehicles, to real-time traffic management systems realized through dedicated infrastructure. The availability of high quality real-world data is often an important prerequisite for the development and reliable deplo…
▽ More
Data-intensive machine learning based techniques increasingly play a prominent role in the development of future mobility solutions - from driver assistance and automation functions in vehicles, to real-time traffic management systems realized through dedicated infrastructure. The availability of high quality real-world data is often an important prerequisite for the development and reliable deployment of such systems in large scale. Towards this endeavour, we present the A9-Dataset based on roadside sensor infrastructure from the 3 km long Providentia++ test field near Munich in Germany. The dataset includes anonymized and precision-timestamped multi-modal sensor and object data in high resolution, covering a variety of traffic situations. As part of the first set of data, which we describe in this paper, we provide camera and LiDAR frames from two overhead gantry bridges on the A9 autobahn with the corresponding objects labeled with 3D bounding boxes. The first set includes in total more than 1000 sensor frames and 14000 traffic objects. The dataset is available for download at https://a9-dataset.com.
△ Less
Submitted 13 May, 2022; v1 submitted 13 April, 2022;
originally announced April 2022.
-
Intelligent Transportation Systems Using External Infrastructure: A Literature Survey
Authors:
Christian Creß,
Zhenshan Bing,
Alois C. Knoll
Abstract:
The main problems in transportation are accidents, increasingly slow traffic flow, and pollution. An intelligent transportation system (ITS) using external infrastructure can overcome these problems. For this reason, the number of such systems is increasing dramatically, and therefore requires an adequate overview. To the best of our knowledge, no current systematic review of existing ITS solution…
▽ More
The main problems in transportation are accidents, increasingly slow traffic flow, and pollution. An intelligent transportation system (ITS) using external infrastructure can overcome these problems. For this reason, the number of such systems is increasing dramatically, and therefore requires an adequate overview. To the best of our knowledge, no current systematic review of existing ITS solutions exists. To fill this knowledge gap, our paper provides an overview of existing ITS that use external infrastructure worldwide. Accordingly, this paper addresses current questions and challenges. For this purpose, we performed a literature review of documents that describe existing ITS solutions from 2009 until today. We categorized the results according to technology levels and analyzed its hardware system setup and value-added contributions. In doing so, we made the ITS solutions comparable and highlighted past development alongside current trends. We analyzed more than 357 papers, including 52 test bed projects. In summary, current ITSs can deliver accurate information about individuals in traffic situations in real-time. However, further research into ITS should focus on more reliable perception of the traffic using modern sensors, plug-and-play mechanisms, and secure real-time distribution of the digital twins in a decentralized manner. By addressing these topics, the development of intelligent transportation systems will be able to take a step towards its comprehensive roll-out.
△ Less
Submitted 25 August, 2022; v1 submitted 10 December, 2021;
originally announced December 2021.
