-
Dense Road Surface Grip Map Prediction from Multimodal Image Data
Authors:
Jyri Maanpää,
Julius Pesonen,
Heikki Hyyti,
Iaroslav Melekhov,
Juho Kannala,
Petri Manninen,
Antero Kukko,
Juha Hyyppä
Abstract:
Slippery road weather conditions are prevalent in many regions and cause a regular risk for traffic. Still, there has been less research on how autonomous vehicles could detect slippery driving conditions on the road to drive safely. In this work, we propose a method to predict a dense grip map from the area in front of the car, based on postprocessed multimodal sensor data. We trained a convoluti…
▽ More
Slippery road weather conditions are prevalent in many regions and cause a regular risk for traffic. Still, there has been less research on how autonomous vehicles could detect slippery driving conditions on the road to drive safely. In this work, we propose a method to predict a dense grip map from the area in front of the car, based on postprocessed multimodal sensor data. We trained a convolutional neural network to predict pixelwise grip values from fused RGB camera, thermal camera, and LiDAR reflectance images, based on weakly supervised ground truth from an optical road weather sensor.
The experiments show that it is possible to predict dense grip values with good accuracy from the used data modalities as the produced grip map follows both ground truth measurements and local weather conditions, such as snowy areas on the road. The model using only the RGB camera or LiDAR reflectance modality provided good baseline results for grip prediction accuracy while using models fusing the RGB camera, thermal camera, and LiDAR modalities improved the grip predictions significantly.
△ Less
Submitted 26 April, 2024;
originally announced April 2024.
-
Earth Virtualization Engines -- A Technical Perspective
Authors:
Torsten Hoefler,
Bjorn Stevens,
Andreas F. Prein,
Johanna Baehr,
Thomas Schulthess,
Thomas F. Stocker,
John Taylor,
Daniel Klocke,
Pekka Manninen,
Piers M. Forster,
Tobias Kölling,
Nicolas Gruber,
Hartwig Anzt,
Claudia Frauen,
Florian Ziemen,
Milan Klöwer,
Karthik Kashinath,
Christoph Schär,
Oliver Fuhrer,
Bryan N. Lawrence
Abstract:
Participants of the Berlin Summit on Earth Virtualization Engines (EVEs) discussed ideas and concepts to improve our ability to cope with climate change. EVEs aim to provide interactive and accessible climate simulations and data for a wide range of users. They combine high-resolution physics-based models with machine learning techniques to improve the fidelity, efficiency, and interpretability of…
▽ More
Participants of the Berlin Summit on Earth Virtualization Engines (EVEs) discussed ideas and concepts to improve our ability to cope with climate change. EVEs aim to provide interactive and accessible climate simulations and data for a wide range of users. They combine high-resolution physics-based models with machine learning techniques to improve the fidelity, efficiency, and interpretability of climate projections. At their core, EVEs offer a federated data layer that enables simple and fast access to exabyte-sized climate data through simple interfaces. In this article, we summarize the technical challenges and opportunities for develo** EVEs, and argue that they are essential for addressing the consequences of climate change.
△ Less
Submitted 16 September, 2023;
originally announced September 2023.
-
Towards High-Definition Maps: a Framework Leveraging Semantic Segmentation to Improve NDT Map Compression and Descriptivity
Authors:
Petri Manninen,
Heikki Hyyti,
Ville Kyrki,
Jyri Maanpää,
Josef Taher,
Juha Hyyppä
Abstract:
High-Definition (HD) maps are needed for robust navigation of autonomous vehicles, limited by the on-board storage capacity. To solve this, we propose a novel framework, Environment-Aware Normal Distributions Transform (EA-NDT), that significantly improves compression of standard NDT map representation. The compressed representation of EA-NDT is based on semantic-aided clustering of point clouds r…
▽ More
High-Definition (HD) maps are needed for robust navigation of autonomous vehicles, limited by the on-board storage capacity. To solve this, we propose a novel framework, Environment-Aware Normal Distributions Transform (EA-NDT), that significantly improves compression of standard NDT map representation. The compressed representation of EA-NDT is based on semantic-aided clustering of point clouds resulting in more optimal cells compared to grid cells of standard NDT. To evaluate EA-NDT, we present an open-source implementation that extracts planar and cylindrical primitive features from a point cloud and further divides them into smaller cells to represent the data as an EA-NDT HD map. We collected an open suburban environment dataset and evaluated EA-NDT HD map representation against the standard NDT representation. Compared to the standard NDT, EA-NDT achieved consistently at least 1.5x higher map compression while maintaining the same descriptive capability. Moreover, we showed that EA-NDT is capable of producing maps with significantly higher descriptivity score when using the same number of cells than the standard NDT.
△ Less
Submitted 10 January, 2023;
originally announced January 2023.
-
Leveraging Road Area Semantic Segmentation with Auxiliary Steering Task
Authors:
Jyri Maanpää,
Iaroslav Melekhov,
Josef Taher,
Petri Manninen,
Juha Hyyppä
Abstract:
Robustness of different pattern recognition methods is one of the key challenges in autonomous driving, especially when driving in the high variety of road environments and weather conditions, such as gravel roads and snowfall. Although one can collect data from these adverse conditions using cars equipped with sensors, it is quite tedious to annotate the data for training. In this work, we addres…
▽ More
Robustness of different pattern recognition methods is one of the key challenges in autonomous driving, especially when driving in the high variety of road environments and weather conditions, such as gravel roads and snowfall. Although one can collect data from these adverse conditions using cars equipped with sensors, it is quite tedious to annotate the data for training. In this work, we address this limitation and propose a CNN-based method that can leverage the steering wheel angle information to improve the road area semantic segmentation. As the steering wheel angle data can be easily acquired with the associated images, one could improve the accuracy of road area semantic segmentation by collecting data in new road environments without manual data annotation. We demonstrate the effectiveness of the proposed approach on two challenging data sets for autonomous driving and show that when the steering task is used in our segmentation model training, it leads to a 0.1-2.9% gain in the road area mIoU (mean Intersection over Union) compared to the corresponding reference transfer learning model.
△ Less
Submitted 19 December, 2022;
originally announced December 2022.
-
Multimodal End-to-End Learning for Autonomous Steering in Adverse Road and Weather Conditions
Authors:
Jyri Maanpää,
Josef Taher,
Petri Manninen,
Leo Pakola,
Iaroslav Melekhov,
Juha Hyyppä
Abstract:
Autonomous driving is challenging in adverse road and weather conditions in which there might not be lane lines, the road might be covered in snow and the visibility might be poor. We extend the previous work on end-to-end learning for autonomous steering to operate in these adverse real-life conditions with multimodal data. We collected 28 hours of driving data in several road and weather conditi…
▽ More
Autonomous driving is challenging in adverse road and weather conditions in which there might not be lane lines, the road might be covered in snow and the visibility might be poor. We extend the previous work on end-to-end learning for autonomous steering to operate in these adverse real-life conditions with multimodal data. We collected 28 hours of driving data in several road and weather conditions and trained convolutional neural networks to predict the car steering wheel angle from front-facing color camera images and lidar range and reflectance data. We compared the CNN model performances based on the different modalities and our results show that the lidar modality improves the performances of different multimodal sensor-fusion models. We also performed on-road tests with different models and they support this observation.
△ Less
Submitted 29 June, 2021; v1 submitted 28 October, 2020;
originally announced October 2020.
-
Scaling and diabatic effects in quantum annealing with a D-Wave device
Authors:
Phillip Weinberg,
Marek Tylutki,
Jami M. Rönkkö,
Jan Westerholm,
Jan A. Åström,
Pekka Manninen,
Päivi Törmä,
Anders W. Sandvik
Abstract:
We discuss quantum annealing of the two-dimensional transverse-field Ising model on a D-Wave device, encoded on $L \times L$ lattices with $L \le 32$. Analyzing the residual energy and deviation from maximal magnetization in the final classical state, we find an optimal $L$ dependent annealing rate $v$ for which the two quantities are minimized. The results are well described by a phenomenological…
▽ More
We discuss quantum annealing of the two-dimensional transverse-field Ising model on a D-Wave device, encoded on $L \times L$ lattices with $L \le 32$. Analyzing the residual energy and deviation from maximal magnetization in the final classical state, we find an optimal $L$ dependent annealing rate $v$ for which the two quantities are minimized. The results are well described by a phenomenological model with two powers of $v$ and $L$-dependent prefactors to describe the competing effects of reduced quantum fluctuations (for which we see evidence of the Kibble-Zurek mechanism) and increasing noise impact when $v$ is lowered. The same scaling form also describes results of numerical solutions of a transverse-field Ising model with the spins coupled to noise sources. We explain why the optimal annealing time is much longer than the coherence time of the individual qubits.
△ Less
Submitted 5 March, 2020; v1 submitted 30 September, 2019;
originally announced September 2019.
