-
COLA: COarse-LAbel multi-source LiDAR semantic segmentation for autonomous driving
Authors:
Jules Sanchez,
Jean-Emmanuel Deschaud,
François Goulette
Abstract:
LiDAR semantic segmentation for autonomous driving has been a growing field of interest in the past few years. Datasets and methods have appeared and expanded very quickly, but methods have not been updated to exploit this new availability of data and continue to rely on the same classical datasets.
Different ways of performing LIDAR semantic segmentation training and inference can be divided in…
▽ More
LiDAR semantic segmentation for autonomous driving has been a growing field of interest in the past few years. Datasets and methods have appeared and expanded very quickly, but methods have not been updated to exploit this new availability of data and continue to rely on the same classical datasets.
Different ways of performing LIDAR semantic segmentation training and inference can be divided into several subfields, which include the following: domain generalization, the ability to segment data coming from unseen domains ; source-to-source segmentation, the ability to segment data coming from the training domain; and pre-training, the ability to create re-usable geometric primitives.
In this work, we aim to improve results in all of these subfields with the novel approach of multi-source training. Multi-source training relies on the availability of various datasets at training time and uses them together rather than relying on only one dataset.
To overcome the common obstacles found for multi-source training, we introduce the coarse labels and call the newly created multi-source dataset COLA. We propose three applications of this new dataset that display systematic improvement over single-source strategies: COLA-DG for domain generalization (up to +10%), COLA-S2S for source-to-source segmentation (up to +5.3%), and COLA-PT for pre-training (up to +12%).
△ Less
Submitted 6 November, 2023;
originally announced November 2023.
-
ParisLuco3D: A high-quality target dataset for domain generalization of LiDAR perception
Authors:
Jules Sanchez,
Louis Soum-Fontez,
Jean-Emmanuel Deschaud,
Francois Goulette
Abstract:
LiDAR is an essential sensor for autonomous driving by collecting precise geometric information regarding a scene. %Exploiting this information for perception is interesting as the amount of available data increases. As the performance of various LiDAR perception tasks has improved, generalizations to new environments and sensors has emerged to test these optimized models in real-world conditions.…
▽ More
LiDAR is an essential sensor for autonomous driving by collecting precise geometric information regarding a scene. %Exploiting this information for perception is interesting as the amount of available data increases. As the performance of various LiDAR perception tasks has improved, generalizations to new environments and sensors has emerged to test these optimized models in real-world conditions.
This paper provides a novel dataset, ParisLuco3D, specifically designed for cross-domain evaluation to make it easier to evaluate the performance utilizing various source datasets. Alongside the dataset, online benchmarks for LiDAR semantic segmentation, LiDAR object detection, and LiDAR tracking are provided to ensure a fair comparison across methods.
The ParisLuco3D dataset, evaluation scripts, and links to benchmarks can be found at the following website:https://npm3d.fr/parisluco3d
△ Less
Submitted 3 June, 2024; v1 submitted 25 October, 2023;
originally announced October 2023.
-
MDT3D: Multi-Dataset Training for LiDAR 3D Object Detection Generalization
Authors:
Louis Soum-Fontez,
Jean-Emmanuel Deschaud,
François Goulette
Abstract:
Supervised 3D Object Detection models have been displaying increasingly better performance in single-domain cases where the training data comes from the same environment and sensor as the testing data. However, in real-world scenarios data from the target domain may not be available for finetuning or for domain adaptation methods. Indeed, 3D object detection models trained on a source dataset with…
▽ More
Supervised 3D Object Detection models have been displaying increasingly better performance in single-domain cases where the training data comes from the same environment and sensor as the testing data. However, in real-world scenarios data from the target domain may not be available for finetuning or for domain adaptation methods. Indeed, 3D object detection models trained on a source dataset with a specific point distribution have shown difficulties in generalizing to unseen datasets. Therefore, we decided to leverage the information available from several annotated source datasets with our Multi-Dataset Training for 3D Object Detection (MDT3D) method to increase the robustness of 3D object detection models when tested in a new environment with a different sensor configuration. To tackle the labelling gap between datasets, we used a new label map** based on coarse labels. Furthermore, we show how we managed the mix of datasets during training and finally introduce a new cross-dataset augmentation method: cross-dataset object injection. We demonstrate that this training paradigm shows improvements for different types of 3D object detection models. The source code and additional results for this research project will be publicly available on GitHub for interested parties to access and utilize: https://github.com/LouisSF/MDT3D
△ Less
Submitted 2 August, 2023;
originally announced August 2023.
-
Multi-IMU Proprioceptive State Estimator for Humanoid Robots
Authors:
Fabio Elnecave Xavier,
Guillaume Burger,
Marine Pétriaux,
Jean-Emmanuel Deschaud,
François Goulette
Abstract:
Algorithms for state estimation of humanoid robots usually assume that the feet remain flat and in a constant position while in contact with the ground. However, this hypothesis is easily violated while walking, especially for human-like gaits with heel-toe motion. This reduces the time during which the contact assumption can be used, or requires higher variances to account for errors. In this pap…
▽ More
Algorithms for state estimation of humanoid robots usually assume that the feet remain flat and in a constant position while in contact with the ground. However, this hypothesis is easily violated while walking, especially for human-like gaits with heel-toe motion. This reduces the time during which the contact assumption can be used, or requires higher variances to account for errors. In this paper, we present a novel state estimator based on the extended Kalman filter that can properly handle any contact configuration. We consider multiple inertial measurement units (IMUs) distributed throughout the robot's structure, including on both feet, which are used to track multiple bodies of the robot. This multi-IMU instrumentation setup also has the advantage of allowing the deformations in the robot's structure to be estimated, improving the kinematic model used in the filter. The proposed approach is validated experimentally on the exoskeleton Atalante and is shown to present low drift, performing better than similar single-IMU filters. The obtained trajectory estimates are accurate enough to construct elevation maps that have little distortion with respect to the ground truth.
△ Less
Submitted 26 July, 2023;
originally announced July 2023.
-
Domain generalization of 3D semantic segmentation in autonomous driving
Authors:
Jules Sanchez,
Jean-Emmanuel Deschaud,
Francois Goulette
Abstract:
Using deep learning, 3D autonomous driving semantic segmentation has become a well-studied subject, with methods that can reach very high performance. Nonetheless, because of the limited size of the training datasets, these models cannot see every type of object and scene found in real-world applications. The ability to be reliable in these various unknown environments is called \textup{domain gen…
▽ More
Using deep learning, 3D autonomous driving semantic segmentation has become a well-studied subject, with methods that can reach very high performance. Nonetheless, because of the limited size of the training datasets, these models cannot see every type of object and scene found in real-world applications. The ability to be reliable in these various unknown environments is called \textup{domain generalization}.
Despite its importance, domain generalization is relatively unexplored in the case of 3D autonomous driving semantic segmentation. To fill this gap, this paper presents the first benchmark for this application by testing state-of-the-art methods and discussing the difficulty of tackling Laser Imaging Detection and Ranging (LiDAR) domain shifts.
We also propose the first method designed to address this domain generalization, which we call 3DLabelProp. This method relies on leveraging the geometry and sequentiality of the LiDAR data to enhance its generalization performances by working on partially accumulated point clouds. It reaches a mean Intersection over Union (mIoU) of 50.4% on SemanticPOSS and of 55.2% on PandaSet solid-state LiDAR while being trained only on SemanticKITTI, making it the state-of-the-art method for generalization (+5% and +33% better, respectively, than the second best method).
The code for this method is available on GitHub: https://github.com/JulesSanchez/3DLabelProp.
△ Less
Submitted 17 August, 2023; v1 submitted 7 December, 2022;
originally announced December 2022.
-
Unsigned Distance Field as an Accurate 3D Scene Representation for Neural Scene Completion
Authors:
Jean Pierre Richa,
Jean-Emmanuel Deschaud,
François Goulette,
Nicolas Dalmasso
Abstract:
Scene Completion is the task of completing missing geometry from a partial scan of a scene. Most previous methods compute an implicit representation from range data using a Truncated Signed Distance Function (T-SDF) computed on a 3D grid as input to neural networks. The truncation decreases but does not remove the border errors introduced by the sign of SDF for open surfaces. As an alternative, we…
▽ More
Scene Completion is the task of completing missing geometry from a partial scan of a scene. Most previous methods compute an implicit representation from range data using a Truncated Signed Distance Function (T-SDF) computed on a 3D grid as input to neural networks. The truncation decreases but does not remove the border errors introduced by the sign of SDF for open surfaces. As an alternative, we present an Unsigned Distance Function (UDF) as an input representation to scene completion neural networks. The proposed UDF is simple, and efficient as a geometry representation, and can be computed on any point cloud. In contrast to usual Signed Distance Functions, our UDF does not require normal computation. To obtain the explicit geometry, we present a method for extracting a point cloud from discretized UDF values on a sparse grid. We compare different SDFs and UDFs for the scene completion task on indoor and outdoor point clouds collected using RGB-D and LiDAR sensors and show improved completion using the proposed UDF function.
△ Less
Submitted 2 December, 2022; v1 submitted 17 March, 2022;
originally announced March 2022.
-
AdaSplats: Adaptive Splatting of Point Clouds for Accurate 3D Modeling and Real-time High-Fidelity LiDAR Simulation
Authors:
Jean Pierre Richa,
Jean-Emmanuel Deschaud,
François Goulette,
Nicolas Dalmasso
Abstract:
LiDAR sensors provide rich 3D information about their surrounding{s} and are becoming increasingly important for autonomous vehicles tasks such as {localization}, semantic segmentation, object detection, and tracking. {Simulation} accelerates the testing, validation, and deployment of autonomous vehicles while {also} reducing cost and eliminating the risks of testing in real-world scenarios. We ad…
▽ More
LiDAR sensors provide rich 3D information about their surrounding{s} and are becoming increasingly important for autonomous vehicles tasks such as {localization}, semantic segmentation, object detection, and tracking. {Simulation} accelerates the testing, validation, and deployment of autonomous vehicles while {also} reducing cost and eliminating the risks of testing in real-world scenarios. We address the problem of high-fidelity LiDAR simulation and present a pipeline that leverages real-world point clouds acquired by mobile map** systems. Point-based geometry representations, more specifically splats {(2D oriented disks with normals)}, have proven their ability to accurately model the underlying surface in large point clouds{, mainly with uniform density}. We introduce an adaptive splat generation method that accurately models the underlying 3D geometry {to handle real-world point clouds with variable densities}, especially for thin structures. Moreover, we introduce a {fast} LiDAR {sensor} simulator, {working} in the splatted model, {that leverages} the GPU parallel architecture with an acceleration structure while focusing on efficiently handling large point clouds. We test our LiDAR simulation in real-world conditions, showing qualitative and quantitative results compared to basic splatting and meshing techniques, demonstrating the interest of our modeling technique.
△ Less
Submitted 26 December, 2022; v1 submitted 17 March, 2022;
originally announced March 2022.
-
COLA: COarse LAbel pre-training for 3D semantic segmentation of sparse LiDAR datasets
Authors:
Jules Sanchez,
Jean-Emmanuel Deschaud,
François Goulette
Abstract:
Transfer learning is a proven technique in 2D computer vision to leverage the large amount of data available and achieve high performance with datasets limited in size due to the cost of acquisition or annotation. In 3D, annotation is known to be a costly task; nevertheless, pre-training methods have only recently been investigated. Due to this cost, unsupervised pre-training has been heavily favo…
▽ More
Transfer learning is a proven technique in 2D computer vision to leverage the large amount of data available and achieve high performance with datasets limited in size due to the cost of acquisition or annotation. In 3D, annotation is known to be a costly task; nevertheless, pre-training methods have only recently been investigated. Due to this cost, unsupervised pre-training has been heavily favored. In this work, we tackle the case of real-time 3D semantic segmentation of sparse autonomous driving LiDAR scans. Such datasets have been increasingly released, but each has a unique label set. We propose here an intermediate-level label set called coarse labels, which can easily be used on any existing and future autonomous driving datasets, thus allowing all the data available to be leveraged at once without any additional manual labeling. This way, we have access to a larger dataset, alongside a simple task of semantic segmentation. With it, we introduce a new pre-training task: coarse label pre-training, also called COLA. We thoroughly analyze the impact of COLA on various datasets and architectures and show that it yields a noticeable performance improvement, especially when only a small dataset is available for the finetuning task.
△ Less
Submitted 21 March, 2023; v1 submitted 14 February, 2022;
originally announced February 2022.
-
Paris-CARLA-3D: A Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D Map**
Authors:
Jean-Emmanuel Deschaud,
David Duque,
Jean Pierre Richa,
Santiago Velasco-Forero,
Beatriz Marcotegui,
and François Goulette
Abstract:
Paris-CARLA-3D is a dataset of several dense colored point clouds of outdoor environments built by a mobile LiDAR and camera system. The data are composed of two sets with synthetic data from the open source CARLA simulator (700 million points) and real data acquired in the city of Paris (60 million points), hence the name Paris-CARLA-3D. One of the advantages of this dataset is to have simulated…
▽ More
Paris-CARLA-3D is a dataset of several dense colored point clouds of outdoor environments built by a mobile LiDAR and camera system. The data are composed of two sets with synthetic data from the open source CARLA simulator (700 million points) and real data acquired in the city of Paris (60 million points), hence the name Paris-CARLA-3D. One of the advantages of this dataset is to have simulated the same LiDAR and camera platform in the open source CARLA simulator as the one used to produce the real data. In addition, manual annotation of the classes using the semantic tags of CARLA was performed on the real data, allowing the testing of transfer methods from the synthetic to the real data. The objective of this dataset is to provide a challenging dataset to evaluate and improve methods on difficult vision tasks for the 3D map** of outdoor environments: semantic segmentation, instance segmentation, and scene completion. For each task, we describe the evaluation protocol as well as the experiments carried out to establish a baseline.
△ Less
Submitted 22 November, 2021;
originally announced November 2021.
-
Riedones3D: a celtic coin dataset for registration and fine-grained clustering
Authors:
Sofiane Horache,
Jean-Emmanuel Deschaud,
François Goulette,
Katherine Gruel,
Thierry Lejars,
Olivier Masson
Abstract:
Clustering coins with respect to their die is an important component of numismatic research and crucial for understanding the economic history of tribes (especially when literary production does not exist, in celtic culture). It is a very hard task that requires a lot of times and expertise. To cluster thousands of coins, automatic methods are becoming necessary. Nevertheless, public datasets for…
▽ More
Clustering coins with respect to their die is an important component of numismatic research and crucial for understanding the economic history of tribes (especially when literary production does not exist, in celtic culture). It is a very hard task that requires a lot of times and expertise. To cluster thousands of coins, automatic methods are becoming necessary. Nevertheless, public datasets for coin die clustering evaluation are too rare, though they are very important for the development of new methods. Therefore, we propose a new 3D dataset of 2 070 scans of coins. With this dataset, we propose two benchmarks, one for point cloud registration, essential for coin die recognition, and a benchmark of coin die clustering. We show how we automatically cluster coins to help experts, and perform a preliminary evaluation for these two tasks. The code of the baseline and the dataset will be publicly available at https://www.npm3d.fr/coins-riedones3d and https://www.chronocarto.eu/spip.php?article84&lang=fr
△ Less
Submitted 30 September, 2021;
originally announced September 2021.
-
CT-ICP: Real-time Elastic LiDAR Odometry with Loop Closure
Authors:
Pierre Dellenbach,
Jean-Emmanuel Deschaud,
Bastien Jacquet,
François Goulette
Abstract:
Multi-beam LiDAR sensors are increasingly used in robotics, particularly with autonomous cars for localization and perception tasks, both relying on the ability to build a precise map of the environment. For this, we propose a new real-time LiDAR-only odometry method called CT-ICP (for Continuous-Time ICP), completed into a full SLAM with a novel loop detection procedure. The core of this method,…
▽ More
Multi-beam LiDAR sensors are increasingly used in robotics, particularly with autonomous cars for localization and perception tasks, both relying on the ability to build a precise map of the environment. For this, we propose a new real-time LiDAR-only odometry method called CT-ICP (for Continuous-Time ICP), completed into a full SLAM with a novel loop detection procedure. The core of this method, is the introduction of the combined continuity in the scan matching, and discontinuity between scans. It allows both the elastic distortion of the scan during the registration for increased precision, and the increased robustness to high frequency motions from the discontinuity.
We build a complete SLAM on top of this odometry, using a fast pure LiDAR loop detection based on elevation image 2D matching, providing a pose graph with loop constraints. To show the robustness of the method, we tested it on seven datasets: KITTI, KITTI-raw, KITTI-360, KITTI-CARLA, ParisLuco, Newer College, and NCLT in driving and high-frequency motion scenarios. Both the CT-ICP odometry and the loop detection are made available online. CT-ICP is currently first, among those giving access to a public code, on the KITTI odometry leaderboard, with an average Relative Translation Error (RTE) of 0.59% and an average time per scan of 60ms on a CPU with a single thread.
△ Less
Submitted 24 February, 2022; v1 submitted 27 September, 2021;
originally announced September 2021.
-
3D Point Cloud Registration with Multi-Scale Architecture and Unsupervised Transfer Learning
Authors:
Sofiane Horache,
Jean-Emmanuel Deschaud,
François Goulette
Abstract:
We propose a method for generalizing deep learning for 3D point cloud registration on new, totally different datasets. It is based on two components, MS-SVConv and UDGE. Using Multi-Scale Sparse Voxel Convolution, MS-SVConv is a fast deep neural network that outputs the descriptors from point clouds for 3D registration between two scenes. UDGE is an algorithm for transferring deep networks on unkn…
▽ More
We propose a method for generalizing deep learning for 3D point cloud registration on new, totally different datasets. It is based on two components, MS-SVConv and UDGE. Using Multi-Scale Sparse Voxel Convolution, MS-SVConv is a fast deep neural network that outputs the descriptors from point clouds for 3D registration between two scenes. UDGE is an algorithm for transferring deep networks on unknown datasets in a unsupervised way. The interest of the proposed method appears while using the two components, MS-SVConv and UDGE, together as a whole, which leads to state-of-the-art results on real world registration datasets such as 3DMatch, ETH and TUM. The code is publicly available at https://github.com/humanpose1/MS-SVConv .
△ Less
Submitted 14 October, 2021; v1 submitted 26 March, 2021;
originally announced March 2021.
-
What's in My LiDAR Odometry Toolbox?
Authors:
Pierre Dellenbach,
Jean-Emmanuel Deschaud,
Bastien Jacquet,
François Goulette
Abstract:
With the democratization of 3D LiDAR sensors, precise LiDAR odometries and SLAM are in high demand. New methods regularly appear, proposing solutions ranging from small variations in classical algorithms to radically new paradigms based on deep learning. Yet it is often difficult to compare these methods, notably due to the few datasets on which the methods can be evaluated and compared. Furthermo…
▽ More
With the democratization of 3D LiDAR sensors, precise LiDAR odometries and SLAM are in high demand. New methods regularly appear, proposing solutions ranging from small variations in classical algorithms to radically new paradigms based on deep learning. Yet it is often difficult to compare these methods, notably due to the few datasets on which the methods can be evaluated and compared. Furthermore, their weaknesses are rarely examined, often letting the user discover the hard way whether a method would be appropriate for a use case. In this paper, we review and organize the main 3D LiDAR odometries into distinct categories. We implemented several approaches (geometric based, deep learning based, and hybrid methods) to conduct an in-depth analysis of their strengths and weaknesses on multiple datasets, guiding the reader through the different LiDAR odometries available. Implementation of the methods has been made publicly available at https://github.com/Kitware/pyLiDAR-SLAM.
△ Less
Submitted 7 October, 2021; v1 submitted 17 March, 2021;
originally announced March 2021.
-
Automatic clustering of Celtic coins based on 3D point cloud pattern analysis
Authors:
Sofiane Horache,
Jean-Emmanuel Deschaud,
François Goulette,
Katherine Gruel,
Thierry Lejars
Abstract:
The recognition and clustering of coins which have been struck by the same die is of interest for archeological studies. Nowadays, this work can only be performed by experts and is very tedious. In this paper, we propose a method to automatically cluster dies, based on 3D scans of coins. It is based on three steps: registration, comparison and graph-based clustering. Experimental results on 90 coi…
▽ More
The recognition and clustering of coins which have been struck by the same die is of interest for archeological studies. Nowadays, this work can only be performed by experts and is very tedious. In this paper, we propose a method to automatically cluster dies, based on 3D scans of coins. It is based on three steps: registration, comparison and graph-based clustering. Experimental results on 90 coins coming from a Celtic treasury from the II-Ith century BC show a clustering quality equivalent to expert's work.
△ Less
Submitted 12 May, 2020;
originally announced May 2020.
-
KPConv: Flexible and Deformable Convolution for Point Clouds
Authors:
Hugues Thomas,
Charles R. Qi,
Jean-Emmanuel Deschaud,
Beatriz Marcotegui,
François Goulette,
Leonidas J. Guibas
Abstract:
We present Kernel Point Convolution (KPConv), a new design of point convolution, i.e. that operates on point clouds without any intermediate representation. The convolution weights of KPConv are located in Euclidean space by kernel points, and applied to the input points close to them. Its capacity to use any number of kernel points gives KPConv more flexibility than fixed grid convolutions. Furth…
▽ More
We present Kernel Point Convolution (KPConv), a new design of point convolution, i.e. that operates on point clouds without any intermediate representation. The convolution weights of KPConv are located in Euclidean space by kernel points, and applied to the input points close to them. Its capacity to use any number of kernel points gives KPConv more flexibility than fixed grid convolutions. Furthermore, these locations are continuous in space and can be learned by the network. Therefore, KPConv can be extended to deformable convolutions that learn to adapt kernel points to local geometry. Thanks to a regular subsampling strategy, KPConv is also efficient and robust to varying densities. Whether they use deformable KPConv for complex tasks, or rigid KPconv for simpler tasks, our networks outperform state-of-the-art classification and segmentation approaches on several datasets. We also offer ablation studies and visualizations to provide understanding of what has been learned by KPConv and to validate the descriptive power of deformable KPConv.
△ Less
Submitted 19 August, 2019; v1 submitted 18 April, 2019;
originally announced April 2019.
-
Computational Fluid Dynamics on 3D Point Set Surfaces
Authors:
Hassan Bouchiba,
Simon Santoso,
Jean-Emmanuel Deschaud,
Luisa Rocha-Da-Silva,
François Goulette,
Thierry Coupez
Abstract:
Computational fluid dynamics (CFD) in many cases requires designing 3D models manually, which is a tedious task that requires specific skills. In this paper, we present a novel method for performing CFD directly on scanned 3D point clouds. The proposed method builds an anisotropic volumetric tetrahedral mesh adapted around a point-sampled surface, without an explicit surface reconstruction step. T…
▽ More
Computational fluid dynamics (CFD) in many cases requires designing 3D models manually, which is a tedious task that requires specific skills. In this paper, we present a novel method for performing CFD directly on scanned 3D point clouds. The proposed method builds an anisotropic volumetric tetrahedral mesh adapted around a point-sampled surface, without an explicit surface reconstruction step. The surface is represented by a new extended implicit moving least squares (EIMLS) scalar representation that extends the definition of the function to the entire computational domain, which makes it possible for use in immersed boundary flow simulations. The workflow we present allows us to compute flows around point-sampled geometries automatically. It also gives a better control of the precision around the surface with a limited number of computational nodes, which is a critical issue in CFD.
△ Less
Submitted 17 December, 2018;
originally announced January 2019.
-
Semantic Classification of 3D Point Clouds with Multiscale Spherical Neighborhoods
Authors:
Hugues Thomas,
Jean-Emmanuel Deschaud,
Beatriz Marcotegui,
François Goulette,
Yann Le Gall
Abstract:
This paper introduces a new definition of multiscale neighborhoods in 3D point clouds. This definition, based on spherical neighborhoods and proportional subsampling, allows the computation of features with a consistent geometrical meaning, which is not the case when using k-nearest neighbors. With an appropriate learning strategy, the proposed features can be used in a random forest to classify 3…
▽ More
This paper introduces a new definition of multiscale neighborhoods in 3D point clouds. This definition, based on spherical neighborhoods and proportional subsampling, allows the computation of features with a consistent geometrical meaning, which is not the case when using k-nearest neighbors. With an appropriate learning strategy, the proposed features can be used in a random forest to classify 3D points. In this semantic classification task, we show that our multiscale features outperform state-of-the-art features using the same experimental conditions. Furthermore, their classification power competes with more elaborate classification approaches including Deep Learning methods.
△ Less
Submitted 1 August, 2018;
originally announced August 2018.
-
Classification of Point Cloud Scenes with Multiscale Voxel Deep Network
Authors:
Xavier Roynard,
Jean-Emmanuel Deschaud,
François Goulette
Abstract:
In this article we describe a new convolutional neural network (CNN) to classify 3D point clouds of urban or indoor scenes. Solutions are given to the problems encountered working on scene point clouds, and a network is described that allows for point classification using only the position of points in a multi-scale neighborhood.
On the reduced-8 Semantic3D benchmark [Hackel et al., 2017], this…
▽ More
In this article we describe a new convolutional neural network (CNN) to classify 3D point clouds of urban or indoor scenes. Solutions are given to the problems encountered working on scene point clouds, and a network is described that allows for point classification using only the position of points in a multi-scale neighborhood.
On the reduced-8 Semantic3D benchmark [Hackel et al., 2017], this network, ranked second, beats the state of the art of point classification methods (those not using a regularization step).
△ Less
Submitted 10 April, 2018;
originally announced April 2018.
-
Paris-Lille-3D: a large and high-quality ground truth urban point cloud dataset for automatic segmentation and classification
Authors:
Xavier Roynard,
Jean-Emmanuel Deschaud,
François Goulette
Abstract:
This paper introduces a new Urban Point Cloud Dataset for Automatic Segmentation and Classification acquired by Mobile Laser Scanning (MLS). We describe how the dataset is obtained from acquisition to post-processing and labeling. This dataset can be used to learn classification algorithm, however, given that a great attention has been paid to the split between the different objects, this dataset…
▽ More
This paper introduces a new Urban Point Cloud Dataset for Automatic Segmentation and Classification acquired by Mobile Laser Scanning (MLS). We describe how the dataset is obtained from acquisition to post-processing and labeling. This dataset can be used to learn classification algorithm, however, given that a great attention has been paid to the split between the different objects, this dataset can also be used to learn the segmentation. The dataset consists of around 2km of MLS point cloud acquired in two cities. The number of points and range of classes make us consider that it can be used to train Deep-Learning methods. Besides we show some results of automatic segmentation and classification. The dataset is available at: http://caor-mines-paristech.fr/fr/paris-lille-3d-dataset/
△ Less
Submitted 10 April, 2018; v1 submitted 30 November, 2017;
originally announced December 2017.
-
On the Diagnostic of Road Pathway Visibility
Authors:
Pierre Charbonnier,
Jean-Philippe Tarel,
Francois Goulette
Abstract:
Visibility distance on the road pathway plays a significant role in road safety and in particular, has a clear impact on the choice of speed limits. Visibility distance is thus of importance for road engineers and authorities. While visibility distance criteria are routinely taken into account in road design, only a few systems exist for estimating it on existing road networks. Most existing syste…
▽ More
Visibility distance on the road pathway plays a significant role in road safety and in particular, has a clear impact on the choice of speed limits. Visibility distance is thus of importance for road engineers and authorities. While visibility distance criteria are routinely taken into account in road design, only a few systems exist for estimating it on existing road networks. Most existing systems comprise a target vehicle followed at a constant distance by an observer vehicle, which only allows to check if a given, fixed visibility distance is available. We propose two new approaches that allow estimating the maximum available visibility distance, involving only one vehicle and based on different sensor technologies, namely binocular stereovision and 3D range sensing (LIDAR). The first approach is based on the processing of two views taken by digital cameras onboard the diagnostic vehicle. The main stages of the process are: road segmentation, edge registration between the two views, road profile 3D reconstruction and finally, maximal road visibility distance estimation. The second approach involves the use of a Terrestrial LIDAR Mobile Map** System. The triangulated 3D model of the road and its surroundings provided by the system is used to simulate targets at different distances, which allows estimating the maximum geometric visibility distance along the pathway. These approaches were developed in the context of the SARI-VIZIR PREDIT project. Both approaches are described, evaluated and compared. Their pros and cons with respect to vehicle following systems are also discussed.
△ Less
Submitted 21 January, 2016;
originally announced January 2016.
-
Invariant EKF Design for Scan Matching-aided Localization
Authors:
Martin Barczyk,
Silvère Bonnabel,
Jean-Emmanuel Deschaud,
François Goulette
Abstract:
Localization in indoor environments is a technique which estimates the robot's pose by fusing data from onboard motion sensors with readings of the environment, in our case obtained by scan matching point clouds captured by a low-cost Kinect depth camera. We develop both an Invariant Extended Kalman Filter (IEKF)-based and a Multiplicative Extended Kalman Filter (MEKF)-based solution to this probl…
▽ More
Localization in indoor environments is a technique which estimates the robot's pose by fusing data from onboard motion sensors with readings of the environment, in our case obtained by scan matching point clouds captured by a low-cost Kinect depth camera. We develop both an Invariant Extended Kalman Filter (IEKF)-based and a Multiplicative Extended Kalman Filter (MEKF)-based solution to this problem. The two designs are successfully validated in experiments and demonstrate the advantage of the IEKF design.
△ Less
Submitted 4 March, 2015;
originally announced March 2015.
-
Colorisation et texturation temps réel d'environnements urbains par système mobile avec scanner laser et caméra fish-eye
Authors:
Jean-Emmanuel Deschaud,
Xavier Brun,
François Goulette
Abstract:
We present here a real time mobile map** system mounted on a vehicle. The terrestrial acquisition system is based on a geolocation system and two sensors, namely, a laser scanner and a camera with a fish-eye lens. We produce 3D colored points cloud and textured models of the environment. Once the system has been calibrated, the data acquisition and processing are done "on the way". This article…
▽ More
We present here a real time mobile map** system mounted on a vehicle. The terrestrial acquisition system is based on a geolocation system and two sensors, namely, a laser scanner and a camera with a fish-eye lens. We produce 3D colored points cloud and textured models of the environment. Once the system has been calibrated, the data acquisition and processing are done "on the way". This article mainly presents our methods of colorization of point cloud, triangulation and texture map**.
△ Less
Submitted 3 December, 2014;
originally announced December 2014.
-
On the Covariance of ICP-based Scan-matching Techniques
Authors:
Silvère Bonnabel,
Martin Barczyk,
François Goulette
Abstract:
This paper considers the problem of estimating the covariance of roto-translations computed by the Iterative Closest Point (ICP) algorithm. The problem is relevant for localization of mobile robots and vehicles equipped with depth-sensing cameras (e.g., Kinect) or Lidar (e.g., Velodyne). The closed-form formulas for covariance proposed in previous literature generally build upon the fact that the…
▽ More
This paper considers the problem of estimating the covariance of roto-translations computed by the Iterative Closest Point (ICP) algorithm. The problem is relevant for localization of mobile robots and vehicles equipped with depth-sensing cameras (e.g., Kinect) or Lidar (e.g., Velodyne). The closed-form formulas for covariance proposed in previous literature generally build upon the fact that the solution to ICP is obtained by minimizing a linear least-squares problem. In this paper, we show this approach needs caution because the rematching step of the algorithm is not explicitly accounted for, and applying it to the point-to-point version of ICP leads to completely erroneous covariances. We then provide a formal mathematical proof why the approach is valid in the point-to-plane version of ICP, which validates the intuition and experimental results of practitioners.
△ Less
Submitted 16 March, 2016; v1 submitted 15 October, 2014;
originally announced October 2014.
-
Experimental Implementation of an Invariant Extended Kalman Filter-based Scan Matching SLAM
Authors:
Martin Barczyk,
Silvère Bonnabel,
Jean-Emmanuel Deschaud,
François Goulette
Abstract:
We describe an application of the Invariant Extended Kalman Filter (IEKF) design methodology to the scan matching SLAM problem. We review the theoretical foundations of the IEKF and its practical interest of guaranteeing robustness to poor state estimates, then implement the filter on a wheeled robot hardware platform. The proposed design is successfully validated in experimental testing.
We describe an application of the Invariant Extended Kalman Filter (IEKF) design methodology to the scan matching SLAM problem. We review the theoretical foundations of the IEKF and its practical interest of guaranteeing robustness to poor state estimates, then implement the filter on a wheeled robot hardware platform. The proposed design is successfully validated in experimental testing.
△ Less
Submitted 20 March, 2014;
originally announced March 2014.
-
Accurate 3D maps from depth images and motion sensors via nonlinear Kalman filtering
Authors:
Thibault Hervier,
Silvère Bonnabel,
François Goulette
Abstract:
This paper investigates the use of depth images as localisation sensors for 3D map building. The localisation information is derived from the 3D data thanks to the ICP (Iterative Closest Point) algorithm. The covariance of the ICP, and thus of the localization error, is analysed, and described by a Fisher Information Matrix. It is advocated this error can be much reduced if the data is fused with…
▽ More
This paper investigates the use of depth images as localisation sensors for 3D map building. The localisation information is derived from the 3D data thanks to the ICP (Iterative Closest Point) algorithm. The covariance of the ICP, and thus of the localization error, is analysed, and described by a Fisher Information Matrix. It is advocated this error can be much reduced if the data is fused with measurements from other motion sensors, or even with prior knowledge on the motion. The data fusion is performed by a recently introduced specific extended Kalman filter, the so-called Invariant EKF, and is directly based on the estimated covariance of the ICP. The resulting filter is very natural, and is proved to possess strong properties. Experiments with a Kinect sensor and a three-axis gyroscope prove clear improvement in the accuracy of the localization, and thus in the accuracy of the built 3D map.
△ Less
Submitted 16 May, 2012;
originally announced May 2012.
