-
Online Invariance Selection for Local Feature Descriptors
Authors:
Rémi Pautrat,
Viktor Larsson,
Martin R. Oswald,
Marc Pollefeys
Abstract:
To be invariant, or not to be invariant: that is the question formulated in this work about local descriptors. A limitation of current feature descriptors is the trade-off between generalization and discriminative power: more invariance means less informative descriptors. We propose to overcome this limitation with a disentanglement of invariance in local descriptors and with an online selection o…
▽ More
To be invariant, or not to be invariant: that is the question formulated in this work about local descriptors. A limitation of current feature descriptors is the trade-off between generalization and discriminative power: more invariance means less informative descriptors. We propose to overcome this limitation with a disentanglement of invariance in local descriptors and with an online selection of the most appropriate invariance given the context. Our framework consists in a joint learning of multiple local descriptors with different levels of invariance and of meta descriptors encoding the regional variations of an image. The similarity of these meta descriptors across images is used to select the right invariance when matching the local descriptors. Our approach, named Local Invariance Selection at Runtime for Descriptors (LISRD), enables descriptors to adapt to adverse changes in images, while remaining discriminative when invariance is not required. We demonstrate that our method can boost the performance of current descriptors and outperforms state-of-the-art descriptors in several matching tasks, when evaluated on challenging datasets with day-night illumination as well as viewpoint changes.
△ Less
Submitted 23 July, 2020; v1 submitted 17 July, 2020;
originally announced July 2020.
-
AdaLAM: Revisiting Handcrafted Outlier Detection
Authors:
Luca Cavalli,
Viktor Larsson,
Martin Ralf Oswald,
Torsten Sattler,
Marc Pollefeys
Abstract:
Local feature matching is a critical component of many computer vision pipelines, including among others Structure-from-Motion, SLAM, and Visual Localization. However, due to limitations in the descriptors, raw matches are often contaminated by a majority of outliers. As a result, outlier detection is a fundamental problem in computer vision, and a wide range of approaches have been proposed over…
▽ More
Local feature matching is a critical component of many computer vision pipelines, including among others Structure-from-Motion, SLAM, and Visual Localization. However, due to limitations in the descriptors, raw matches are often contaminated by a majority of outliers. As a result, outlier detection is a fundamental problem in computer vision, and a wide range of approaches have been proposed over the last decades. In this paper we revisit handcrafted approaches to outlier filtering. Based on best practices, we propose a hierarchical pipeline for effective outlier detection as well as integrate novel ideas which in sum lead to AdaLAM, an efficient and competitive approach to outlier rejection. AdaLAM is designed to effectively exploit modern parallel hardware, resulting in a very fast, yet very accurate, outlier filter. We validate AdaLAM on multiple large and diverse datasets, and we submit to the Image Matching Challenge (CVPR2020), obtaining competitive results with simple baseline descriptors. We show that AdaLAM is more than competitive to current state of the art, both in terms of efficiency and effectiveness.
△ Less
Submitted 7 June, 2020;
originally announced June 2020.
-
Accurate Map** and Planning for Autonomous Racing
Authors:
Leiv Andresen,
Adrian Brandemuehl,
Alex Hönger,
Benson Kuan,
Niclas Vödisch,
Hermann Blum,
Victor Reijgwart,
Lukas Bernreiter,
Lukas Schaupp,
Jen Jen Chung,
Mathias Bürki,
Martin R. Oswald,
Roland Siegwart,
Abel Gawel
Abstract:
This paper presents the perception, map**, and planning pipeline implemented on an autonomous race car. It was developed by the 2019 AMZ driverless team for the Formula Student Germany (FSG) 2019 driverless competition, where it won 1st place overall. The presented solution combines early fusion of camera and LiDAR data, a layered map** approach, and a planning approach that uses Bayesian filt…
▽ More
This paper presents the perception, map**, and planning pipeline implemented on an autonomous race car. It was developed by the 2019 AMZ driverless team for the Formula Student Germany (FSG) 2019 driverless competition, where it won 1st place overall. The presented solution combines early fusion of camera and LiDAR data, a layered map** approach, and a planning approach that uses Bayesian filtering to achieve high-speed driving on unknown race tracks while creating accurate maps. We benchmark the method against our team's previous solution, which won FSG 2018, and show improved accuracy when driving at the same speeds. Furthermore, the new pipeline makes it possible to reliably raise the maximum driving speed in unknown environments from 3~m/s to 12~m/s while still map** with an acceptable RMSE of 0.29~m.
△ Less
Submitted 17 September, 2020; v1 submitted 11 March, 2020;
originally announced March 2020.
-
Learned Multi-View Texture Super-Resolution
Authors:
Audrey Richard,
Ian Cherabier,
Martin R. Oswald,
Vagia Tsiminaki,
Marc Pollefeys,
Konrad Schindler
Abstract:
We present a super-resolution method capable of creating a high-resolution texture map for a virtual 3D object from a set of lower-resolution images of that object. Our architecture unifies the concepts of (i) multi-view super-resolution based on the redundancy of overlap** views and (ii) single-view super-resolution based on a learned prior of high-resolution (HR) image structure. The principle…
▽ More
We present a super-resolution method capable of creating a high-resolution texture map for a virtual 3D object from a set of lower-resolution images of that object. Our architecture unifies the concepts of (i) multi-view super-resolution based on the redundancy of overlap** views and (ii) single-view super-resolution based on a learned prior of high-resolution (HR) image structure. The principle of multi-view super-resolution is to invert the image formation process and recover the latent HR texture from multiple lower-resolution projections. We map that inverse problem into a block of suitably designed neural network layers, and combine it with a standard encoder-decoder network for learned single-image super-resolution. Wiring the image formation model into the network avoids having to learn perspective map** from textures to images, and elegantly handles a varying number of input views. Experiments demonstrate that the combination of multi-view observations and learned prior yields improved texture maps.
△ Less
Submitted 14 January, 2020;
originally announced January 2020.
-
RoutedFusion: Learning Real-time Depth Map Fusion
Authors:
Silvan Weder,
Johannes L. Schönberger,
Marc Pollefeys,
Martin R. Oswald
Abstract:
The efficient fusion of depth maps is a key part of most state-of-the-art 3D reconstruction methods. Besides requiring high accuracy, these depth fusion methods need to be scalable and real-time capable. To this end, we present a novel real-time capable machine learning-based method for depth map fusion. Similar to the seminal depth map fusion approach by Curless and Levoy, we only update a local…
▽ More
The efficient fusion of depth maps is a key part of most state-of-the-art 3D reconstruction methods. Besides requiring high accuracy, these depth fusion methods need to be scalable and real-time capable. To this end, we present a novel real-time capable machine learning-based method for depth map fusion. Similar to the seminal depth map fusion approach by Curless and Levoy, we only update a local group of voxels to ensure real-time capability. Instead of a simple linear fusion of depth information, we propose a neural network that predicts non-linear updates to better account for typical fusion errors. Our network is composed of a 2D depth routing network and a 3D depth fusion network which efficiently handle sensor-specific noise and outliers. This is especially useful for surface edges and thin objects for which the original approach suffers from thickening artifacts. Our method outperforms the traditional fusion approach and related learned approaches on both synthetic and real data. We demonstrate the performance of our method in reconstructing fine geometric details from noise and outlier contaminated data on various scenes.
△ Less
Submitted 3 April, 2020; v1 submitted 13 January, 2020;
originally announced January 2020.
-
Learned Semantic Multi-Sensor Depth Map Fusion
Authors:
Denys Rozumnyi,
Ian Cherabier,
Marc Pollefeys,
Martin R. Oswald
Abstract:
Volumetric depth map fusion based on truncated signed distance functions has become a standard method and is used in many 3D reconstruction pipelines. In this paper, we are generalizing this classic method in multiple ways: 1) Semantics: Semantic information enriches the scene representation and is incorporated into the fusion process. 2) Multi-Sensor: Depth information can originate from differen…
▽ More
Volumetric depth map fusion based on truncated signed distance functions has become a standard method and is used in many 3D reconstruction pipelines. In this paper, we are generalizing this classic method in multiple ways: 1) Semantics: Semantic information enriches the scene representation and is incorporated into the fusion process. 2) Multi-Sensor: Depth information can originate from different sensors or algorithms with very different noise and outlier statistics which are considered during data fusion. 3) Scene denoising and completion: Sensors can fail to recover depth for certain materials and light conditions, or data is missing due to occlusions. Our method denoises the geometry, closes holes and computes a watertight surface for every semantic class. 4) Learning: We propose a neural network reconstruction method that unifies all these properties within a single powerful framework. Our method learns sensor or algorithm properties jointly with semantic depth fusion and scene completion and can also be used as an expert system, e.g. to unify the strengths of various photometric stereo algorithms. Our approach is the first to unify all these properties. Experimental evaluations on both synthetic and real data sets demonstrate clear improvements.
△ Less
Submitted 2 September, 2019;
originally announced September 2019.
-
3D Instance Segmentation via Multi-Task Metric Learning
Authors:
Jean Lahoud,
Bernard Ghanem,
Marc Pollefeys,
Martin R. Oswald
Abstract:
We propose a novel method for instance label segmentation of dense 3D voxel grids. We target volumetric scene representations, which have been acquired with depth sensors or multi-view stereo methods and which have been processed with semantic 3D reconstruction or scene completion methods. The main task is to learn shape information about individual object instances in order to accurately separate…
▽ More
We propose a novel method for instance label segmentation of dense 3D voxel grids. We target volumetric scene representations, which have been acquired with depth sensors or multi-view stereo methods and which have been processed with semantic 3D reconstruction or scene completion methods. The main task is to learn shape information about individual object instances in order to accurately separate them, including connected and incompletely scanned objects. We solve the 3D instance-labeling problem with a multi-task learning strategy. The first goal is to learn an abstract feature embedding, which groups voxels with the same instance label close to each other while separating clusters with different instance labels from each other. The second goal is to learn instance information by densely estimating directional information of the instance's center of mass for each voxel. This is particularly useful to find instance boundaries in the clustering post-processing step, as well as, for scoring the segmentation quality for the first goal. Both synthetic and real-world experiments demonstrate the viability and merits of our approach. In fact, it achieves state-of-the-art performance on the ScanNet 3D instance segmentation benchmark.
△ Less
Submitted 31 October, 2019; v1 submitted 20 June, 2019;
originally announced June 2019.
-
Semantically Informed Multiview Surface Refinement
Authors:
Maros Blaha,
Mathias Rothermel,
Martin R. Oswald,
Torsten Sattler,
Audrey Richard,
Jan D. Wegner,
Marc Pollefeys,
Konrad Schindler
Abstract:
We present a method to jointly refine the geometry and semantic segmentation of 3D surface meshes. Our method alternates between updating the shape and the semantic labels. In the geometry refinement step, the mesh is deformed with variational energy minimization, such that it simultaneously maximizes photo-consistency and the compatibility of the semantic segmentations across a set of calibrated…
▽ More
We present a method to jointly refine the geometry and semantic segmentation of 3D surface meshes. Our method alternates between updating the shape and the semantic labels. In the geometry refinement step, the mesh is deformed with variational energy minimization, such that it simultaneously maximizes photo-consistency and the compatibility of the semantic segmentations across a set of calibrated images. Label-specific shape priors account for interactions between the geometry and the semantic labels in 3D. In the semantic segmentation step, the labels on the mesh are updated with MRF inference, such that they are compatible with the semantic segmentations in the input images. Also, this step includes prior assumptions about the surface shape of different semantic classes. The priors induce a tight coupling, where semantic information influences the shape update and vice versa. Specifically, we introduce priors that favor (i) adaptive smoothing, depending on the class label; (ii) straightness of class boundaries; and (iii) semantic labels that are consistent with the surface orientation. The novel mesh-based reconstruction is evaluated in a series of experiments with real and synthetic data. We compare both to state-of-the-art, voxel-based semantic 3D reconstruction, and to purely geometric mesh refinement, and demonstrate that the proposed scheme yields improved 3D geometry as well as an improved semantic segmentation.
△ Less
Submitted 26 June, 2017;
originally announced June 2017.
-
Geometrie der Zahlen. Ein Überblick
Authors:
Nicola M. R. Oswald
Abstract:
This article provides a historical overview of Geometry of Numbers.
1. Figures, 2. The circuit problem and its relatives, 3. Minkowski lattice point set, 4. The young Hermann Minkowski, 5. The geometry of numbers develops, 6. Minkowski geometry in the context of time, 7. Successive minima, 8. Minkowski space-time, 9. Voronoi and Blichfeldt, 10. The schools in Manchester and Vienna, 11. packing…
▽ More
This article provides a historical overview of Geometry of Numbers.
1. Figures, 2. The circuit problem and its relatives, 3. Minkowski lattice point set, 4. The young Hermann Minkowski, 5. The geometry of numbers develops, 6. Minkowski geometry in the context of time, 7. Successive minima, 8. Minkowski space-time, 9. Voronoi and Blichfeldt, 10. The schools in Manchester and Vienna, 11. packing problems
△ Less
Submitted 13 May, 2016;
originally announced May 2016.
-
Aspects of Zeta-Function Theory in the Mathematical Works of Adolf Hurwitz
Authors:
Nicola M. R. Oswald,
Jörn Steuding
Abstract:
Adolf Hurwitz is rather famous for his celebrated contributions to Riemann surfaces, modular forms, diophantine equations and approximation as well as to certain aspects of algebra. His early work on an important generalization of Dirichlet's $L$-series, nowadays called Hurwitz zeta-function, is the only published work settled in the very active field of research around the Riemann zeta-function a…
▽ More
Adolf Hurwitz is rather famous for his celebrated contributions to Riemann surfaces, modular forms, diophantine equations and approximation as well as to certain aspects of algebra. His early work on an important generalization of Dirichlet's $L$-series, nowadays called Hurwitz zeta-function, is the only published work settled in the very active field of research around the Riemann zeta-function and its relatives. His mathematical diaries, however, provide another picture, namely a lifelong interest in the development of zeta-function theory. In this note we shall investigate his early work, its origin and its reception, as well as Hurwitz's further studies of the Riemann zeta-function and allied Dirichlet series from his diaries. It turns out that Hurwitz already in 1889 knew about the essential analytic properties of the Epstein zeta-function (including its functional equation) 13 years before Paul Epstein.
△ Less
Submitted 2 June, 2015;
originally announced June 2015.
