-
Self-Supervised Traversability Prediction by Learning to Reconstruct Safe Terrain
Authors:
Robin Schmid,
Deegan Atha,
Frederik Schöller,
Sharmita Dey,
Seyed Fakoorian,
Kyohei Otsu,
Barry Ridge,
Marko Bjelonic,
Lorenz Wellhausen,
Marco Hutter,
Ali-akbar Agha-mohammadi
Abstract:
Navigating off-road with a fast autonomous vehicle depends on a robust perception system that differentiates traversable from non-traversable terrain. Typically, this depends on a semantic understanding which is based on supervised learning from images annotated by a human expert. This requires a significant investment in human time, assumes correct expert classification, and small details can lea…
▽ More
Navigating off-road with a fast autonomous vehicle depends on a robust perception system that differentiates traversable from non-traversable terrain. Typically, this depends on a semantic understanding which is based on supervised learning from images annotated by a human expert. This requires a significant investment in human time, assumes correct expert classification, and small details can lead to misclassification. To address these challenges, we propose a method for predicting high- and low-risk terrains from only past vehicle experience in a self-supervised fashion. First, we develop a tool that projects the vehicle trajectory into the front camera image. Second, occlusions in the 3D representation of the terrain are filtered out. Third, an autoencoder trained on masked vehicle trajectory regions identifies low- and high-risk terrains based on the reconstruction error. We evaluated our approach with two models and different bottleneck sizes with two different training and testing sites with a fourwheeled off-road vehicle. Comparison with two independent test sets of semantic labels from similar terrain as training sites demonstrates the ability to separate the ground as low-risk and the vegetation as high-risk with 81.1% and 85.1% accuracy.
△ Less
Submitted 2 August, 2022;
originally announced August 2022.
-
Early Recall, Late Precision: Multi-Robot Semantic Object Map** under Operational Constraints in Perceptually-Degraded Environments
Authors:
Xianmei Lei,
Taeyeon Kim,
Nicolas Marchal,
Daniel Pastor,
Barry Ridge,
Frederik Schöller,
Edward Terry,
Fernando Chavez,
Thomas Touma,
Kyohei Otsu,
Ali Agha
Abstract:
Semantic object map** in uncertain, perceptually degraded environments during long-range multi-robot autonomous exploration tasks such as search-and-rescue is important and challenging. During such missions, high recall is desirable to avoid missing true target objects and high precision is also critical to avoid wasting valuable operational time on false positives. Given recent advancements in…
▽ More
Semantic object map** in uncertain, perceptually degraded environments during long-range multi-robot autonomous exploration tasks such as search-and-rescue is important and challenging. During such missions, high recall is desirable to avoid missing true target objects and high precision is also critical to avoid wasting valuable operational time on false positives. Given recent advancements in visual perception algorithms, the former is largely solvable autonomously, but the latter is difficult to address without the supervision of a human operator. However, operational constraints such as mission time, computational requirements, mesh network bandwidth and so on, can make the operator's task infeasible unless properly managed. We propose the Early Recall, Late Precision (EaRLaP) semantic object map** pipeline to solve this problem. EaRLaP was used by Team CoSTAR in DARPA Subterranean Challenge, where it successfully detected all the artifacts encountered by the team of robots. We will discuss these results and performance of the EaRLaP on various datasets.
△ Less
Submitted 20 June, 2022;
originally announced June 2022.
-
Machine Vision based Sample-Tube Localization for Mars Sample Return
Authors:
Shreyansh Daftry,
Barry Ridge,
William Seto,
Tu-Hoa Pham,
Peter Ilhardt,
Gerard Maggiolino,
Mark Van der Merwe,
Alex Brinkman,
John Mayo,
Eric Kulczyski,
Renaud Detry
Abstract:
A potential Mars Sample Return (MSR) architecture is being jointly studied by NASA and ESA. As currently envisioned, the MSR campaign consists of a series of 3 missions: sample cache, fetch and return to Earth. In this paper, we focus on the fetch part of the MSR, and more specifically the problem of autonomously detecting and localizing sample tubes deposited on the Martian surface. Towards this…
▽ More
A potential Mars Sample Return (MSR) architecture is being jointly studied by NASA and ESA. As currently envisioned, the MSR campaign consists of a series of 3 missions: sample cache, fetch and return to Earth. In this paper, we focus on the fetch part of the MSR, and more specifically the problem of autonomously detecting and localizing sample tubes deposited on the Martian surface. Towards this end, we study two machine-vision based approaches: First, a geometry-driven approach based on template matching that uses hard-coded filters and a 3D shape model of the tube; and second, a data-driven approach based on convolutional neural networks (CNNs) and learned features. Furthermore, we present a large benchmark dataset of sample-tube images, collected in representative outdoor environments and annotated with ground truth segmentation masks and locations. The dataset was acquired systematically across different terrain, illumination conditions and dust-coverage; and benchmarking was performed to study the feasibility of each approach, their relative strengths and weaknesses, and robustness in the presence of adverse environmental conditions.
△ Less
Submitted 17 March, 2021;
originally announced March 2021.
-
Rover Relocalization for Mars Sample Return by Virtual Template Synthesis and Matching
Authors:
Tu-Hoa Pham,
William Seto,
Shreyansh Daftry,
Barry Ridge,
Johanna Hansen,
Tristan Thrush,
Mark Van der Merwe,
Gerard Maggiolino,
Alexander Brinkman,
John Mayo,
Yang Cheng,
Curtis Padgett,
Eric Kulczycki,
Renaud Detry
Abstract:
We consider the problem of rover relocalization in the context of the notional Mars Sample Return campaign. In this campaign, a rover (R1) needs to be capable of autonomously navigating and localizing itself within an area of approximately 50 x 50 m using reference images collected years earlier by another rover (R0). We propose a visual localizer that exhibits robustness to the relatively barren…
▽ More
We consider the problem of rover relocalization in the context of the notional Mars Sample Return campaign. In this campaign, a rover (R1) needs to be capable of autonomously navigating and localizing itself within an area of approximately 50 x 50 m using reference images collected years earlier by another rover (R0). We propose a visual localizer that exhibits robustness to the relatively barren terrain that we expect to find in relevant areas, and to large lighting and viewpoint differences between R0 and R1. The localizer synthesizes partial renderings of a mesh built from reference R0 images and matches those to R1 images. We evaluate our method on a dataset totaling 2160 images covering the range of expected environmental conditions (terrain, lighting, approach angle). Experimental results show the effectiveness of our approach. This work informs the Mars Sample Return campaign on the choice of a site where Perseverance (R0) will place a set of sample tubes for future retrieval by another rover (R1).
△ Less
Submitted 4 March, 2021;
originally announced March 2021.