Showing 1–12 of 12 results for author: Milioto, A

OverlapNet: Loop Closing for LiDAR-based SLAM

Authors: Xieyuanli Chen, Thomas Läbe, Andres Milioto, Timo Röhling, Olga Vysotska, Alexandre Haag, Jens Behley, Cyrill Stachniss

Abstract: Simultaneous localization and map** (SLAM) is a fundamental capability required by most autonomous systems. In this paper, we address the problem of loop closing for SLAM based on 3D laser scans recorded by autonomous cars. Our approach utilizes a deep neural network exploiting different cues generated from LiDAR data for finding loop closures. It estimates an image overlap generalized to range… ▽ More Simultaneous localization and map** (SLAM) is a fundamental capability required by most autonomous systems. In this paper, we address the problem of loop closing for SLAM based on 3D laser scans recorded by autonomous cars. Our approach utilizes a deep neural network exploiting different cues generated from LiDAR data for finding loop closures. It estimates an image overlap generalized to range images and provides a relative yaw angle estimate between pairs of scans. Based on such predictions, we tackle loop closure detection and integrate our approach into an existing SLAM system to improve its map** results. We evaluate our approach on sequences of the KITTI odometry benchmark and the Ford campus dataset. We show that our method can effectively detect loop closures surpassing the detection performance of state-of-the-art methods. To highlight the generalization capabilities of our approach, we evaluate our model on the Ford campus dataset while using only KITTI for training. The experiments show that the learned representation is able to provide reliable loop closure candidates, also in unseen environments. △ Less

Submitted 24 May, 2021; originally announced May 2021.

Comments: Accepted by RSS 2020. Code: https://github.com/PRBonn/OverlapNet

SuMa++: Efficient LiDAR-based Semantic SLAM

Authors: Xieyuanli Chen, Andres Milioto, Emanuele Palazzolo, Philippe Giguère, Jens Behley, Cyrill Stachniss

Abstract: Reliable and accurate localization and map** are key components of most autonomous systems. Besides geometric information about the mapped environment, the semantics plays an important role to enable intelligent navigation behaviors. In most realistic environments, this task is particularly complicated due to dynamics caused by moving objects, which can corrupt the map** step or derail localiz… ▽ More Reliable and accurate localization and map** are key components of most autonomous systems. Besides geometric information about the mapped environment, the semantics plays an important role to enable intelligent navigation behaviors. In most realistic environments, this task is particularly complicated due to dynamics caused by moving objects, which can corrupt the map** step or derail localization. In this paper, we propose an extension of a recently published surfel-based map** approach exploiting three-dimensional laser range scans by integrating semantic information to facilitate the map** process. The semantic information is efficiently extracted by a fully convolutional neural network and rendered on a spherical projection of the laser range data. This computed semantic segmentation results in point-wise labels for the whole scan, allowing us to build a semantically-enriched map with labeled surfels. This semantic map enables us to reliably filter moving objects, but also improve the projective scan matching via semantic constraints. Our experimental evaluation on challenging highways sequences from KITTI dataset with very few static structures and a large amount of moving cars shows the advantage of our semantic SLAM approach in comparison to a purely geometric, state-of-the-art approach. △ Less

Submitted 24 May, 2021; originally announced May 2021.

Comments: Accepted by IROS 2019. Code: https://github.com/PRBonn/semantic_suma

A Benchmark for LiDAR-based Panoptic Segmentation based on KITTI

Authors: Jens Behley, Andres Milioto, Cyrill Stachniss

Abstract: Panoptic segmentation is the recently introduced task that tackles semantic segmentation and instance segmentation jointly. In this paper, we present an extension of SemanticKITTI, which is a large-scale dataset providing dense point-wise semantic labels for all sequences of the KITTI Odometry Benchmark, for training and evaluation of laser-based panoptic segmentation. We provide the data and disc… ▽ More Panoptic segmentation is the recently introduced task that tackles semantic segmentation and instance segmentation jointly. In this paper, we present an extension of SemanticKITTI, which is a large-scale dataset providing dense point-wise semantic labels for all sequences of the KITTI Odometry Benchmark, for training and evaluation of laser-based panoptic segmentation. We provide the data and discuss the processing steps needed to enrich a given semantic annotation with temporally consistent instance information, i.e., instance information that supplements the semantic labels and identifies the same instance over sequences of LiDAR point clouds. Additionally, we present two strong baselines that combine state-of-the-art LiDAR-based semantic segmentation approaches with a state-of-the-art detector enriching the segmentation with instance information and that allow other researchers to compare their approaches against. We hope that our extension of SemanticKITTI with strong baselines enables the creation of novel algorithms for LiDAR-based panoptic segmentation as much as it has for the original semantic segmentation and semantic scene completion tasks. Data, code, and an online evaluation using a hidden test set will be published on http://semantic-kitti.org. △ Less

Submitted 4 March, 2020; originally announced March 2020.

Building an Aerial-Ground Robotics System for Precision Farming: An Adaptable Solution

Authors: Alberto Pretto, Stéphanie Aravecchia, Wolfram Burgard, Nived Chebrolu, Christian Dornhege, Tillmann Falck, Freya Fleckenstein, Alessandra Fontenla, Marco Imperoli, Raghav Khanna, Frank Liebisch, Philipp Lottes, Andres Milioto, Daniele Nardi, Sandro Nardi, Johannes Pfeifer, Marija Popović, Ciro Potena, Cédric Pradalier, Elisa Rothacker-Feder, Inkyu Sa, Alexander Schaefer, Roland Siegwart, Cyrill Stachniss, Achim Walter , et al. (3 additional authors not shown)

Abstract: The application of autonomous robots in agriculture is gaining increasing popularity thanks to the high impact it may have on food security, sustainability, resource use efficiency, reduction of chemical treatments, and the optimization of human effort and yield. With this vision, the Flourish research project aimed to develop an adaptable robotic solution for precision farming that combines the a… ▽ More The application of autonomous robots in agriculture is gaining increasing popularity thanks to the high impact it may have on food security, sustainability, resource use efficiency, reduction of chemical treatments, and the optimization of human effort and yield. With this vision, the Flourish research project aimed to develop an adaptable robotic solution for precision farming that combines the aerial survey capabilities of small autonomous unmanned aerial vehicles (UAVs) with targeted intervention performed by multi-purpose unmanned ground vehicles (UGVs). This paper presents an overview of the scientific and technological advances and outcomes obtained in the project. We introduce multi-spectral perception algorithms and aerial and ground-based systems developed for monitoring crop density, weed pressure, crop nitrogen nutrition status, and to accurately classify and locate weeds. We then introduce the navigation and map** systems tailored to our robots in the agricultural environment, as well as the modules for collaborative map**. We finally present the ground intervention hardware, software solutions, and interfaces we implemented and tested in different field conditions and with different crops. We describe a real use case in which a UAV collaborates with a UGV to monitor the field and to perform selective spraying without human intervention. △ Less

Submitted 7 June, 2022; v1 submitted 8 November, 2019; originally announced November 2019.

Comments: Published in IEEE Robotics & Automation Magazine, vol. 28, no. 3, pp. 29-49, Sept. 2021

Journal ref: IEEE Robotics & Automation Magazine, vol. 28, no. 3, pp. 29-49, Sept. 2021

Detection of Single Grapevine Berries in Images Using Fully Convolutional Neural Networks

Authors: Laura Zabawa, Anna Kicherer, Lasse Klingbeil, Andres Milioto, Reinhard Töpfer, Heiner Kuhlmann, Ribana Roscher

Abstract: Yield estimation and forecasting are of special interest in the field of grapevine breeding and viticulture. The number of harvested berries per plant is strongly correlated with the resulting quality. Therefore, early yield forecasting can enable a focused thinning of berries to ensure a high quality end product. Traditionally yield estimation is done by extrapolating from a small sample size and… ▽ More Yield estimation and forecasting are of special interest in the field of grapevine breeding and viticulture. The number of harvested berries per plant is strongly correlated with the resulting quality. Therefore, early yield forecasting can enable a focused thinning of berries to ensure a high quality end product. Traditionally yield estimation is done by extrapolating from a small sample size and by utilizing historic data. Moreover, it needs to be carried out by skilled experts with much experience in this field. Berry detection in images offers a cheap, fast and non-invasive alternative to the otherwise time-consuming and subjective on-site analysis by experts. We apply fully convolutional neural networks on images acquired with the Phenoliner, a field phenoty** platform. We count single berries in images to avoid the error-prone detection of grapevine clusters. Clusters are often overlap** and can vary a lot in the size which makes the reliable detection of them difficult. We address especially the detection of white grapes directly in the vineyard. The detection of single berries is formulated as a classification task with three classes, namely 'berry', 'edge' and 'background'. A connected component algorithm is applied to determine the number of berries in one image. We compare the automatically counted number of berries with the manually detected berries in 60 images showing Riesling plants in vertical shoot positioned trellis (VSP) and semi minimal pruned hedges (SMPH). We are able to detect berries correctly within the VSP system with an accuracy of 94.0 \% and for the SMPH system with 85.6 \%. △ Less

Submitted 1 May, 2019; originally announced May 2019.

Journal ref: CVPR Workshop on Computer Vision Problems in Plant Phenoty**, 2019

SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences

Authors: Jens Behley, Martin Garbade, Andres Milioto, Jan Quenzel, Sven Behnke, Cyrill Stachniss, Juergen Gall

Abstract: Semantic scene understanding is important for various applications. In particular, self-driving cars need a fine-grained understanding of the surfaces and objects in their vicinity. Light detection and ranging (LiDAR) provides precise geometric information about the environment and is thus a part of the sensor suites of almost all self-driving cars. Despite the relevance of semantic scene understa… ▽ More Semantic scene understanding is important for various applications. In particular, self-driving cars need a fine-grained understanding of the surfaces and objects in their vicinity. Light detection and ranging (LiDAR) provides precise geometric information about the environment and is thus a part of the sensor suites of almost all self-driving cars. Despite the relevance of semantic scene understanding for this application, there is a lack of a large dataset for this task which is based on an automotive LiDAR. In this paper, we introduce a large dataset to propel research on laser-based semantic segmentation. We annotated all sequences of the KITTI Vision Odometry Benchmark and provide dense point-wise annotations for the complete $360^{o}$ field-of-view of the employed automotive LiDAR. We propose three benchmark tasks based on this dataset: (i) semantic segmentation of point clouds using a single scan, (ii) semantic segmentation using multiple past scans, and (iii) semantic scene completion, which requires to anticipate the semantic scene in the future. We provide baseline experiments and show that there is a need for more sophisticated models to efficiently tackle these tasks. Our dataset opens the door for the development of more advanced methods, but also provides plentiful data to investigate new research directions. △ Less

Submitted 16 August, 2019; v1 submitted 2 April, 2019; originally announced April 2019.

Comments: ICCV2019. See teaser video at http://bit.ly/SemanticKITTI-teaser

Geometrical Stem Detection from Image Data for Precision Agriculture

Authors: F. Langer, L. Mandtler, A. Milioto, E. Palazzolo, C. Stachniss

Abstract: High efficiency in precision farming depends on accurate tools to perform weed detection and map** of crops. This allows for precise removal of harmful weeds with a lower amount of pesticides, as well as increase of the harvest's yield by providing the farmer with valuable information. In this paper, we address the problem of fully automatic stem detection from image data for this purpose. Our a… ▽ More High efficiency in precision farming depends on accurate tools to perform weed detection and map** of crops. This allows for precise removal of harmful weeds with a lower amount of pesticides, as well as increase of the harvest's yield by providing the farmer with valuable information. In this paper, we address the problem of fully automatic stem detection from image data for this purpose. Our approach runs on mobile agricultural robots taking RGB images. After processing the images to obtain a vegetation mask, our approach separates each plant into its individual leaves and later estimates a precise stem position. This allows an upstream map** algorithm to add the high-resolution stem positions as a semantic aggregate to the global map of the robot, which can be used for weeding and for analyzing crop statistics. We implemented our approach and thoroughly tested it on three different datasets with vegetation masks and stem position ground truth. The experiments presented in this paper conclude that our module is able to detect leaves and estimate the stem's position at a rate of 56 Hz on a single CPU. We furthermore provide the software to the community. △ Less

Submitted 13 December, 2018; originally announced December 2018.

Comments: Note that this work has been published without author's consent by WSEAS TRANSACTIONS on SYSTEMS, so please cite this arxiv paper if you want to reference to our work

Joint Stem Detection and Crop-Weed Classification for Plant-specific Treatment in Precision Farming

Authors: Philipp Lottes, Jens Behley, Nived Chebrolu, Andres Milioto, Cyrill Stachniss

Abstract: Applying agrochemicals is the default procedure for conventional weed control in crop production, but has negative impacts on the environment. Robots have the potential to treat every plant in the field individually and thus can reduce the required use of such chemicals. To achieve that, robots need the ability to identify crops and weeds in the field and must additionally select effective treatme… ▽ More Applying agrochemicals is the default procedure for conventional weed control in crop production, but has negative impacts on the environment. Robots have the potential to treat every plant in the field individually and thus can reduce the required use of such chemicals. To achieve that, robots need the ability to identify crops and weeds in the field and must additionally select effective treatments. While certain types of weed can be treated mechanically, other types need to be treated by (selective) spraying. In this paper, we present an approach that provides the necessary information for effective plant-specific treatment. It outputs the stem location for weeds, which allows for mechanical treatments, and the covered area of the weed for selective spraying. Our approach uses an end-to-end trainable fully convolutional network that simultaneously estimates stem positions as well as the covered area of crops and weeds. It jointly learns the class-wise stem detection and the pixel-wise semantic segmentation. Experimental evaluations on different real-world datasets show that our approach is able to reliably solve this problem. Compared to state-of-the-art approaches, our approach not only substantially improves the stem detection accuracy, i.e., distinguishing crop and weed stems, but also provides an improvement in the semantic segmentation performance. △ Less

Submitted 9 June, 2018; originally announced June 2018.

Comments: Submitted to the International Conference on Intelligent Robots and Systems (IROS), 2018

Fully Convolutional Networks with Sequential Information for Robust Crop and Weed Detection in Precision Farming

Authors: Philipp Lottes, Jens Behley, Andres Milioto, Cyrill Stachniss

Abstract: Reducing the use of agrochemicals is an important component towards sustainable agriculture. Robots that can perform targeted weed control offer the potential to contribute to this goal, for example, through specialized weeding actions such as selective spraying or mechanical weed removal. A prerequisite of such systems is a reliable and robust plant classification system that is able to distingui… ▽ More Reducing the use of agrochemicals is an important component towards sustainable agriculture. Robots that can perform targeted weed control offer the potential to contribute to this goal, for example, through specialized weeding actions such as selective spraying or mechanical weed removal. A prerequisite of such systems is a reliable and robust plant classification system that is able to distinguish crop and weed in the field. A major challenge in this context is the fact that different fields show a large variability. Thus, classification systems have to robustly cope with substantial environmental changes with respect to weed pressure and weed types, growth stages of the crop, visual appearance, and soil conditions. In this paper, we propose a novel crop-weed classification system that relies on a fully convolutional network with an encoder-decoder structure and incorporates spatial information by considering image sequences. Exploiting the crop arrangement information that is observable from the image sequences enables our system to robustly estimate a pixel-wise labeling of the images into crop and weed, i.e., a semantic segmentation. We provide a thorough experimental evaluation, which shows that our system generalizes well to previously unseen fields under varying environmental conditions --- a key capability to actually use such systems in precision framing. We provide comparisons to other state-of-the-art approaches and show that our system substantially improves the accuracy of crop-weed classification without requiring a retraining of the model. △ Less

Submitted 9 June, 2018; originally announced June 2018.

Comments: Accepted for publication in IEEE Robotics and Automation Letters (RA-L), 2018

Ocean Eddy Identification and Tracking using Neural Networks

Authors: Katharina Franz, Ribana Roscher, Andres Milioto, Susanne Wenzel, Jürgen Kusche

Abstract: Global climate change plays an essential role in our daily life. Mesoscale ocean eddies have a significant impact on global warming, since they affect the ocean dynamics, the energy as well as the mass transports of ocean circulation. From satellite altimetry we can derive high-resolution, global maps containing ocean signals with dominating coherent eddy structures. The aim of this study is the d… ▽ More Global climate change plays an essential role in our daily life. Mesoscale ocean eddies have a significant impact on global warming, since they affect the ocean dynamics, the energy as well as the mass transports of ocean circulation. From satellite altimetry we can derive high-resolution, global maps containing ocean signals with dominating coherent eddy structures. The aim of this study is the development and evaluation of a deep-learning based approach for the analysis of eddies. In detail, we develop an eddy identification and tracking framework with two different approaches that are mainly based on feature learning with convolutional neural networks. Furthermore, state-of-the-art image processing tools and object tracking methods are used to support the eddy tracking. In contrast to previous methods, our framework is able to learn a representation of the data in which eddies can be detected and tracked in more objective and robust way. We show the detection and tracking results on sea level anomalies (SLA) data from the area of Australia and the East Australia current, and compare our two eddy detection and tracking approaches to identify the most robust and objective method. △ Less

Submitted 15 May, 2018; v1 submitted 20 March, 2018; originally announced March 2018.

Comments: accepted for International Geoscience and Remote Sensing Symposium 2018

Bonnet: An Open-Source Training and Deployment Framework for Semantic Segmentation in Robotics using CNNs

Authors: Andres Milioto, Cyrill Stachniss

Abstract: The ability to interpret a scene is an important capability for a robot that is supposed to interact with its environment. The knowledge of what is in front of the robot is, for example, relevant for navigation, manipulation, or planning. Semantic segmentation labels each pixel of an image with a class label and thus provides a detailed semantic annotation of the surroundings to the robot. Convolu… ▽ More The ability to interpret a scene is an important capability for a robot that is supposed to interact with its environment. The knowledge of what is in front of the robot is, for example, relevant for navigation, manipulation, or planning. Semantic segmentation labels each pixel of an image with a class label and thus provides a detailed semantic annotation of the surroundings to the robot. Convolutional neural networks (CNNs) are popular methods for addressing this type of problem. The available software for training and the integration of CNNs for real robots, however, is quite fragmented and often difficult to use for non-experts, despite the availability of several high-quality open-source frameworks for neural network implementation and training. In this paper, we propose a tool called Bonnet, which addresses this fragmentation problem by building a higher abstraction that is specific for the semantic segmentation task. It provides a modular approach to simplify the training of a semantic segmentation CNN independently of the used dataset and the intended task. Furthermore, we also address the deployment on a real robotic platform. Thus, we do not propose a new CNN approach in this paper. Instead, we provide a stable and easy-to-use tool to make this technology more approachable in the context of autonomous systems. In this sense, we aim at closing a gap between computer vision research and its use in robotics research. We provide an open-source codebase for training and deployment. The training interface is implemented in Python using TensorFlow and the deployment interface provides a C++ library that can be easily integrated in an existing robotics codebase, a ROS node, and two standalone applications for label prediction in images and videos. △ Less

Submitted 1 February, 2019; v1 submitted 25 February, 2018; originally announced February 2018.

Comments: To be published in to IEEE International Conference on Robotics and Automation 2019

Real-time Semantic Segmentation of Crop and Weed for Precision Agriculture Robots Leveraging Background Knowledge in CNNs

Authors: Andres Milioto, Philipp Lottes, Cyrill Stachniss

Abstract: Precision farming robots, which target to reduce the amount of herbicides that need to be brought out in the fields, must have the ability to identify crops and weeds in real time to trigger weeding actions. In this paper, we address the problem of CNN-based semantic segmentation of crop fields separating sugar beet plants, weeds, and background solely based on RGB data. We propose a CNN that expl… ▽ More Precision farming robots, which target to reduce the amount of herbicides that need to be brought out in the fields, must have the ability to identify crops and weeds in real time to trigger weeding actions. In this paper, we address the problem of CNN-based semantic segmentation of crop fields separating sugar beet plants, weeds, and background solely based on RGB data. We propose a CNN that exploits existing vegetation indexes and provides a classification in real time. Furthermore, it can be effectively re-trained to so far unseen fields with a comparably small amount of training data. We implemented and thoroughly evaluated our system on a real agricultural robot operating in different fields in Germany and Switzerland. The results show that our system generalizes well, can operate at around 20Hz, and is suitable for online operation in the fields. △ Less

Submitted 2 March, 2018; v1 submitted 20 September, 2017; originally announced September 2017.

Comments: Accepted for publication at IEEE International Conference on Robotics and Automation 2018 (ICRA 2018)