-
Scalable Autonomous Drone Flight in the Forest with Visual-Inertial SLAM and Dense Submaps Built without LiDAR
Authors:
Sebastián Barbas Laina,
Simon Boche,
Sotiris Papatheodorou,
Dimos Tzoumanikas,
Simon Schaefer,
Hanzhi Chen,
Stefan Leutenegger
Abstract:
Forestry constitutes a key element for a sustainable future, while it is supremely challenging to introduce digital processes to improve efficiency. The main limitation is the difficulty of obtaining accurate maps at high temporal and spatial resolution as a basis for informed forestry decision-making, due to the vast area forests extend over and the sheer number of trees. To address this challeng…
▽ More
Forestry constitutes a key element for a sustainable future, while it is supremely challenging to introduce digital processes to improve efficiency. The main limitation is the difficulty of obtaining accurate maps at high temporal and spatial resolution as a basis for informed forestry decision-making, due to the vast area forests extend over and the sheer number of trees. To address this challenge, we present an autonomous Micro Aerial Vehicle (MAV) system which purely relies on cost-effective and light-weight passive visual and inertial sensors to perform under-canopy autonomous navigation. We leverage visual-inertial simultaneous localization and map** (VI-SLAM) for accurate MAV state estimates and couple it with a volumetric occupancy submap** system to achieve a scalable map** framework which can be directly used for path planning. As opposed to a monolithic map, submaps inherently deal with inevitable drift and corrections from VI-SLAM, since they move with pose estimates as they are updated. To ensure the safety of the MAV during navigation, we also propose a novel reference trajectory anchoring scheme that moves and deforms the reference trajectory the MAV is tracking upon state updates from the VI-SLAM system in a consistent way, even upon large changes in state estimates due to loop-closures. We thoroughly validate our system in both real and simulated forest environments with high tree densities in excess of 400 trees per hectare and at speeds up to 3 m/s - while not encountering a single collision or system failure. To the best of our knowledge this is the first system which achieves this level of performance in such unstructured environment using low-cost passive visual sensors and fully on-board computation including VI-SLAM.
△ Less
Submitted 14 March, 2024;
originally announced March 2024.
-
Control-Barrier-Aided Teleoperation with Visual-Inertial SLAM for Safe MAV Navigation in Complex Environments
Authors:
Siqi Zhou,
Sotiris Papatheodorou,
Stefan Leutenegger,
Angela P. Schoellig
Abstract:
In this paper, we consider a Micro Aerial Vehicle (MAV) system teleoperated by a non-expert and introduce a perceptive safety filter that leverages Control Barrier Functions (CBFs) in conjunction with Visual-Inertial Simultaneous Localization and Map** (VI-SLAM) and dense 3D occupancy map** to guarantee safe navigation in complex and unstructured environments. Our system relies solely on onboa…
▽ More
In this paper, we consider a Micro Aerial Vehicle (MAV) system teleoperated by a non-expert and introduce a perceptive safety filter that leverages Control Barrier Functions (CBFs) in conjunction with Visual-Inertial Simultaneous Localization and Map** (VI-SLAM) and dense 3D occupancy map** to guarantee safe navigation in complex and unstructured environments. Our system relies solely on onboard IMU measurements, stereo infrared images, and depth images and autonomously corrects teleoperated inputs when they are deemed unsafe. We define a point in 3D space as unsafe if it satisfies either of two conditions: (i) it is occupied by an obstacle, or (ii) it remains unmapped. At each time step, an occupancy map of the environment is updated by the VI-SLAM by fusing the onboard measurements, and a CBF is constructed to parameterize the (un)safe region in the 3D space. Given the CBF and state feedback from the VI-SLAM module, a safety filter computes a certified reference that best matches the teleoperation input while satisfying the safety constraint encoded by the CBF. In contrast to existing perception-based safe control frameworks, we directly close the perception-action loop and demonstrate the full capability of safe control in combination with real-time VI-SLAM without any external infrastructure or prior knowledge of the environment. We verify the efficacy of the perceptive safety filter in real-time MAV experiments using exclusively onboard sensing and computation and show that the teleoperated MAV is able to safely navigate through unknown environments despite arbitrary inputs sent by the teleoperator.
△ Less
Submitted 7 March, 2024;
originally announced March 2024.
-
Finding Things in the Unknown: Semantic Object-Centric Exploration with an MAV
Authors:
Sotiris Papatheodorou,
Nils Funk,
Dimos Tzoumanikas,
Christopher Choi,
Binbin Xu,
Stefan Leutenegger
Abstract:
Exploration of unknown space with an autonomous mobile robot is a well-studied problem. In this work we broaden the scope of exploration, moving beyond the pure geometric goal of uncovering as much free space as possible. We believe that for many practical applications, exploration should be contextualised with semantic and object-level understanding of the environment for task-specific exploratio…
▽ More
Exploration of unknown space with an autonomous mobile robot is a well-studied problem. In this work we broaden the scope of exploration, moving beyond the pure geometric goal of uncovering as much free space as possible. We believe that for many practical applications, exploration should be contextualised with semantic and object-level understanding of the environment for task-specific exploration. Here, we study the task of both finding specific objects in unknown space as well as reconstructing them to a target level of detail. We therefore extend our environment reconstruction to not only consist of a background map, but also object-level and semantically fused submaps. Importantly, we adapt our previous objective function of uncovering as much free space as possible in as little time as possible with two additional elements: first, we require a maximum observation distance of background surfaces to ensure target objects are not missed by image-based detectors because they are too small to be detected. Second, we require an even smaller maximum distance to the found objects in order to reconstruct them with the desired accuracy. We further created a Micro Aerial Vehicle (MAV) semantic exploration simulator based on Habitat in order to quantitatively demonstrate how our framework can be used to efficiently find specific objects as part of exploration. Finally, we showcase this capability can be deployed in real-world scenes involving our drone equipped with an Intel RealSense D455 RGB-D camera.
△ Less
Submitted 3 March, 2023; v1 submitted 28 February, 2023;
originally announced February 2023.
-
Volumetric Occupancy Map** With Probabilistic Depth Completion for Robotic Navigation
Authors:
Marija Popovic,
Florian Thomas,
Sotiris Papatheodorou,
Nils Funk,
Teresa Vidal-Calleja,
Stefan Leutenegger
Abstract:
In robotic applications, a key requirement for safe and efficient motion planning is the ability to map obstacle-free space in unknown, cluttered 3D environments. However, commodity-grade RGB-D cameras commonly used for sensing fail to register valid depth values on shiny, glossy, bright, or distant surfaces, leading to missing data in the map. To address this issue, we propose a framework leverag…
▽ More
In robotic applications, a key requirement for safe and efficient motion planning is the ability to map obstacle-free space in unknown, cluttered 3D environments. However, commodity-grade RGB-D cameras commonly used for sensing fail to register valid depth values on shiny, glossy, bright, or distant surfaces, leading to missing data in the map. To address this issue, we propose a framework leveraging probabilistic depth completion as an additional input for spatial map**. We introduce a deep learning architecture providing uncertainty estimates for the depth completion of RGB-D images. Our pipeline exploits the inferred missing depth values and depth uncertainty to complement raw depth images and improve the speed and quality of free space map**. Evaluations on synthetic data show that our approach maps significantly more correct free space with relatively low error when compared against using raw data alone in different indoor environments; thereby producing more complete maps that can be directly used for robotic navigation tasks. The performance of our framework is validated using real-world data.
△ Less
Submitted 22 March, 2021; v1 submitted 5 December, 2020;
originally announced December 2020.
-
Elastic and Efficient LiDAR Reconstruction for Large-Scale Exploration Tasks
Authors:
Yiduo Wang,
Nils Funk,
Milad Ramezani,
Sotiris Papatheodorou,
Marija Popovic,
Marco Camurri,
Stefan Leutenegger,
Maurice Fallon
Abstract:
We present an efficient, elastic 3D LiDAR reconstruction framework which can reconstruct up to maximum LiDAR ranges (60 m) at multiple frames per second, thus enabling robot exploration in large-scale environments. Our approach only requires a CPU. We focus on three main challenges of large-scale reconstruction: integration of long-range LiDAR scans at high frequency, the capacity to deform the re…
▽ More
We present an efficient, elastic 3D LiDAR reconstruction framework which can reconstruct up to maximum LiDAR ranges (60 m) at multiple frames per second, thus enabling robot exploration in large-scale environments. Our approach only requires a CPU. We focus on three main challenges of large-scale reconstruction: integration of long-range LiDAR scans at high frequency, the capacity to deform the reconstruction after loop closures are detected, and scalability for long-duration exploration. Our system extends upon a state-of-the-art efficient RGB-D volumetric reconstruction technique, called supereight, to support LiDAR scans and a newly developed submap** technique to allow for dynamic correction of the 3D reconstruction. We then introduce a novel pose graph clustering and submap fusion feature to make the proposed system more scalable for large environments. We evaluate the performance using two public datasets including outdoor exploration with a handheld device and a drone, and with a mobile robot exploring an underground room network. Experimental results demonstrate that our system can reconstruct at 3 Hz with 60 m sensor range and ~5 cm resolution, while state-of-the-art approaches can only reconstruct to 25 cm resolution or 20 m range at the same frequency.
△ Less
Submitted 9 April, 2021; v1 submitted 19 October, 2020;
originally announced October 2020.
-
Multi-Resolution 3D Map** with Explicit Free Space Representation for Fast and Accurate Mobile Robot Motion Planning
Authors:
Nils Funk,
Juan Tarrio,
Sotiris Papatheodorou,
Marija Popovic,
Pablo F. Alcantarilla,
Stefan Leutenegger
Abstract:
With the aim of bridging the gap between high quality reconstruction and mobile robot motion planning, we propose an efficient system that leverages the concept of adaptive-resolution volumetric map**, which naturally integrates with the hierarchical decomposition of space in an octree data structure. Instead of a Truncated Signed Distance Function (TSDF), we adopt map** of occupancy probabili…
▽ More
With the aim of bridging the gap between high quality reconstruction and mobile robot motion planning, we propose an efficient system that leverages the concept of adaptive-resolution volumetric map**, which naturally integrates with the hierarchical decomposition of space in an octree data structure. Instead of a Truncated Signed Distance Function (TSDF), we adopt map** of occupancy probabilities in log-odds representation, which allows to represent both surfaces, as well as the entire free, i.e. observed space, as opposed to unobserved space. We introduce a method for choosing resolution -- on the fly -- in real-time by means of a multi-scale max-min pooling of the input depth image. The notion of explicit free space map** paired with the spatial hierarchy in the data structure, as well as map resolution, allows for collision queries, as needed for robot motion planning, at unprecedented speed. We quantitatively evaluate map** accuracy, memory, runtime performance, and planning performance showing improvements over the state of the art, particularly in cases requiring high resolution maps.
△ Less
Submitted 30 January, 2021; v1 submitted 15 October, 2020;
originally announced October 2020.
-
Fast Frontier-based Information-driven Autonomous Exploration with an MAV
Authors:
Anna Dai,
Sotiris Papatheodorou,
Nils Funk,
Dimos Tzoumanikas,
Stefan Leutenegger
Abstract:
Exploration and collision-free navigation through an unknown environment is a fundamental task for autonomous robots. In this paper, a novel exploration strategy for Micro Aerial Vehicles (MAVs) is presented. The goal of the exploration strategy is the reduction of map entropy regarding occupancy probabilities, which is reflected in a utility function to be maximised. We achieve fast and efficient…
▽ More
Exploration and collision-free navigation through an unknown environment is a fundamental task for autonomous robots. In this paper, a novel exploration strategy for Micro Aerial Vehicles (MAVs) is presented. The goal of the exploration strategy is the reduction of map entropy regarding occupancy probabilities, which is reflected in a utility function to be maximised. We achieve fast and efficient exploration performance with tight integration between our octree-based occupancy map** approach, frontier extraction, and motion planning-as a hybrid between frontier-based and sampling-based exploration methods. The computationally expensive frontier clustering employed in classic frontier-based exploration is avoided by exploiting the implicit grou** of frontier voxels in the underlying octree map representation. Candidate next-views are sampled from the map frontiers and are evaluated using a utility function combining map entropy and travel time, where the former is computed efficiently using sparse raycasting. These optimisations along with the targeted exploration of frontier-based methods result in a fast and computationally efficient exploration planner. The proposed method is evaluated using both simulated and real-world experiments, demonstrating clear advantages over state-of-the-art approaches.
△ Less
Submitted 13 February, 2020; v1 submitted 11 February, 2020;
originally announced February 2020.
-
Distributed surveillance by a swarm of UAVs operating under positional uncertainty
Authors:
Nikolaos Bousias,
Sotiris Papatheodorou,
Mariliza Tzes,
Anthony Tzes
Abstract:
This article proposes a collaborative control framework for an autonomous aerial swarm tasked with the surveillance of a convex region of interest. Each Mobile Aerial Agent (MAA) is equipped with a Pan-Tilt-Zoom (PTZ) camera of conical FOV and suffers from sensor-induced positional uncertainty. By utilizing a Voronoi-free tessellation strategy and a gradient scheme, the heterogeneous swarm self-or…
▽ More
This article proposes a collaborative control framework for an autonomous aerial swarm tasked with the surveillance of a convex region of interest. Each Mobile Aerial Agent (MAA) is equipped with a Pan-Tilt-Zoom (PTZ) camera of conical FOV and suffers from sensor-induced positional uncertainty. By utilizing a Voronoi-free tessellation strategy and a gradient scheme, the heterogeneous swarm self-organizes in a distributed manner to monotonically achieve optimal collective visual coverage of the region of interest, both in terms of quality and total area. Simulation studies are offered to investigate the effectiveness of the suggested scheme.
△ Less
Submitted 23 October, 2019;
originally announced October 2019.
-
Distributed area coverage control with imprecise robot localization: Simulation and experimental studies
Authors:
Sotiris Papatheodorou,
Anthony Tzes,
Konstantinos Giannousakis,
Yiannis Stergiopoulos
Abstract:
This article examines the area coverage problem for a network of mobile robots with imprecise agents' localization. Each robot has uniform radial sensing ability, governed by first order kinodynamics. The convex-space is partitioned based on the Guaranteed Voronoi (GV) principle and each robot's area of responsibility corresponds to its GV-cell, bounded by hyperbolic arcs. The proposed control law…
▽ More
This article examines the area coverage problem for a network of mobile robots with imprecise agents' localization. Each robot has uniform radial sensing ability, governed by first order kinodynamics. The convex-space is partitioned based on the Guaranteed Voronoi (GV) principle and each robot's area of responsibility corresponds to its GV-cell, bounded by hyperbolic arcs. The proposed control law is distributed, demanding the positioning information about its GV-Delaunay neighbors. Simulation and experimental studies are offered to highlight the efficiency of the proposed control law.
△ Less
Submitted 14 December, 2016;
originally announced December 2016.
-
Collaborative Visual Area Coverage
Authors:
Sotiris Papatheodorou,
Anthony Tzes,
Yiannis Stergiopoulos
Abstract:
This article examines the problem of visual area coverage by a network of Mobile Aerial Agents (MAAs). Each MAA is assumed to be equipped with a downwards facing camera with a conical field of view which covers all points within a circle on the ground. The diameter of that circle is proportional to the altitude of the MAA, whereas the quality of the covered area decreases with the altitude. A dist…
▽ More
This article examines the problem of visual area coverage by a network of Mobile Aerial Agents (MAAs). Each MAA is assumed to be equipped with a downwards facing camera with a conical field of view which covers all points within a circle on the ground. The diameter of that circle is proportional to the altitude of the MAA, whereas the quality of the covered area decreases with the altitude. A distributed control law that maximizes a joint coverage-quality criterion by adjusting the MAAs' spatial coordinates is developed. The effectiveness of the proposed control scheme is evaluated through simulation studies.
△ Less
Submitted 6 December, 2016;
originally announced December 2016.
-
Collaborative Visual Area Coverage using Unmanned Aerial Vehicles
Authors:
Sotiris Papatheodorou,
Anthony Tzes,
Yiannis Stergiopoulos
Abstract:
This article addresses the visual area coverage problem using a team of Unmanned Aerial Vehicles (UAVs). The UAVs are assumed to be equipped with a downward facing camera covering all points of interest within a circle on the ground. The diameter of this circular conic-section increases as the UAV flies at a larger height, yet the quality of the observed area is inverse proportional to the UAV's h…
▽ More
This article addresses the visual area coverage problem using a team of Unmanned Aerial Vehicles (UAVs). The UAVs are assumed to be equipped with a downward facing camera covering all points of interest within a circle on the ground. The diameter of this circular conic-section increases as the UAV flies at a larger height, yet the quality of the observed area is inverse proportional to the UAV's height. The objective is to provide a distributed control algorithm that maximizes a combined coverage-quality criterion by adjusting the UAV's altitude. Simulation studies are offered to highlight the effectiveness of the suggested scheme.
△ Less
Submitted 6 December, 2016;
originally announced December 2016.
-
Watermarking Digital Images Based on a Content Based Image Retrieval Technique
Authors:
Dimitrios K. Tsolis,
Spyros Sioutas,
Theodore S. Papatheodorou
Abstract:
The current work is focusing on the implementation of a robust watermarking algorithm for digital images, which is based on an innovative spread spectrum analysis algorithm for watermark embedding and on a content-based image retrieval technique for watermark detection. The highly robust watermark algorithms are applying "detectable watermarks" for which a detection mechanism checks if the water…
▽ More
The current work is focusing on the implementation of a robust watermarking algorithm for digital images, which is based on an innovative spread spectrum analysis algorithm for watermark embedding and on a content-based image retrieval technique for watermark detection. The highly robust watermark algorithms are applying "detectable watermarks" for which a detection mechanism checks if the watermark exists or no (a Boolean decision) based on a watermarking key. The problem is that the detection of a watermark in a digital image library containing thousands of images means that the watermark detection algorithm is necessary to apply all the keys to the digital images. This application is non-efficient for very large image databases. On the other hand "readable" watermarks may prove weaker but easier to detect as only the detection mechanism is required. The proposed watermarking algorithm combine's the advantages of both "detectable" and "readable" watermarks. The result is a fast and robust watermarking algorithm.
△ Less
Submitted 17 October, 2008;
originally announced October 2008.
