-
Challenges and Opportunities for Large-Scale Exploration with Air-Ground Teams using Semantics
Authors:
Fernando Cladera,
Ian D. Miller,
Zachary Ravichandran,
Varun Murali,
Jason Hughes,
M. Ani Hsieh,
C. J. Taylor,
Vijay Kumar
Abstract:
One common and desirable application of robots is exploring potentially hazardous and unstructured environments. Air-ground collaboration offers a synergistic approach to addressing such exploration challenges. In this paper, we demonstrate a system for large-scale exploration using a team of aerial and ground robots. Our system uses semantics as lingua franca, and relies on fully opportunistic co…
▽ More
One common and desirable application of robots is exploring potentially hazardous and unstructured environments. Air-ground collaboration offers a synergistic approach to addressing such exploration challenges. In this paper, we demonstrate a system for large-scale exploration using a team of aerial and ground robots. Our system uses semantics as lingua franca, and relies on fully opportunistic communications. We highlight the unique challenges from this approach, explain our system architecture and showcase lessons learned during our experiments. All our code is open-source, encouraging researchers to use it and build upon.
△ Less
Submitted 12 May, 2024;
originally announced May 2024.
-
EvDNeRF: Reconstructing Event Data with Dynamic Neural Radiance Fields
Authors:
Anish Bhattacharya,
Ratnesh Madaan,
Fernando Cladera,
Sai Vemprala,
Rogerio Bonatti,
Kostas Daniilidis,
Ashish Kapoor,
Vijay Kumar,
Nikolai Matni,
Jayesh K. Gupta
Abstract:
We present EvDNeRF, a pipeline for generating event data and training an event-based dynamic NeRF, for the purpose of faithfully reconstructing eventstreams on scenes with rigid and non-rigid deformations that may be too fast to capture with a standard camera. Event cameras register asynchronous per-pixel brightness changes at MHz rates with high dynamic range, making them ideal for observing fast…
▽ More
We present EvDNeRF, a pipeline for generating event data and training an event-based dynamic NeRF, for the purpose of faithfully reconstructing eventstreams on scenes with rigid and non-rigid deformations that may be too fast to capture with a standard camera. Event cameras register asynchronous per-pixel brightness changes at MHz rates with high dynamic range, making them ideal for observing fast motion with almost no motion blur. Neural radiance fields (NeRFs) offer visual-quality geometric-based learnable rendering, but prior work with events has only considered reconstruction of static scenes. Our EvDNeRF can predict eventstreams of dynamic scenes from a static or moving viewpoint between any desired timestamps, thereby allowing it to be used as an event-based simulator for a given scene. We show that by training on varied batch sizes of events, we can improve test-time predictions of events at fine time resolutions, outperforming baselines that pair standard dynamic NeRFs with event generators. We release our simulated and real datasets, as well as code for multi-view event-based data generation and the training and evaluation of EvDNeRF models (https://github.com/anish-bhattacharya/EvDNeRF).
△ Less
Submitted 6 December, 2023; v1 submitted 3 October, 2023;
originally announced October 2023.
-
Enabling Large-scale Heterogeneous Collaboration with Opportunistic Communications
Authors:
Fernando Cladera,
Zachary Ravichandran,
Ian D. Miller,
M. Ani Hsieh,
C. J. Taylor,
Vijay Kumar
Abstract:
Multi-robot collaboration in large-scale environments with limited-sized teams and without external infrastructure is challenging, since the software framework required to support complex tasks must be robust to unreliable and intermittent communication links. In this work, we present MOCHA (Multi-robot Opportunistic Communication for Heterogeneous Collaboration), a framework for resilient multi-r…
▽ More
Multi-robot collaboration in large-scale environments with limited-sized teams and without external infrastructure is challenging, since the software framework required to support complex tasks must be robust to unreliable and intermittent communication links. In this work, we present MOCHA (Multi-robot Opportunistic Communication for Heterogeneous Collaboration), a framework for resilient multi-robot collaboration that enables large-scale exploration in the absence of continuous communications. MOCHA is based on a gossip communication protocol that allows robots to interact opportunistically whenever communication links are available, propagating information on a peer-to-peer basis. We demonstrate the performance of MOCHA through real-world experiments with commercial-off-the-shelf (COTS) communication hardware. We further explore the system's scalability in simulation, evaluating the performance of our approach as the number of robots increases and communication ranges vary. Finally, we demonstrate how MOCHA can be tightly integrated with the planning stack of autonomous robots. We show a communication-aware planning algorithm for a high-altitude aerial robot executing a collaborative task while maximizing the amount of information shared with ground robots. The source code for MOCHA and the high-altitude UAV planning system is available open source: http://github.com/KumarRobotics/MOCHA, http://github.com/KumarRobotics/air_router.
△ Less
Submitted 27 September, 2023;
originally announced September 2023.
-
SEER: Safe Efficient Exploration for Aerial Robots using Learning to Predict Information Gain
Authors:
Yuezhan Tao,
Yuwei Wu,
Beiming Li,
Fernando Cladera,
Alex Zhou,
Dinesh Thakur,
Vijay Kumar
Abstract:
We address the problem of efficient 3-D exploration in indoor environments for micro aerial vehicles with limited sensing capabilities and payload/power constraints. We develop an indoor exploration framework that uses learning to predict the occupancy of unseen areas, extracts semantic features, samples viewpoints to predict information gains for different exploration goals, and plans informative…
▽ More
We address the problem of efficient 3-D exploration in indoor environments for micro aerial vehicles with limited sensing capabilities and payload/power constraints. We develop an indoor exploration framework that uses learning to predict the occupancy of unseen areas, extracts semantic features, samples viewpoints to predict information gains for different exploration goals, and plans informative trajectories to enable safe and smart exploration. Extensive experimentation in simulated and real-world environments shows the proposed approach outperforms the state-of-the-art exploration framework by 24% in terms of the total path length in a structured indoor environment and with a higher success rate during exploration.
△ Less
Submitted 13 August, 2023; v1 submitted 22 September, 2022;
originally announced September 2022.
-
Active Metric-Semantic Map** by Multiple Aerial Robots
Authors:
Xu Liu,
Ankit Prabhu,
Fernando Cladera,
Ian D. Miller,
Lifeng Zhou,
Camillo J. Taylor,
Vijay Kumar
Abstract:
Traditional approaches for active map** focus on building geometric maps. For most real-world applications, however, actionable information is related to semantically meaningful objects in the environment. We propose an approach to the active metric-semantic map** problem that enables multiple heterogeneous robots to collaboratively build a map of the environment. The robots actively explore t…
▽ More
Traditional approaches for active map** focus on building geometric maps. For most real-world applications, however, actionable information is related to semantically meaningful objects in the environment. We propose an approach to the active metric-semantic map** problem that enables multiple heterogeneous robots to collaboratively build a map of the environment. The robots actively explore to minimize the uncertainties in both semantic (object classification) and geometric (object modeling) information. We represent the environment using informative but sparse object models, each consisting of a basic shape and a semantic class label, and characterize uncertainties empirically using a large amount of real-world data. Given a prior map, we use this model to select actions for each robot to minimize uncertainties. The performance of our algorithm is demonstrated through multi-robot experiments in diverse real-world environments. The proposed framework is applicable to a wide range of real-world problems, such as precision agriculture, infrastructure inspection, and asset map** in factories. A demo video can be found at https://youtu.be/S86SgXi54oU.
△ Less
Submitted 13 August, 2023; v1 submitted 17 September, 2022;
originally announced September 2022.
-
Stronger Together: Air-Ground Robotic Collaboration Using Semantics
Authors:
Ian D. Miller,
Fernando Cladera,
Trey Smith,
Camillo Jose Taylor,
Vijay Kumar
Abstract:
In this work, we present an end-to-end heterogeneous multi-robot system framework where ground robots are able to localize, plan, and navigate in a semantic map created in real time by a high-altitude quadrotor. The ground robots choose and deconflict their targets independently, without any external intervention. Moreover, they perform cross-view localization by matching their local maps with the…
▽ More
In this work, we present an end-to-end heterogeneous multi-robot system framework where ground robots are able to localize, plan, and navigate in a semantic map created in real time by a high-altitude quadrotor. The ground robots choose and deconflict their targets independently, without any external intervention. Moreover, they perform cross-view localization by matching their local maps with the overhead map using semantics. The communication backbone is opportunistic and distributed, allowing the entire system to operate with no external infrastructure aside from GPS for the quadrotor. We extensively tested our system by performing different missions on top of our framework over multiple experiments in different environments. Our ground robots travelled over 6 km autonomously with minimal intervention in the real world and over 96 km in simulation without interventions.
△ Less
Submitted 28 June, 2022;
originally announced June 2022.
-
Mine Tunnel Exploration using Multiple Quadrupedal Robots
Authors:
Ian D. Miller,
Fernando Cladera,
Anthony Cowley,
Shreyas S. Shivakumar,
Elijah S. Lee,
Laura Jarin-Lipschitz,
Akhilesh Bhat,
Neil Rodrigues,
Alex Zhou,
Avraham Cohen,
Adarsh Kulkarni,
James Laney,
Camillo Jose Taylor,
Vijay Kumar
Abstract:
Robotic exploration of underground environments is a particularly challenging problem due to communication, endurance, and traversability constraints which necessitate high degrees of autonomy and agility. These challenges are further exacerbated by the need to minimize human intervention for practical applications. While legged robots have the ability to traverse extremely challenging terrain, th…
▽ More
Robotic exploration of underground environments is a particularly challenging problem due to communication, endurance, and traversability constraints which necessitate high degrees of autonomy and agility. These challenges are further exacerbated by the need to minimize human intervention for practical applications. While legged robots have the ability to traverse extremely challenging terrain, they also engender new challenges for planning, estimation, and control. In this work, we describe a fully autonomous system for multi-robot mine exploration and map** using legged quadrupeds, as well as a distributed database mesh networking system for reporting data. In addition, we show results from the DARPA Subterranean Challenge (SubT) Tunnel Circuit demonstrating localization of artifacts after traversals of hundreds of meters. These experiments describe fully autonomous exploration of an unknown Global Navigation Satellite System (GNSS)-denied environment undertaken by legged robots.
△ Less
Submitted 3 February, 2020; v1 submitted 20 September, 2019;
originally announced September 2019.
