Demonstrating Mobile Manipulation in the Wild: A Metrics-Driven Approach
Authors:
Max Bajracharya,
James Borders,
Richard Cheng,
Dan Helmick,
Lukas Kaul,
Dan Kruse,
John Leichty,
Jeremy Ma,
Carolyn Matl,
Frank Michel,
Chavdar Papazov,
Josh Petersen,
Krishna Shankar,
Mark Tjersland
Abstract:
We present our general-purpose mobile manipulation system consisting of a custom robot platform and key algorithms spanning perception and planning. To extensively test the system in the wild and benchmark its performance, we choose a grocery shop** scenario in an actual, unmodified grocery store. We derive key performance metrics from detailed robot log data collected during six week-long field…
▽ More
We present our general-purpose mobile manipulation system consisting of a custom robot platform and key algorithms spanning perception and planning. To extensively test the system in the wild and benchmark its performance, we choose a grocery shop** scenario in an actual, unmodified grocery store. We derive key performance metrics from detailed robot log data collected during six week-long field tests, spread across 18 months. These objective metrics, gained from complex yet repeatable tests, drive the direction of our research efforts and let us continuously improve our system's performance. We find that thorough end-to-end system-level testing of a complex mobile manipulation system can serve as a reality-check for state-of-the-art methods in robotics. This effectively grounds robotics research efforts in real world needs and challenges, which we deem highly useful for the advancement of the field. To this end, we share our key insights and takeaways to inspire and accelerate similar system-level research projects.
△ Less
Submitted 2 January, 2024;
originally announced January 2024.
A Mobile Manipulation System for One-Shot Teaching of Complex Tasks in Homes
Authors:
Max Bajracharya,
James Borders,
Dan Helmick,
Thomas Kollar,
Michael Laskey,
John Leichty,
Jeremy Ma,
Umashankar Nagarajan,
Akiyoshi Ochiai,
Josh Petersen,
Krishna Shankar,
Kevin Stone,
Yutaka Takaoka
Abstract:
We describe a mobile manipulation hardware and software system capable of autonomously performing complex human-level tasks in real homes, after being taught the task with a single demonstration from a person in virtual reality. This is enabled by a highly capable mobile manipulation robot, whole-body task space hybrid position/force control, teaching of parameterized primitives linked to a robust…
▽ More
We describe a mobile manipulation hardware and software system capable of autonomously performing complex human-level tasks in real homes, after being taught the task with a single demonstration from a person in virtual reality. This is enabled by a highly capable mobile manipulation robot, whole-body task space hybrid position/force control, teaching of parameterized primitives linked to a robust learned dense visual embeddings representation of the scene, and a task graph of the taught behaviors. We demonstrate the robustness of the approach by presenting results for performing a variety of tasks, under different environmental conditions, in multiple real homes. Our approach achieves 85% overall success rate on three tasks that consist of an average of 45 behaviors each.
△ Less
Submitted 3 March, 2020; v1 submitted 30 September, 2019;
originally announced October 2019.
