-
PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigators
Authors:
Kuo-Hao Zeng,
Zichen Zhang,
Kiana Ehsani,
Rose Hendrix,
Jordi Salvador,
Alvaro Herrasti,
Ross Girshick,
Aniruddha Kembhavi,
Luca Weihs
Abstract:
We present PoliFormer (Policy Transformer), an RGB-only indoor navigation agent trained end-to-end with reinforcement learning at scale that generalizes to the real-world without adaptation despite being trained purely in simulation. PoliFormer uses a foundational vision transformer encoder with a causal transformer decoder enabling long-term memory and reasoning. It is trained for hundreds of mil…
▽ More
We present PoliFormer (Policy Transformer), an RGB-only indoor navigation agent trained end-to-end with reinforcement learning at scale that generalizes to the real-world without adaptation despite being trained purely in simulation. PoliFormer uses a foundational vision transformer encoder with a causal transformer decoder enabling long-term memory and reasoning. It is trained for hundreds of millions of interactions across diverse environments, leveraging parallelized, multi-machine rollouts for efficient training with high throughput. PoliFormer is a masterful navigator, producing state-of-the-art results across two distinct embodiments, the LoCoBot and Stretch RE-1 robots, and four navigation benchmarks. It breaks through the plateaus of previous work, achieving an unprecedented 85.5% success rate in object goal navigation on the CHORES-S benchmark, a 28.5% absolute improvement. PoliFormer can also be trivially extended to a variety of downstream applications such as object tracking, multi-object navigation, and open-vocabulary navigation with no finetuning.
△ Less
Submitted 28 June, 2024;
originally announced June 2024.
-
Imitating Shortest Paths in Simulation Enables Effective Navigation and Manipulation in the Real World
Authors:
Kiana Ehsani,
Tanmay Gupta,
Rose Hendrix,
Jordi Salvador,
Luca Weihs,
Kuo-Hao Zeng,
Kunal Pratap Singh,
Ye** Kim,
Winson Han,
Alvaro Herrasti,
Ranjay Krishna,
Dustin Schwenk,
Eli VanderBilt,
Aniruddha Kembhavi
Abstract:
Reinforcement learning (RL) with dense rewards and imitation learning (IL) with human-generated trajectories are the most widely used approaches for training modern embodied agents. RL requires extensive reward sha** and auxiliary losses and is often too slow and ineffective for long-horizon tasks. While IL with human supervision is effective, collecting human trajectories at scale is extremely…
▽ More
Reinforcement learning (RL) with dense rewards and imitation learning (IL) with human-generated trajectories are the most widely used approaches for training modern embodied agents. RL requires extensive reward sha** and auxiliary losses and is often too slow and ineffective for long-horizon tasks. While IL with human supervision is effective, collecting human trajectories at scale is extremely expensive. In this work, we show that imitating shortest-path planners in simulation produces agents that, given a language instruction, can proficiently navigate, explore, and manipulate objects in both simulation and in the real world using only RGB sensors (no depth map or GPS coordinates). This surprising result is enabled by our end-to-end, transformer-based, SPOC architecture, powerful visual encoders paired with extensive image augmentation, and the dramatic scale and diversity of our training data: millions of frames of shortest-path-expert trajectories collected inside approximately 200,000 procedurally generated houses containing 40,000 unique 3D assets. Our models, data, training code, and newly proposed 10-task benchmarking suite CHORES will be open-sourced.
△ Less
Submitted 5 December, 2023;
originally announced December 2023.
-
Open X-Embodiment: Robotic Learning Datasets and RT-X Models
Authors:
Open X-Embodiment Collaboration,
Abby O'Neill,
Abdul Rehman,
Abhinav Gupta,
Abhiram Maddukuri,
Abhishek Gupta,
Abhishek Padalkar,
Abraham Lee,
Acorn Pooley,
Agrim Gupta,
Ajay Mandlekar,
A**kya Jain,
Albert Tung,
Alex Bewley,
Alex Herzog,
Alex Irpan,
Alexander Khazatsky,
Anant Rai,
Anchit Gupta,
Andrew Wang,
Andrey Kolobov,
Anikait Singh,
Animesh Garg,
Aniruddha Kembhavi,
Annie Xie
, et al. (267 additional authors not shown)
Abstract:
Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for many applications. Can such a consolidation happen in robotics? Conventionally, robotic learning method…
▽ More
Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for many applications. Can such a consolidation happen in robotics? Conventionally, robotic learning methods train a separate model for every application, every robot, and even every environment. Can we instead train generalist X-robot policy that can be adapted efficiently to new robots, tasks, and environments? In this paper, we provide datasets in standardized data formats and models to make it possible to explore this possibility in the context of robotic manipulation, alongside experimental results that provide an example of effective X-robot policies. We assemble a dataset from 22 different robots collected through a collaboration between 21 institutions, demonstrating 527 skills (160266 tasks). We show that a high-capacity model trained on this data, which we call RT-X, exhibits positive transfer and improves the capabilities of multiple robots by leveraging experience from other platforms. More details can be found on the project website https://robotics-transformer-x.github.io.
△ Less
Submitted 1 June, 2024; v1 submitted 13 October, 2023;
originally announced October 2023.
-
Phone2Proc: Bringing Robust Robots Into Our Chaotic World
Authors:
Matt Deitke,
Rose Hendrix,
Luca Weihs,
Ali Farhadi,
Kiana Ehsani,
Aniruddha Kembhavi
Abstract:
Training embodied agents in simulation has become mainstream for the embodied AI community. However, these agents often struggle when deployed in the physical world due to their inability to generalize to real-world environments. In this paper, we present Phone2Proc, a method that uses a 10-minute phone scan and conditional procedural generation to create a distribution of training scenes that are…
▽ More
Training embodied agents in simulation has become mainstream for the embodied AI community. However, these agents often struggle when deployed in the physical world due to their inability to generalize to real-world environments. In this paper, we present Phone2Proc, a method that uses a 10-minute phone scan and conditional procedural generation to create a distribution of training scenes that are semantically similar to the target environment. The generated scenes are conditioned on the wall layout and arrangement of large objects from the scan, while also sampling lighting, clutter, surface textures, and instances of smaller objects with randomized placement and materials. Leveraging just a simple RGB camera, training with Phone2Proc shows massive improvements from 34.7% to 70.7% success rate in sim-to-real ObjectNav performance across a test suite of over 200 trials in diverse real-world environments, including homes, offices, and RoboTHOR. Furthermore, Phone2Proc's diverse distribution of generated scenes makes agents remarkably robust to changes in the real world, such as human movement, object rearrangement, lighting changes, or clutter.
△ Less
Submitted 8 December, 2022;
originally announced December 2022.
-
Science goals and new mission concepts for future exploration of Titan's atmosphere geology and habitability: Titan POlar Scout/orbitEr and In situ lake lander and DrONe explorer (POSEIDON)
Authors:
Sébastien Rodriguez,
Sandrine Vinatier,
Daniel Cordier,
Gabriel Tobie,
Richard K. Achterberg,
Carrie M. Anderson,
Sarah V. Badman,
Jason W. Barnes,
Erika L. Barth,
Bruno Bézard,
Nathalie Carrasco,
Benjamin Charnay,
Roger N. Clark,
Patrice Coll,
Thomas Cornet,
Athena Coustenis,
Isabelle Couturier-Tamburelli,
Michel Dobrijevic,
F. Michael Flasar,
Remco de Kok,
Caroline Freissinet,
Marina Galand,
Thomas Gautier,
Wolf D. Geppert,
Caitlin A. Griffith
, et al. (39 additional authors not shown)
Abstract:
In response to ESA Voyage 2050 announcement of opportunity, we propose an ambitious L-class mission to explore one of the most exciting bodies in the Solar System, Saturn largest moon Titan. Titan, a "world with two oceans", is an organic-rich body with interior-surface-atmosphere interactions that are comparable in complexity to the Earth. Titan is also one of the few places in the Solar System w…
▽ More
In response to ESA Voyage 2050 announcement of opportunity, we propose an ambitious L-class mission to explore one of the most exciting bodies in the Solar System, Saturn largest moon Titan. Titan, a "world with two oceans", is an organic-rich body with interior-surface-atmosphere interactions that are comparable in complexity to the Earth. Titan is also one of the few places in the Solar System with habitability potential. Titan remarkable nature was only partly revealed by the Cassini-Huygens mission and still holds mysteries requiring a complete exploration using a variety of vehicles and instruments. The proposed mission concept POSEIDON (Titan POlar Scout/orbitEr and In situ lake lander DrONe explorer) would perform joint orbital and in situ investigations of Titan. It is designed to build on and exceed the scope and scientific/technological accomplishments of Cassini-Huygens, exploring Titan in ways that were not previously possible, in particular through full close-up and in situ coverage over long periods of time. In the proposed mission architecture, POSEIDON consists of two major elements: a spacecraft with a large set of instruments that would orbit Titan, preferably in a low-eccentricity polar orbit, and a suite of in situ investigation components, i.e. a lake lander, a "heavy" drone (possibly amphibious) and/or a fleet of mini-drones, dedicated to the exploration of the polar regions. The ideal arrival time at Titan would be slightly before the next northern Spring equinox (2039), as equinoxes are the most active periods to monitor still largely unknown atmospheric and surface seasonal changes. The exploration of Titan northern latitudes with an orbiter and in situ element(s) would be highly complementary with the upcoming NASA New Frontiers Dragonfly mission that will provide in situ exploration of Titan equatorial regions in the mid-2030s.
△ Less
Submitted 20 October, 2021;
originally announced October 2021.
-
The Saturn Ring Skimmer Mission Concept: The next step to explore Saturn's rings, atmosphere, interior, and inner magnetosphere
Authors:
Matthew S. Tiscareno,
Mar Vaquero,
Matthew M. Hedman,
Hao Cao,
Paul R. Estrada,
Andrew P. Ingersoll,
Kelly E. Miller,
Marzia Parisi,
David. H. Atkinson,
Shawn M. Brooks,
Jeffrey N. Cuzzi,
James Fuller,
Amanda R. Hendrix,
Robert E. Johnson,
Tommi Koskinen,
William S. Kurth,
Jonathan I. Lunine,
Philip D. Nicholson,
Carol S. Paty,
Rebecca Schindhelm,
Mark R. Showalter,
Linda J. Spilker,
Nathan J. Strange,
Wendy Tseng
Abstract:
The innovative Saturn Ring Skimmer mission concept enables a wide range of investigations that address fundamental questions about Saturn and its rings, as well as giant planets and astrophysical disk systems in general. This mission would provide new insights into the dynamical processes that operate in astrophysical disk systems by observing individual particles in Saturn's rings for the first t…
▽ More
The innovative Saturn Ring Skimmer mission concept enables a wide range of investigations that address fundamental questions about Saturn and its rings, as well as giant planets and astrophysical disk systems in general. This mission would provide new insights into the dynamical processes that operate in astrophysical disk systems by observing individual particles in Saturn's rings for the first time. The Ring Skimmer would also constrain the origin, history, and fate of Saturn's rings by determining their compositional evolution and material transport rates. In addition, the Ring Skimmer would reveal how the rings, magnetosphere, and planet operate as an inter-connected system by making direct measurements of the ring's atmosphere, Saturn's inner magnetosphere and the material owing from the rings into the planet. At the same time, this mission would clarify the dynamical processes operating in the planet's visible atmosphere and deep interior by making extensive high-resolution observations of cloud features and repeated measurements of the planet's extremely dynamic gravitational field. Given the scientific potential of this basic mission concept, we advocate that it be studied in depth as a potential option for the New Frontiers program.
△ Less
Submitted 16 September, 2020; v1 submitted 30 July, 2020;
originally announced July 2020.
-
Ultraviolet-Based Science in the Solar System: Advances and Next Steps
Authors:
Amanda R. Hendrix,
Tracy M. Becker,
Dennis Bodewits,
E. Todd Bradley,
Shawn Brooks,
Ben Byron,
Josh Cahill,
John Clarke,
Lori Feaga,
Paul Feldman,
G. Randall Gladstone,
Candice J. Hansen,
Charles Hibbitts,
Tommi T. Koskinen,
Lizeth Magana,
Philippa Molyneux,
Shouleh Nikzad,
John Noonan,
Wayne Pryor,
Ujjwal Raut,
Kurt D. Retherford,
Lorenz Roth,
Emilie Royer,
Ella Sciamma-O'Brien,
Alan Stern
, et al. (3 additional authors not shown)
Abstract:
We review the importance of recent UV observations of solar system targets and discuss the need for further measurements, instrumentation and laboratory work in the coming decade.
In the past decade, numerous important advances have been made in solar system science using ultraviolet (UV) spectroscopic techniques. Formerly used nearly exclusively for studies of giant planet atmospheres, planetar…
▽ More
We review the importance of recent UV observations of solar system targets and discuss the need for further measurements, instrumentation and laboratory work in the coming decade.
In the past decade, numerous important advances have been made in solar system science using ultraviolet (UV) spectroscopic techniques. Formerly used nearly exclusively for studies of giant planet atmospheres, planetary exospheres and cometary emissions, UV imaging spectroscopy has recently been more widely applied. The geyser-like plume at Saturn's moon Enceladus was discovered in part as a result of UV stellar occultation observations, and this technique was used to characterize the plume and jets during the entire Cassini mission. Evidence for a similar style of activity has been found at Jupiter's moon Europa using Hubble Space Telescope (HST) UV emission and absorption imaging. At other moons and small bodies throughout the solar system, UV spectroscopy has been utilized to search for activity, probe surface composition, and delineate space weathering effects; UV photometric studies have been used to uncover regolith structure. Insights from UV imaging spectroscopy of solar system surfaces have been gained largely in the last 1-2 decades, including studies of surface composition, space weathering effects (e.g. radiolytic products) and volatiles on asteroids (e.g. [2][39][48][76][84]), the Moon (e.g. [30][46][49]), comet nuclei (e.g. [85]) and icy satellites (e.g. [38][41-44][45][47][65]). The UV is sensitive to some species, minor contaminants and grain sizes often not detected in other spectral regimes.
In the coming decade, HST observations will likely come to an end. New infrastructure to bolster future UV studies is critically needed. These needs include both developmental work to help improve future UV observations and laboratory work to help interpret spacecraft data. UV instrumentation will be a critical tool on missions to a variety of targets in the coming decade, especially for the rapidly expanding application of UV reflectance investigations of atmosphereless bodies.
△ Less
Submitted 28 July, 2020;
originally announced July 2020.
-
Toward Ergonomic Risk Prediction via Segmentation of Indoor Object Manipulation Actions Using Spatiotemporal Convolutional Networks
Authors:
Behnoosh Parsa,
Ekta U. Samani,
Rose Hendrix,
Cameron Devine,
Shashi M. Singh,
Santosh Devasia,
Ashis G. Banerjee
Abstract:
Automated real-time prediction of the ergonomic risks of manipulating objects is a key unsolved challenge in develo** effective human-robot collaboration systems for logistics and manufacturing applications. We present a foundational paradigm to address this challenge by formulating the problem as one of action segmentation from RGB-D camera videos. Spatial features are first learned using a dee…
▽ More
Automated real-time prediction of the ergonomic risks of manipulating objects is a key unsolved challenge in develo** effective human-robot collaboration systems for logistics and manufacturing applications. We present a foundational paradigm to address this challenge by formulating the problem as one of action segmentation from RGB-D camera videos. Spatial features are first learned using a deep convolutional model from the video frames, which are then fed sequentially to temporal convolutional networks to semantically segment the frames into a hierarchy of actions, which are either ergonomically safe, require monitoring, or need immediate attention. For performance evaluation, in addition to an open-source kitchen dataset, we collected a new dataset comprising twenty individuals picking up and placing objects of varying weights to and from cabinet and table locations at various heights. Results show very high (87-94)\% F1 overlap scores among the ground truth and predicted frame labels for videos lasting over two minutes and consisting of a large number of actions.
△ Less
Submitted 26 June, 2019; v1 submitted 13 February, 2019;
originally announced February 2019.
-
Energy Options for Future Humans on Titan
Authors:
Amanda R. Hendrix,
Yuk L. Yung
Abstract:
We review the possibilities for in situ energy resources on Titan for use by future humans, including chemical, nuclear, wind, solar, geothermal and hydropower. All of these options, with the possible exception of geothermal, represent effective sources of power. Combustion of methane (after electrolysis of the native water), in combination with another source of power such as nuclear, is a viable…
▽ More
We review the possibilities for in situ energy resources on Titan for use by future humans, including chemical, nuclear, wind, solar, geothermal and hydropower. All of these options, with the possible exception of geothermal, represent effective sources of power. Combustion of methane (after electrolysis of the native water), in combination with another source of power such as nuclear, is a viable option; another chemical source of energy is the hydrogenation of acetylene. The large seas Kraken and Ligeia potentially represent effective sources of hydropower. Wind power, particularly at altitudes ~40 km, is expected to be productive. Despite the distance from the sun and the absorbing atmosphere, solar power is (as on Earth) an extremely efficient source of power on Titan.
△ Less
Submitted 2 July, 2017;
originally announced July 2017.