Alexa, play with robot: Introducing the First Alexa Prize SimBot Challenge on Embodied AI
Authors:
Hangjie Shi,
Leslie Ball,
Govind Thattai,
Desheng Zhang,
Lucy Hu,
Qiaozi Gao,
Suhaila Shakiah,
Xiaofeng Gao,
Aishwarya Padmakumar,
Bofei Yang,
Cadence Chung,
Dinakar Guthy,
Gaurav Sukhatme,
Karthika Arumugam,
Matthew Wen,
Osman Ipek,
Patrick Lange,
Rohan Khanna,
Shreyas Pansare,
Vasu Sharma,
Chao Zhang,
Cris Flagg,
Daniel Pressel,
Lavina Vaz,
Luke Dai
, et al. (17 additional authors not shown)
Abstract:
The Alexa Prize program has empowered numerous university students to explore, experiment, and showcase their talents in building conversational agents through challenges like the SocialBot Grand Challenge and the TaskBot Challenge. As conversational agents increasingly appear in multimodal and embodied contexts, it is important to explore the affordances of conversational interaction augmented wi…
▽ More
The Alexa Prize program has empowered numerous university students to explore, experiment, and showcase their talents in building conversational agents through challenges like the SocialBot Grand Challenge and the TaskBot Challenge. As conversational agents increasingly appear in multimodal and embodied contexts, it is important to explore the affordances of conversational interaction augmented with computer vision and physical embodiment. This paper describes the SimBot Challenge, a new challenge in which university teams compete to build robot assistants that complete tasks in a simulated physical environment. This paper provides an overview of the SimBot Challenge, which included both online and offline challenge phases. We describe the infrastructure and support provided to the teams including Alexa Arena, the simulated environment, and the ML toolkit provided to teams to accelerate their building of vision and language models. We summarize the approaches the participating teams took to overcome research challenges and extract key lessons learned. Finally, we provide analysis of the performance of the competing SimBots during the competition.
△ Less
Submitted 9 August, 2023;
originally announced August 2023.
Alexa Arena: A User-Centric Interactive Platform for Embodied AI
Authors:
Qiaozi Gao,
Govind Thattai,
Suhaila Shakiah,
Xiaofeng Gao,
Shreyas Pansare,
Vasu Sharma,
Gaurav Sukhatme,
Hangjie Shi,
Bofei Yang,
Desheng Zheng,
Lucy Hu,
Karthika Arumugam,
Shui Hu,
Matthew Wen,
Dinakar Guthy,
Cadence Chung,
Rohan Khanna,
Osman Ipek,
Leslie Ball,
Kate Bland,
Heather Rocker,
Yadunandana Rao,
Michael Johnston,
Reza Ghanadan,
Arindam Mandal
, et al. (2 additional authors not shown)
Abstract:
We introduce Alexa Arena, a user-centric simulation platform for Embodied AI (EAI) research. Alexa Arena provides a variety of multi-room layouts and interactable objects, for the creation of human-robot interaction (HRI) missions. With user-friendly graphics and control mechanisms, Alexa Arena supports the development of gamified robotic tasks readily accessible to general human users, thus openi…
▽ More
We introduce Alexa Arena, a user-centric simulation platform for Embodied AI (EAI) research. Alexa Arena provides a variety of multi-room layouts and interactable objects, for the creation of human-robot interaction (HRI) missions. With user-friendly graphics and control mechanisms, Alexa Arena supports the development of gamified robotic tasks readily accessible to general human users, thus opening a new venue for high-efficiency HRI data collection and EAI system evaluation. Along with the platform, we introduce a dialog-enabled instruction-following benchmark and provide baseline results for it. We make Alexa Arena publicly available to facilitate research in building generalizable and assistive embodied agents.
△ Less
Submitted 7 June, 2023; v1 submitted 2 March, 2023;
originally announced March 2023.
