-
Field Notes on Deploying Research Robots in Public Spaces
Authors:
Fanjun Bu,
Alexandra Bremers,
Mark Colley,
Wendy Ju
Abstract:
Human-robot interaction requires to be studied in the wild. In the summers of 2022 and 2023, we deployed two trash barrel service robots through the wizard-of-oz protocol in public spaces to study human-robot interactions in urban settings. We deployed the robots at two different public plazas in downtown Manhattan and Brooklyn for a collective of 20 hours of field time. To date, relatively few lo…
▽ More
Human-robot interaction requires to be studied in the wild. In the summers of 2022 and 2023, we deployed two trash barrel service robots through the wizard-of-oz protocol in public spaces to study human-robot interactions in urban settings. We deployed the robots at two different public plazas in downtown Manhattan and Brooklyn for a collective of 20 hours of field time. To date, relatively few long-term human-robot interaction studies have been conducted in shared public spaces. To support researchers aiming to fill this gap, we would like to share some of our insights and learned lessons that would benefit both researchers and practitioners on how to deploy robots in public spaces. We share best practices and lessons learned with the HRI research community to encourage more in-the-wild research of robots in public spaces and call for the community to share their lessons learned to a GitHub repository.
△ Less
Submitted 28 April, 2024;
originally announced April 2024.
-
A utility belt for an agricultural robot: reflection-in-action for applied design research
Authors:
Natalie Friedman,
Asmita Mehta,
Kari Love,
Alexandra Bremers,
Awsaf Ahmed,
Wendy Ju
Abstract:
Clothing for robots can help expand a robot's functionality and also clarify the robot's purpose to bystanders. In studying how to design clothing for robots, we can shed light on the functional role of aesthetics in interactive system design. We present a case study of designing a utility belt for an agricultural robot. We use reflection-in-action to consider the ways that observation, in situ ma…
▽ More
Clothing for robots can help expand a robot's functionality and also clarify the robot's purpose to bystanders. In studying how to design clothing for robots, we can shed light on the functional role of aesthetics in interactive system design. We present a case study of designing a utility belt for an agricultural robot. We use reflection-in-action to consider the ways that observation, in situ making, and documentation serve to illuminate how pragmatic, aesthetic, and intellectual inquiry are layered in this applied design research project. Themes explored in this pictorial include 1) contextual discovery of materials, tools, and practices, 2) design space exploration of materials in context, 3) improvising spaces for making, and 4) social processes in design. These themes emerged from the qualitative coding of 25 reflection-in-action videos from the researcher. We conclude with feedback on the utility belt prototypes for an agriculture robot and our learnings about context, materials, and people needed to design successful novel clothing forms for robots.
△ Less
Submitted 21 December, 2023;
originally announced December 2023.
-
Initiative and Materiality: Exploring Mixed-Initiative Calculators with the Tangible Human-A.I. Interaction Framework
Authors:
Lunshi Zhou,
Alexandra Bremers,
Wendy Ju
Abstract:
How can interactions with A.I. systems be designed? This paper explores the design space for A.I. interaction to develop tools for designers to think about tangible and physical A.I. interactions. Our proposed framework consists of two dimensions: initiative (human, mixed, or machine) and materiality (physical, combined, or digital form). A particularly interesting area of interactions we identify…
▽ More
How can interactions with A.I. systems be designed? This paper explores the design space for A.I. interaction to develop tools for designers to think about tangible and physical A.I. interactions. Our proposed framework consists of two dimensions: initiative (human, mixed, or machine) and materiality (physical, combined, or digital form). A particularly interesting area of interactions we identify is the quadrant of physical, machine-initiated interactions. With our framework, we examine calculator interactions and attempt to expand these to the tangible, mixed-initiative space. We illustrate each area in our proposed framework with one representative example of a calculator -- a common and well-known example of a computing device. We discuss existing examples of calculators and speculative future interactions with mixed-initiative and physical calculator systems. We reflect on the implications of our framework for the larger task of designing human-A.I. collaborative systems. Designers can also apply this framework as a guideline for analogous solutions to problems in the same domain.
△ Less
Submitted 10 November, 2023;
originally announced November 2023.
-
(Social) Trouble on the Road: Understanding and Addressing Social Discomfort in Shared Car Trips
Authors:
Alexandra Bremers,
Natalie Friedman,
Sam Lee,
Tong Wu,
Eric Laurier,
Malte Jung,
Jorge Ortiz,
Wendy Ju
Abstract:
Unpleasant social interactions on the road can negatively affect driving safety. At the same time, researchers have attempted to address social discomfort by exploring Conversational User Interfaces (CUIs) as social mediators. Before knowing whether CUIs could reduce social discomfort in a car, it is necessary to understand the nature of social discomfort in shared rides. To this end, we recorded…
▽ More
Unpleasant social interactions on the road can negatively affect driving safety. At the same time, researchers have attempted to address social discomfort by exploring Conversational User Interfaces (CUIs) as social mediators. Before knowing whether CUIs could reduce social discomfort in a car, it is necessary to understand the nature of social discomfort in shared rides. To this end, we recorded nine families going on drives and performed interaction analysis on this data. We define three strategies to address social discomfort: contextual mediation, social mediation, and social support. We discuss considerations for engineering and design, and explore the limitations of current large language models in addressing social discomfort on the road.
△ Less
Submitted 24 May, 2024; v1 submitted 7 November, 2023;
originally announced November 2023.
-
Implicit collaboration with a drawing machine through dance movements
Authors:
Itay Grinberg,
Alexandra Bremers,
Louisa Pancoast,
Wendy Ju
Abstract:
In this demonstration, we exhibit the initial results of an ongoing body of exploratory work, investigating the potential for creative machines to communicate and collaborate with people through movement as a form of implicit interaction. The paper describes a Wizard-of-Oz demo, where a hidden wizard controls an AxiDraw drawing robot while a participant collaborates with it to draw a custom postca…
▽ More
In this demonstration, we exhibit the initial results of an ongoing body of exploratory work, investigating the potential for creative machines to communicate and collaborate with people through movement as a form of implicit interaction. The paper describes a Wizard-of-Oz demo, where a hidden wizard controls an AxiDraw drawing robot while a participant collaborates with it to draw a custom postcard. This demonstration aims to gather perspectives from the computational fabrication community regarding how practitioners of fabrication with machines experience interacting with a mixed-initiative collaborative machine.
△ Less
Submitted 29 September, 2023;
originally announced October 2023.
-
Driver Profiling and Bayesian Workload Estimation Using Naturalistic Peripheral Detection Study Data
Authors:
Nermin Caber,
Bashar I. Ahmad,
Jiaming Liang,
Simon Godsill,
Alexandra Bremers,
Philip Thomas,
David Oxtoby,
Lee Skrypchuk
Abstract:
Monitoring drivers' mental workload facilitates initiating and maintaining safe interactions with in-vehicle information systems, and thus delivers adaptive human machine interaction with reduced impact on the primary task of driving. In this paper, we tackle the problem of workload estimation from driving performance data. First, we present a novel on-road study for collecting subjective workload…
▽ More
Monitoring drivers' mental workload facilitates initiating and maintaining safe interactions with in-vehicle information systems, and thus delivers adaptive human machine interaction with reduced impact on the primary task of driving. In this paper, we tackle the problem of workload estimation from driving performance data. First, we present a novel on-road study for collecting subjective workload data via a modified peripheral detection task in naturalistic settings. Key environmental factors that induce a high mental workload are identified via video analysis, e.g. junctions and behaviour of vehicle in front. Second, a supervised learning framework using state-of-the-art time series classifiers (e.g. convolutional neural network and transform techniques) is introduced to profile drivers based on the average workload they experience during a journey. A Bayesian filtering approach is then proposed for sequentially estimating, in (near) real-time, the driver's instantaneous workload. This computationally efficient and flexible method can be easily personalised to a driver (e.g. incorporate their inferred average workload profile), adapted to driving/environmental contexts (e.g. road type) and extended with data streams from new sources. The efficacy of the presented profiling and instantaneous workload estimation approaches are demonstrated using the on-road study data, showing $F_{1}$ scores of up to 92% and 81%, respectively.
△ Less
Submitted 8 September, 2023; v1 submitted 26 March, 2023;
originally announced March 2023.
-
The Bystander Affect Detection (BAD) Dataset for Failure Detection in HRI
Authors:
Alexandra Bremers,
Maria Teresa Parreira,
Xuanyu Fang,
Natalie Friedman,
Adolfo Ramirez-Aristizabal,
Alexandria Pabst,
Mirjana Spasojevic,
Michael Kuniavsky,
Wendy Ju
Abstract:
For a robot to repair its own error, it must first know it has made a mistake. One way that people detect errors is from the implicit reactions from bystanders -- their confusion, smirks, or giggles clue us in that something unexpected occurred. To enable robots to detect and act on bystander responses to task failures, we developed a novel method to elicit bystander responses to human and robot e…
▽ More
For a robot to repair its own error, it must first know it has made a mistake. One way that people detect errors is from the implicit reactions from bystanders -- their confusion, smirks, or giggles clue us in that something unexpected occurred. To enable robots to detect and act on bystander responses to task failures, we developed a novel method to elicit bystander responses to human and robot errors. Using 46 different stimulus videos featuring a variety of human and machine task failures, we collected a total of 2452 webcam videos of human reactions from 54 participants. To test the viability of the collected data, we used the bystander reaction dataset as input to a deep-learning model, BADNet, to predict failure occurrence. We tested different data labeling methods and learned how they affect model performance, achieving precisions above 90%. We discuss strategies to model bystander reactions and predict failure and how this approach can be used in real-world robotic deployments to detect errors and improve robot performance. As part of this work, we also contribute with the "Bystander Affect Detection" (BAD) dataset of bystander reactions, supporting the development of better prediction models.
△ Less
Submitted 8 March, 2023;
originally announced March 2023.
-
Using Social Cues to Recognize Task Failures for HRI: Overview, State-of-the-Art, and Future Directions
Authors:
Alexandra Bremers,
Alexandria Pabst,
Maria Teresa Parreira,
Wendy Ju
Abstract:
Robots that carry out tasks and interact in complex environments will inevitably commit errors. Error detection is thus an essential ability for robots to master to work efficiently and productively. People can leverage social feedback to get an indication of whether an action was successful or not. With advances in computing and artificial intelligence (AI), it is increasingly possible for robots…
▽ More
Robots that carry out tasks and interact in complex environments will inevitably commit errors. Error detection is thus an essential ability for robots to master to work efficiently and productively. People can leverage social feedback to get an indication of whether an action was successful or not. With advances in computing and artificial intelligence (AI), it is increasingly possible for robots to achieve a similar capability of collecting social feedback. In this work, we take this one step further and propose a framework for how social cues can be used as feedback signals to recognize task failures for human-robot interaction (HRI). Our proposed framework sets out a research agenda based on insights from the literature on behavioral science, human-robot interaction, and machine learning to focus on three areas: 1) social cues as feedback (from behavioral science), 2) recognizing task failures in robots (from HRI), and 3) approaches for autonomous detection of HRI task failures based on social cues (from machine learning). We propose a taxonomy of error detection based on self-awareness and social feedback. Finally, we provide recommendations for HRI researchers and practitioners interested in develo** robots that detect task errors using human social cues. This article is intended for interdisciplinary HRI researchers and practitioners, where the third theme of our analysis provides more technical details aiming toward the practical implementation of these systems.
△ Less
Submitted 29 May, 2024; v1 submitted 27 January, 2023;
originally announced January 2023.
