-
Socially Pertinent Robots in Gerontological Healthcare
Authors:
Xavier Alameda-Pineda,
Angus Addlesee,
Daniel Hernández García,
Chris Reinke,
Soraya Arias,
Federica Arrigoni,
Alex Auternaud,
Lauriane Blavette,
Cigdem Beyan,
Luis Gomez Camara,
Ohad Cohen,
Alessandro Conti,
Sébastien Dacunha,
Christian Dondrup,
Yoav Ellinson,
Francesco Ferro,
Sharon Gannot,
Florian Gras,
Nancie Gunson,
Radu Horaud,
Moreno D'Incà,
Imad Kimouche,
Séverin Lemaignan,
Oliver Lemon,
Cyril Liotard
, et al. (19 additional authors not shown)
Abstract:
Despite the many recent achievements in develo** and deploying social robotics, there are still many underexplored environments and applications for which systematic evaluation of such systems by end-users is necessary. While several robotic platforms have been used in gerontological healthcare, the question of whether or not a social interactive robot with multi-modal conversational capabilitie…
▽ More
Despite the many recent achievements in develo** and deploying social robotics, there are still many underexplored environments and applications for which systematic evaluation of such systems by end-users is necessary. While several robotic platforms have been used in gerontological healthcare, the question of whether or not a social interactive robot with multi-modal conversational capabilities will be useful and accepted in real-life facilities is yet to be answered. This paper is an attempt to partially answer this question, via two waves of experiments with patients and companions in a day-care gerontological facility in Paris with a full-sized humanoid robot endowed with social and conversational interaction capabilities. The software architecture, developed during the H2020 SPRING project, together with the experimental protocol, allowed us to evaluate the acceptability (AES) and usability (SUS) with more than 60 end-users. Overall, the users are receptive to this technology, especially when the robot perception and action skills are robust to environmental clutter and flexible to handle a plethora of different interactions.
△ Less
Submitted 11 April, 2024;
originally announced April 2024.
-
Detecting Agreement in Multi-party Conversational AI
Authors:
Laura Schauer,
Jason Sweeney,
Charlie Lyttle,
Zein Said,
Aron Szeles,
Cale Clark,
Katie McAskill,
Xander Wickham,
Tom Byars,
Daniel Hernández Garcia,
Nancie Gunson,
Angus Addlesee,
Oliver Lemon
Abstract:
Today, conversational systems are expected to handle conversations in multi-party settings, especially within Socially Assistive Robots (SARs). However, practical usability remains difficult as there are additional challenges to overcome, such as speaker recognition, addressee recognition, and complex turn-taking. In this paper, we present our work on a multi-party conversational system, which inv…
▽ More
Today, conversational systems are expected to handle conversations in multi-party settings, especially within Socially Assistive Robots (SARs). However, practical usability remains difficult as there are additional challenges to overcome, such as speaker recognition, addressee recognition, and complex turn-taking. In this paper, we present our work on a multi-party conversational system, which invites two users to play a trivia quiz game. The system detects users' agreement or disagreement on a final answer and responds accordingly. Our evaluation includes both performance and user assessment results, with a focus on detecting user agreement. Our annotated transcripts and the code for the proposed system have been released open-source on GitHub.
△ Less
Submitted 6 November, 2023;
originally announced November 2023.
-
Detecting agreement in multi-party dialogue: evaluating speaker diarisation versus a procedural baseline to enhance user engagement
Authors:
Angus Addlesee,
Daniel Denley,
Andy Edmondson,
Nancie Gunson,
Daniel Hernández Garcia,
Alexandre Kha,
Oliver Lemon,
James Ndubuisi,
Neil O'Reilly,
Lia Perochaud,
Raphaël Valeri,
Miebaka Worika
Abstract:
Conversational agents participating in multi-party interactions face significant challenges in dialogue state tracking, since the identity of the speaker adds significant contextual meaning. It is common to utilise diarisation models to identify the speaker. However, it is not clear if these are accurate enough to correctly identify specific conversational events such as agreement or disagreement…
▽ More
Conversational agents participating in multi-party interactions face significant challenges in dialogue state tracking, since the identity of the speaker adds significant contextual meaning. It is common to utilise diarisation models to identify the speaker. However, it is not clear if these are accurate enough to correctly identify specific conversational events such as agreement or disagreement during a real-time interaction. This study uses a cooperative quiz, where the conversational agent acts as quiz-show host, to determine whether diarisation or a frequency-and-proximity-based method is more accurate at determining agreement, and whether this translates to feelings of engagement from the players. Experimental results show that our procedural system was more engaging to players, and was more accurate at detecting agreement, reaching an average accuracy of 0.44 compared to 0.28 for the diarised system.
△ Less
Submitted 6 November, 2023;
originally announced November 2023.
-
Multi-party Goal Tracking with LLMs: Comparing Pre-training, Fine-tuning, and Prompt Engineering
Authors:
Angus Addlesee,
Weronika Sieińska,
Nancie Gunson,
Daniel Hernández Garcia,
Christian Dondrup,
Oliver Lemon
Abstract:
This paper evaluates the extent to which current Large Language Models (LLMs) can capture task-oriented multi-party conversations (MPCs). We have recorded and transcribed 29 MPCs between patients, their companions, and a social robot in a hospital. We then annotated this corpus for multi-party goal-tracking and intent-slot recognition. People share goals, answer each other's goals, and provide oth…
▽ More
This paper evaluates the extent to which current Large Language Models (LLMs) can capture task-oriented multi-party conversations (MPCs). We have recorded and transcribed 29 MPCs between patients, their companions, and a social robot in a hospital. We then annotated this corpus for multi-party goal-tracking and intent-slot recognition. People share goals, answer each other's goals, and provide other people's goals in MPCs - none of which occur in dyadic interactions. To understand user goals in MPCs, we compared three methods in zero-shot and few-shot settings: we fine-tuned T5, created pre-training tasks to train DialogLM using LED, and employed prompt engineering techniques with GPT-3.5-turbo, to determine which approach can complete this novel task with limited data. GPT-3.5-turbo significantly outperformed the others in a few-shot setting. The `reasoning' style prompt, when given 7% of the corpus as example annotated conversations, was the best performing method. It correctly annotated 62.32% of the goal tracking MPCs, and 69.57% of the intent-slot recognition MPCs. A `story' style prompt increased model hallucination, which could be detrimental if deployed in safety-critical settings. We conclude that multi-party conversations still challenge state-of-the-art LLMs.
△ Less
Submitted 29 August, 2023;
originally announced August 2023.
-
Explainable Representations of the Social State: A Model for Social Human-Robot Interactions
Authors:
Daniel Hernández García,
Yanchao Yu,
Weronika Sieińska,
Jose L. Part,
Nancie Gunson,
Oliver Lemon,
Christian Dondrup
Abstract:
In this paper, we propose a minimum set of concepts and signals needed to track the social state during Human-Robot Interaction. We look into the problem of complex continuous interactions in a social context with multiple humans and robots, and discuss the creation of an explainable and tractable representation/model of their social interaction. We discuss these representations according to their…
▽ More
In this paper, we propose a minimum set of concepts and signals needed to track the social state during Human-Robot Interaction. We look into the problem of complex continuous interactions in a social context with multiple humans and robots, and discuss the creation of an explainable and tractable representation/model of their social interaction. We discuss these representations according to their representational and communicational properties, and organize them into four cognitive domains (scene-understanding, behaviour-profiling, mental-state, and dialogue-grounding).
△ Less
Submitted 9 October, 2020;
originally announced October 2020.
