-
Synthetic vs Human Emotional Faces: What Changes in Humans' Decoding Accuracy
Authors:
Terry Amorese,
Marialucia Cuciniello,
Alessandro Vinciarelli,
Gennaro Cordasco,
Anna Esposito
Abstract:
Considered the increasing use of assistive technologies in the shape of virtual agents, it is necessary to investigate those factors which characterize and affect the interaction between the user and the agent, among these emerges the way in which people interpret and decode synthetic emotions, i.e., emotional expressions conveyed by virtual agents. For these reasons, an article is proposed, which…
▽ More
Considered the increasing use of assistive technologies in the shape of virtual agents, it is necessary to investigate those factors which characterize and affect the interaction between the user and the agent, among these emerges the way in which people interpret and decode synthetic emotions, i.e., emotional expressions conveyed by virtual agents. For these reasons, an article is proposed, which involved 278 participants split in differently aged groups (young, middle-aged, and elders). Within each age group, some participants were administered a naturalistic decoding task, a recognition task of human emotional faces, while others were administered a synthetic decoding task, namely emotional expressions conveyed by virtual agents. Participants were required to label pictures of female and male humans or virtual agents of different ages (young, middle-aged, and old) displaying static expressions of disgust, anger, sadness, fear, happiness, surprise, and neutrality. Results showed that young participants showed better recognition performances (compared to older groups) of anger, sadness, and neutrality, while female participants showed better recognition performances(compared to males) ofsadness, fear, and neutrality; sadness and fear were better recognized when conveyed by real human faces, while happiness, surprise, and neutrality were better recognized when represented by virtual agents. Young faces were better decoded when expressing anger and surprise, middle-aged faces were better decoded when expressing sadness, fear, and happiness , while old faces were better decoded in the case of disgust; on average, female faces where better decoded compared to male ones.
△ Less
Submitted 16 April, 2024;
originally announced April 2024.
-
Exploring Emotion Expression Recognition in Older Adults Interacting with a Virtual Coach
Authors:
Cristina Palmero,
Mikel deVelasco,
Mohamed Amine Hmani,
Aymen Mtibaa,
Leila Ben Letaifa,
Pau Buch-Cardona,
Raquel Justo,
Terry Amorese,
Eduardo González-Fraile,
Begoña Fernández-Ruanova,
Jofre Tenorio-Laranga,
Anna Torp Johansen,
Micaela Rodrigues da Silva,
Liva Jenny Martinussen,
Maria Stylianou Korsnes,
Gennaro Cordasco,
Anna Esposito,
Mounim A. El-Yacoubi,
Dijana Petrovska-Delacrétaz,
M. Inés Torres,
Sergio Escalera
Abstract:
The EMPATHIC project aimed to design an emotionally expressive virtual coach capable of engaging healthy seniors to improve well-being and promote independent aging. One of the core aspects of the system is its human sensing capabilities, allowing for the perception of emotional states to provide a personalized experience. This paper outlines the development of the emotion expression recognition m…
▽ More
The EMPATHIC project aimed to design an emotionally expressive virtual coach capable of engaging healthy seniors to improve well-being and promote independent aging. One of the core aspects of the system is its human sensing capabilities, allowing for the perception of emotional states to provide a personalized experience. This paper outlines the development of the emotion expression recognition module of the virtual coach, encompassing data collection, annotation design, and a first methodological approach, all tailored to the project requirements. With the latter, we investigate the role of various modalities, individually and combined, for discrete emotion expression recognition in this context: speech from audio, and facial expressions, gaze, and head dynamics from video. The collected corpus includes users from Spain, France, and Norway, and was annotated separately for the audio and video channels with distinct emotional labels, allowing for a performance comparison across cultures and label types. Results confirm the informative power of the modalities studied for the emotional categories considered, with multimodal methods generally outperforming others (around 68% accuracy with audio labels and 72-74% with video labels). The findings are expected to contribute to the limited literature on emotion recognition applied to older adults in conversational human-machine interaction.
△ Less
Submitted 9 November, 2023;
originally announced November 2023.
-
Identifying synthetic voices qualities for conversational agents
Authors:
M. Cuciniello,
T. Amorese,
G. Cordasco,
S. Marrone,
F. Marulli,
F. Cavallo,
O. Gordeeva,
Z. Callejas Carrión,
A. Esposito
Abstract:
The present study aims to explore user acceptance and perceptions toward different quality levels of synthetical voices. To achieve this, four voices have been exploited considering two main factors: the quality of the voices (low vs high) and their gender (male and female). 186 volunteers were recruited and subsequently allocated into four groups of different ages respec-tively, adolescents, youn…
▽ More
The present study aims to explore user acceptance and perceptions toward different quality levels of synthetical voices. To achieve this, four voices have been exploited considering two main factors: the quality of the voices (low vs high) and their gender (male and female). 186 volunteers were recruited and subsequently allocated into four groups of different ages respec-tively, adolescents, young adults, middle-aged and seniors. After having randomly listened to each voice, participants were asked to fill the Virtual Agent Voice Acceptance Questionnaire (VAVAQ). Outcomes show that the two higher quality voices of Antonio and Giulia were more appreciated than the low-quality voices of Edoardo and Clara by the whole sample in terms of pragmatic, hedonic and attractiveness qualities attributed to the voices. Concerning preferences towards differently aged voices, it clearly appeared that they varied according to participants age' ranges examined. Furthermore, in terms of suitability to perform different tasks, participants considered Antonio and Giulia equally adapt for healthcare and front office jobs. Antonio was also judged to be significantly more qualified to accomplish protection and security tasks, while Edoardo was classified as the absolute least skilled in conducting household chores.
△ Less
Submitted 9 May, 2022;
originally announced May 2022.
-
Seniors' acceptance of virtual humanoid agents
Authors:
Anna Esposito,
Terry Amorese,
Marialucia Cuciniello,
Antonietta M. Esposito,
Alda Troncone,
Maria Ines Torres,
Stephan Schlögl,
Gennaro Cordasco
Abstract:
This paper reports on a study conducted as part of the EU EMPATHIC project, whose goal is to develop an empathic virtual coach capable of enhancing seniors' well-being, focusing on user requirements and expectations with respect to participants' age and technology experiences (i.e. participants' familiarity with technological devices such as smartphones, laptops, and tablets). The data shows that…
▽ More
This paper reports on a study conducted as part of the EU EMPATHIC project, whose goal is to develop an empathic virtual coach capable of enhancing seniors' well-being, focusing on user requirements and expectations with respect to participants' age and technology experiences (i.e. participants' familiarity with technological devices such as smartphones, laptops, and tablets). The data shows that seniors' favorite technological device is the smartphone, and this device was also the one that scored the highest in terms of easiness to use. We found statistically significant differences on the preferences expressed by seniors toward the gender of the agents. Seniors (independently from their gender) prefer to interact with female humanoid agents on both the pragmatic and hedonic dimensions of an interactive system and are more in favor to commit themselves in a long-lasting interaction with them. In addition, we found statistically significant effects of the seniors' technology savviness on the hedonic qualities of the proposed interactive systems. Seniors with technological experience felt less motivated and judged the proposed agents less captivating, exciting, and appealing.
△ Less
Submitted 2 May, 2021;
originally announced May 2021.
