-
Exploring Emotion Expression Recognition in Older Adults Interacting with a Virtual Coach
Authors:
Cristina Palmero,
Mikel deVelasco,
Mohamed Amine Hmani,
Aymen Mtibaa,
Leila Ben Letaifa,
Pau Buch-Cardona,
Raquel Justo,
Terry Amorese,
Eduardo González-Fraile,
Begoña Fernández-Ruanova,
Jofre Tenorio-Laranga,
Anna Torp Johansen,
Micaela Rodrigues da Silva,
Liva Jenny Martinussen,
Maria Stylianou Korsnes,
Gennaro Cordasco,
Anna Esposito,
Mounim A. El-Yacoubi,
Dijana Petrovska-Delacrétaz,
M. Inés Torres,
Sergio Escalera
Abstract:
The EMPATHIC project aimed to design an emotionally expressive virtual coach capable of engaging healthy seniors to improve well-being and promote independent aging. One of the core aspects of the system is its human sensing capabilities, allowing for the perception of emotional states to provide a personalized experience. This paper outlines the development of the emotion expression recognition m…
▽ More
The EMPATHIC project aimed to design an emotionally expressive virtual coach capable of engaging healthy seniors to improve well-being and promote independent aging. One of the core aspects of the system is its human sensing capabilities, allowing for the perception of emotional states to provide a personalized experience. This paper outlines the development of the emotion expression recognition module of the virtual coach, encompassing data collection, annotation design, and a first methodological approach, all tailored to the project requirements. With the latter, we investigate the role of various modalities, individually and combined, for discrete emotion expression recognition in this context: speech from audio, and facial expressions, gaze, and head dynamics from video. The collected corpus includes users from Spain, France, and Norway, and was annotated separately for the audio and video channels with distinct emotional labels, allowing for a performance comparison across cultures and label types. Results confirm the informative power of the modalities studied for the emotional categories considered, with multimodal methods generally outperforming others (around 68% accuracy with audio labels and 72-74% with video labels). The findings are expected to contribute to the limited literature on emotion recognition applied to older adults in conversational human-machine interaction.
△ Less
Submitted 9 November, 2023;
originally announced November 2023.
-
The EMPATHIC Project: Mid-term Achievements
Authors:
M. I. Torres,
J. M. Olaso,
C. Montenegro,
R. Santana,
A. Vázquez,
R. Justo,
J. A. Lozano,
S. Schlögl,
G. Chollet,
N. Dugan,
M. Irvine,
N. Glackin,
C. Pickard,
A. Esposito,
G. Cordasco,
A. Troncone,
D. Petrovska-Delacretaz,
A. Mtibaa,
M. A. Hmani,
M. S. Korsnes,
L. J. Martinussen,
S. Escalera,
C. Palmero Cantariño,
O. Deroo,
O. Gordeeva
, et al. (4 additional authors not shown)
Abstract:
The goal of active aging is to promote changes in the elderly community so as to maintain an active, independent and socially-engaged lifestyle. Technological advancements currently provide the necessary tools to foster and monitor such processes. This paper reports on mid-term achievements of the European H2020 EMPATHIC project, which aims to research, innovate, explore and validate new interacti…
▽ More
The goal of active aging is to promote changes in the elderly community so as to maintain an active, independent and socially-engaged lifestyle. Technological advancements currently provide the necessary tools to foster and monitor such processes. This paper reports on mid-term achievements of the European H2020 EMPATHIC project, which aims to research, innovate, explore and validate new interaction paradigms and platforms for future generations of personalized virtual coaches to assist the elderly and their carers to reach the active aging goal, in the vicinity of their home. The project focuses on evidence-based, user-validated research and integration of intelligent technology, and context sensing methods through automatic voice, eye and facial analysis, integrated with visual and spoken dialogue system capabilities. In this paper, we describe the current status of the system, with a special emphasis on its components and their integration, the creation of a Wizard of Oz platform, and findings gained from user interaction studies conducted throughout the first 18 months of the project.
△ Less
Submitted 5 May, 2021;
originally announced May 2021.
-
The EMPATHIC Project: Building an Expressive, Advanced Virtual Coach to Improve Independent Healthy-Life-Years of the Elderly
Authors:
Luisa Brinkschulte,
Natascha Mariacher,
Stephan Schlögl,
María Inés Torres,
Raquel Justo,
Javier Mikel Olaso,
Anna Esposito,
Gennaro Cordasco,
Gérard Chollet,
Cornelius Glackin,
Colin Pickard,
Dijana Petrovska-Delacretaz,
Mohamed Amine Hmani,
Ayment Mtibaa,
Anaïs Fernandez,
Daria Kyslitska,
Begoña Fernandez-Ruanova,
Jofre Tenorio-Laranga,
Mari Aksnes,
Maria Stylianou Korsnes,
Miriam Reiner,
Fredrik Lindner,
Olivier Deroo,
Olga Gordeeva
Abstract:
This paper outlines the EMPATHIC Research & Innovation project, which aims to research, innovate, explore and validate new interaction paradigms and plat-forms for future generations of Personalized Virtual Coaches to assist elderly people living independently at and around their home. Innovative multimodal face analytics, adaptive spoken dialogue systems, and natural language inter-faces are part…
▽ More
This paper outlines the EMPATHIC Research & Innovation project, which aims to research, innovate, explore and validate new interaction paradigms and plat-forms for future generations of Personalized Virtual Coaches to assist elderly people living independently at and around their home. Innovative multimodal face analytics, adaptive spoken dialogue systems, and natural language inter-faces are part of what the project investigates and innovates, aiming to help dependent aging persons and their carers. It will uses remote, non-intrusive technologies to extract physiological markers of emotional states and adapt respective coach responses. In doing so, it aims to develop causal models for emotionally believable coach-user interactions, which shall engage elders and thus keep off loneliness, sustain health, enhance quality of life, and simplify access to future telecare services. Through measurable end-user validations performed in Spain, Norway and France (and complementary user evaluations in Italy), the proposed methods and solutions will have to demonstrate useful-ness, reliability, flexibility and robustness.
△ Less
Submitted 28 April, 2021;
originally announced April 2021.
-
X-vectors: New Quantitative Biomarkers for Early Parkinson's Disease Detection from Speech
Authors:
Laetitia Jeancolas,
Dijana Petrovska-Delacrétaz,
Graziella Mangone,
Badr-Eddine Benkelfat,
Jean-Christophe Corvol,
Marie Vidailhet,
Stéphane Lehéricy,
Habib Benali
Abstract:
Many articles have used voice analysis to detect Parkinson's disease (PD), but few have focused on the early stages of the disease and the gender effect. In this article, we have adapted the latest speaker recognition system, called x-vectors, in order to detect an early stage of PD from voice analysis. X-vectors are embeddings extracted from a deep neural network, which provide robust speaker rep…
▽ More
Many articles have used voice analysis to detect Parkinson's disease (PD), but few have focused on the early stages of the disease and the gender effect. In this article, we have adapted the latest speaker recognition system, called x-vectors, in order to detect an early stage of PD from voice analysis. X-vectors are embeddings extracted from a deep neural network, which provide robust speaker representations and improve speaker recognition when large amounts of training data are used. Our goal was to assess whether, in the context of early PD detection, this technique would outperform the more standard classifier MFCC-GMM (Mel-Frequency Cepstral Coefficients - Gaussian Mixture Model) and, if so, under which conditions. We recorded 221 French speakers (including recently diagnosed PD subjects and healthy controls) with a high-quality microphone and with their own telephone. Men and women were analyzed separately in order to have more precise models and to assess a possible gender effect. Several experimental and methodological aspects were tested in order to analyze their impacts on classification performance. We assessed the impact of audio segment duration, data augmentation, type of dataset used for the neural network training, kind of speech tasks, and back-end analyses. X-vectors technique provided better classification performances than MFCC-GMM for text-independent tasks, and seemed to be particularly suited for the early detection of PD in women (7 to 15% improvement). This result was observed for both recording types (high-quality microphone and telephone).
△ Less
Submitted 7 July, 2020;
originally announced July 2020.
-
3D Face Pose and Animation Tracking via Eigen-Decomposition based Bayesian Approach
Authors:
Ngoc-Trung Tran,
Fakhr-Eddine Ababsa,
Maurice Charbit,
Jacques Feldmar,
Dijana Petrovska-Delacrétaz,
Gérard Chollet
Abstract:
This paper presents a new method to track both the face pose and the face animation with a monocular camera. The approach is based on the 3D face model CANDIDE and on the SIFT (Scale Invariant Feature Transform) descriptors, extracted around a few given landmarks (26 selected vertices of CANDIDE model) with a Bayesian approach. The training phase is performed on a synthetic database generated from…
▽ More
This paper presents a new method to track both the face pose and the face animation with a monocular camera. The approach is based on the 3D face model CANDIDE and on the SIFT (Scale Invariant Feature Transform) descriptors, extracted around a few given landmarks (26 selected vertices of CANDIDE model) with a Bayesian approach. The training phase is performed on a synthetic database generated from the first video frame. At each current frame, the face pose and animation parameters are estimated via a Bayesian approach, with a Gaussian prior and a Gaussian likelihood function whose the mean and the covariance matrix eigenvalues are updated from the previous frame using eigen decomposition. Numerical results on pose estimation and landmark locations are reported using the Boston University Face Tracking (BUFT) database and Talking Face video. They show that our approach, compared to six other published algorithms, provides a very good compromise and presents a promising perspective due to the good results in terms of landmark localization.
△ Less
Submitted 28 August, 2019;
originally announced August 2019.
