Showing 1–9 of 9 results for author: Salam, H

Improving Personalisation in Valence and Arousal Prediction using Data Augmentation

Authors: Munachiso Nwadike, Jialin Li, Hanan Salam

Abstract: In the field of emotion recognition and Human-Machine Interaction (HMI), personalised approaches have exhibited their efficacy in capturing individual-specific characteristics and enhancing affective prediction accuracy. However, personalisation techniques often face the challenge of limited data for target individuals. This paper presents our work on an enhanced personalisation strategy, that lev… ▽ More In the field of emotion recognition and Human-Machine Interaction (HMI), personalised approaches have exhibited their efficacy in capturing individual-specific characteristics and enhancing affective prediction accuracy. However, personalisation techniques often face the challenge of limited data for target individuals. This paper presents our work on an enhanced personalisation strategy, that leverages data augmentation to develop tailored models for continuous valence and arousal prediction. Our proposed approach, Distance Weighting Augmentation (DWA), employs a weighting-based augmentation method that expands a target individual's dataset, leveraging distance metrics to identify similar samples at the segment-level. Experimental results on the MuSe-Personalisation 2023 Challenge dataset demonstrate that our method significantly improves the performance of features sets which have low baseline performance, on the test set. This improvement in poor-performing features comes without sacrificing performance on high-performing features. In particular, our method achieves a maximum combined testing CCC of 0.78, compared to the reported baseline score of 0.76 (reproduced at 0.72). It also achieved a peak arousal and valence scores of 0.81 and 0.76, compared to reproduced baseline scores of 0.76 and 0.67 respectively. Through this work, we make significant contributions to the advancement of personalised affective computing models, enhancing the practicality and adaptability of data-level personalisation in real world contexts. △ Less

Submitted 13 April, 2024; originally announced April 2024.

Accelerated Proximal Iterative re-Weighted $\ell_1$ Alternating Minimization for Image Deblurring

Authors: Tarmizi Adam, Alexander Malyshev, Mohd Fikree Hassan, Nur Syarafina Mohamed, Md Sah Hj Salam

Abstract: The quadratic penalty alternating minimization (AM) method is widely used for solving the convex $\ell_1$ total variation (TV) image deblurring problem. However, quadratic penalty AM for solving the nonconvex nonsmooth $\ell_p$, $0 < p < 1$ TV image deblurring problems is less studied. In this paper, we propose two algorithms, namely proximal iterative re-weighted $\ell_1$ AM (PIRL1-AM) and its ac… ▽ More The quadratic penalty alternating minimization (AM) method is widely used for solving the convex $\ell_1$ total variation (TV) image deblurring problem. However, quadratic penalty AM for solving the nonconvex nonsmooth $\ell_p$, $0 < p < 1$ TV image deblurring problems is less studied. In this paper, we propose two algorithms, namely proximal iterative re-weighted $\ell_1$ AM (PIRL1-AM) and its accelerated version, accelerated proximal iterative re-weighted $\ell_1$ AM (APIRL1-AM) for solving the nonconvex nonsmooth $\ell_p$ TV image deblurring problem. The proposed algorithms are derived from the proximal iterative re-weighted $\ell_1$ (IRL1) algorithm and the proximal gradient algorithm. Numerical results show that PIRL1-AM is effective in retaining sharp edges in image deblurring while APIRL1-AM can further provide convergence speed up in terms of the number of algorithm iterations and computational time. △ Less

Submitted 10 September, 2023; originally announced September 2023.

A Survey on Personalized Affective Computing in Human-Machine Interaction

Authors: Jialin Li, Alia Waleed, Hanan Salam

Abstract: In computing, the aim of personalization is to train a model that caters to a specific individual or group of people by optimizing one or more performance metrics and adhering to specific constraints. In this paper, we discuss the need for personalization in affective and personality computing (hereinafter referred to as affective computing). We present a survey of state-of-the-art approaches for… ▽ More In computing, the aim of personalization is to train a model that caters to a specific individual or group of people by optimizing one or more performance metrics and adhering to specific constraints. In this paper, we discuss the need for personalization in affective and personality computing (hereinafter referred to as affective computing). We present a survey of state-of-the-art approaches for personalization in affective computing. Our review spans training techniques and objectives towards the personalization of affective computing models. We group existing approaches into seven categories: (1) Target-specific Models, (2) Group-specific Models, (3) Weighting-based Approaches, (4) Fine-tuning Approaches, (5) Multitask Learning, (6) Generative-based Models, and (7) Feature Augmentation. Additionally, we provide a statistical meta-analysis of the surveyed literature, analyzing the prevalence of different affective computing tasks, interaction modes, interaction contexts, and the level of personalization among the surveyed works. Based on that, we provide a road-map for those who are interested in exploring this direction. △ Less

Submitted 1 April, 2023; originally announced April 2023.

Automatic Context-Driven Inference of Engagement in HMI: A Survey

Authors: Hanan Salam, Oya Celiktutan, Hatice Gunes, Mohamed Chetouani

Abstract: An integral part of seamless human-human communication is engagement, the process by which two or more participants establish, maintain, and end their perceived connection. Therefore, to develop successful human-centered human-machine interaction applications, automatic engagement inference is one of the tasks required to achieve engaging interactions between humans and machines, and to make machi… ▽ More An integral part of seamless human-human communication is engagement, the process by which two or more participants establish, maintain, and end their perceived connection. Therefore, to develop successful human-centered human-machine interaction applications, automatic engagement inference is one of the tasks required to achieve engaging interactions between humans and machines, and to make machines attuned to their users, hence enhancing user satisfaction and technology acceptance. Several factors contribute to engagement state inference, which include the interaction context and interactants' behaviours and identity. Indeed, engagement is a multi-faceted and multi-modal construct that requires high accuracy in the analysis and interpretation of contextual, verbal and non-verbal cues. Thus, the development of an automated and intelligent system that accomplishes this task has been proven to be challenging so far. This paper presents a comprehensive survey on previous work in engagement inference for human-machine interaction, entailing interdisciplinary definition, engagement components and factors, publicly available datasets, ground truth assessment, and most commonly used features and methods, serving as a guide for the development of future human-machine interaction interfaces with reliable context-aware engagement inference capability. An in-depth review across embodied and disembodied interaction modes, and an emphasis on the interaction context of which engagement perception modules are integrated sets apart the presented survey from existing surveys. △ Less

Submitted 30 September, 2022; originally announced September 2022.

AI-based Approach for Safety Signals Detection from Social Networks: Application to the Levothyrox Scandal in 2017 on Doctissimo Forum

Authors: Valentin Roche, Jean-Philippe Robert, Hanan Salam

Abstract: Social media can be an important source of information facilitating the detection of new safety signals in pharmacovigilance. Various approaches have investigated the analysis of social media data using AI such as NLP techniques for detecting adverse drug events. Existing approaches have focused on the extraction and identification of Adverse Drug Reactions, Drug-Drug Interactions and drug misuse.… ▽ More Social media can be an important source of information facilitating the detection of new safety signals in pharmacovigilance. Various approaches have investigated the analysis of social media data using AI such as NLP techniques for detecting adverse drug events. Existing approaches have focused on the extraction and identification of Adverse Drug Reactions, Drug-Drug Interactions and drug misuse. However, non of the works tackled the detection of potential safety signals by taking into account the evolution in time of relevant indicators. Moreover, despite the success of deep learning in various healthcare applications, it was not explored for this task. We propose an AI-based approach for the detection of potential pharmaceutical safety signals from patients' reviews that can be used as part of the pharmacovigilance surveillance process to flag the necessity of an in-depth pharmacovigilance investigation. We focus on the Levothyrox case in France which triggered huge attention from the media following the change of the medication formula, leading to an increase in the frequency of adverse drug reactions normally reported by patients. Our approach is two-fold. (1) We investigate various NLP-based indicators extracted from patients' reviews including words and n-grams frequency, semantic similarity, Adverse Drug Reactions mentions, and sentiment analysis. (2) We propose a deep learning architecture, named Word Cloud Convolutional Neural Network (WC-CNN) which trains a CNN on word clouds extracted from the patients comments. We study the effect of different time resolutions and different NLP pre-processing techniques on the model performance. Our results show that the proposed indicators could be used in the future to effectively detect new safety signals. The WC-CNN model trained on word clouds extracted at monthly resolution outperforms the others with an accuracy of 75%. △ Less

Submitted 1 February, 2022; originally announced March 2022.

Distinguishing Engagement Facets: An Essential Component for AI-based Interactive Healthcare

Authors: Hanan Salam

Abstract: Engagement in Human-Machine Interaction is the process by which entities participating in the interaction establish, maintain, and end their perceived connection. It is essential to monitor the engagement state of patients in various AI-based interactive healthcare paradigms. This includes medical conditions that alter social behavior such as Autism Spectrum Disorder (ASD) or Attention-Deficit/Hyp… ▽ More Engagement in Human-Machine Interaction is the process by which entities participating in the interaction establish, maintain, and end their perceived connection. It is essential to monitor the engagement state of patients in various AI-based interactive healthcare paradigms. This includes medical conditions that alter social behavior such as Autism Spectrum Disorder (ASD) or Attention-Deficit/Hyperactivity Disorder (ADHD). Engagement is a multi-faceted construct which is composed of behavioral, emotional, and mental components. Previous research has neglected this multi-faceted nature of engagement and focused on the detection of engagement level or binary engagement label. In this paper, a system is presented to distinguish these facets using contextual and relational features. This can facilitate further fine-grained analysis. Several machine learning classifiers including traditional and deep learning models are compared for this task. An F-Score of 0.74 was obtained on a balanced dataset of 22242 instances with neural network-based classification. The proposed framework shall serve as a baseline for further research on engagement facets recognition, and its integration is socially assistive robotic applications. △ Less

Submitted 2 March, 2023; v1 submitted 22 November, 2021; originally announced November 2021.

PAD: a graphical and numerical enhancement of structural coding to facilitate thematic analysis of a literature corpus

Authors: Etienne-Victor Depasquale, Humaira Abdul Salam, Franco Davoli

Abstract: We suggest an enhancement to structural coding through the use of (a) causally bound codes, (b) basic constructs of graph theory and (c) statistics. As is the norm with structural coding, the codes are collected into categories. The categories are represented by nodes (graph theory). The causality is illustrated through links (graph theory) between the nodes and the entire set of linked nodes is c… ▽ More We suggest an enhancement to structural coding through the use of (a) causally bound codes, (b) basic constructs of graph theory and (c) statistics. As is the norm with structural coding, the codes are collected into categories. The categories are represented by nodes (graph theory). The causality is illustrated through links (graph theory) between the nodes and the entire set of linked nodes is collected into a single directed acyclic graph. The number of occurrences of the nodes and the links provide the input required to analyze relative frequency of occurrence, as well as opening a scope for further statistical analysis. While our raw data was a corpus of literature from a specific discipline, this enhancement is accessible to any qualitative analysis that recognizes causality in its structural codes. △ Less

Submitted 26 July, 2021; originally announced July 2021.

AudVowelConsNet: A Phoneme-Level Based Deep CNN Architecture for Clinical Depression Diagnosis

Authors: Muhammad Muzammel, Hanan Salam, Yann Hoffmann, Mohamed Chetouani, Alice Othmani

Abstract: Depression is a common and serious mood disorder that negatively affects the patient's capacity of functioning normally in daily tasks. Speech is proven to be a vigorous tool in depression diagnosis. Research in psychiatry concentrated on performing fine-grained analysis on word-level speech components contributing to the manifestation of depression in speech and revealed significant variations at… ▽ More Depression is a common and serious mood disorder that negatively affects the patient's capacity of functioning normally in daily tasks. Speech is proven to be a vigorous tool in depression diagnosis. Research in psychiatry concentrated on performing fine-grained analysis on word-level speech components contributing to the manifestation of depression in speech and revealed significant variations at the phoneme-level in depressed speech. On the other hand, research in Machine Learning-based automatic recognition of depression from speech focused on the exploration of various acoustic features for the detection of depression and its severity level. Few have focused on incorporating phoneme-level speech components in automatic assessment systems. In this paper, we propose an Artificial Intelligence (AI) based application for clinical depression recognition and assessment from speech. We investigate the acoustic characteristics of phoneme units, specifically vowels and consonants for depression recognition via Deep Learning. We present and compare three spectrogram-based Deep Neural Network architectures, trained on phoneme consonant and vowel units and their fusion respectively. Our experiments show that the deep learned consonant-based acoustic characteristics lead to better recognition results than vowel-based ones. The fusion of vowel and consonant speech characteristics through a deep network significantly outperforms the single space networks as well as the state-of-art deep learning approaches on the DAIC-WOZ database. △ Less

Submitted 4 November, 2020; v1 submitted 30 October, 2020; originally announced October 2020.

Comments: 12 pages, 8 figures

Deep Multi-Facial Patches Aggregation Network For Facial Expression Recognition

Authors: Ahmed Rachid Hazourli, Amine Djeghri, Hanan Salam, Alice Othmani

Abstract: In this paper, we propose an approach for Facial Expressions Recognition (FER) based on a deep multi-facial patches aggregation network. Deep features are learned from facial patches using deep sub-networks and aggregated within one deep architecture for expression classification . Several problems may affect the performance of deep-learning based FER approaches, in particular, the small size of e… ▽ More In this paper, we propose an approach for Facial Expressions Recognition (FER) based on a deep multi-facial patches aggregation network. Deep features are learned from facial patches using deep sub-networks and aggregated within one deep architecture for expression classification . Several problems may affect the performance of deep-learning based FER approaches, in particular, the small size of existing FER datasets which might not be sufficient to train large deep learning networks. Moreover, it is extremely time-consuming to collect and annotate a large number of facial images. To account for this, we propose two data augmentation techniques for facial expression generation to expand FER labeled training datasets. We evaluate the proposed framework on three FER datasets. Results show that the proposed approach achieves state-of-art FER deep learning approaches performance when the model is trained and tested on images from the same dataset. Moreover, the proposed data augmentation techniques improve the expression recognition rate, and thus can be a solution for training deep learning FER models using small datasets. The accuracy degrades significantly when testing for dataset bias. △ Less

Submitted 20 February, 2020; originally announced February 2020.

Comments: This article arXiv:2002.09298 is an updated version of arXiv:1909.10305