-
Exploring a Multimodal Fusion-based Deep Learning Network for Detecting Facial Palsy
Authors:
Nicole Heng Yim Oo,
Min Hun Lee,
Jeong Hoon Lim
Abstract:
Algorithmic detection of facial palsy offers the potential to improve current practices, which usually involve labor-intensive and subjective assessment by clinicians. In this paper, we present a multimodal fusion-based deep learning model that utilizes unstructured data (i.e. an image frame with facial line segments) and structured data (i.e. features of facial expressions) to detect facial palsy…
▽ More
Algorithmic detection of facial palsy offers the potential to improve current practices, which usually involve labor-intensive and subjective assessment by clinicians. In this paper, we present a multimodal fusion-based deep learning model that utilizes unstructured data (i.e. an image frame with facial line segments) and structured data (i.e. features of facial expressions) to detect facial palsy. We then contribute to a study to analyze the effect of different data modalities and the benefits of a multimodal fusion-based approach using videos of 21 facial palsy patients. Our experimental results show that among various data modalities (i.e. unstructured data - RGB images and images of facial line segments and structured data - coordinates of facial landmarks and features of facial expressions), the feed-forward neural network using features of facial expression achieved the highest precision of 76.22 while the ResNet-based model using images of facial line segments achieved the highest recall of 83.47. When we leveraged both images of facial line segments and features of facial expressions, our multimodal fusion-based deep learning model slightly improved the precision score to 77.05 at the expense of a decrease in the recall score.
△ Less
Submitted 26 May, 2024;
originally announced May 2024.
-
Google Crowdsourced Speech Corpora and Related Open-Source Resources for Low-Resource Languages and Dialects: An Overview
Authors:
Alena Butryna,
Shan-Hui Cathy Chu,
Isin Demirsahin,
Alexander Gutkin,
Linne Ha,
Fei He,
Martin Jansche,
Cibu Johny,
Anna Katanova,
Oddur Kjartansson,
Chenfang Li,
Tatiana Merkulova,
Yin May Oo,
Knot Pipatsrisawat,
Clara Rivera,
Supheakmungkol Sarin,
Pasindu de Silva,
Keshan Sodimana,
Richard Sproat,
Theeraphol Wattanavekin,
Jaka Aris Eko Wibawa
Abstract:
This paper presents an overview of a program designed to address the growing need for develo** freely available speech resources for under-represented languages. At present we have released 38 datasets for building text-to-speech and automatic speech recognition applications for languages and dialects of South and Southeast Asia, Africa, Europe and South America. The paper describes the methodol…
▽ More
This paper presents an overview of a program designed to address the growing need for develo** freely available speech resources for under-represented languages. At present we have released 38 datasets for building text-to-speech and automatic speech recognition applications for languages and dialects of South and Southeast Asia, Africa, Europe and South America. The paper describes the methodology used for develo** such corpora and presents some of our findings that could benefit under-represented language communities.
△ Less
Submitted 13 October, 2020;
originally announced October 2020.
-
On Constructing the Value Function for Optimal Trajectory Problem and its Application to Image Processing
Authors:
Myong-Song Ho,
Gwang-Hui Ju,
Yong-Bom O,
Gwang-Ho Jong
Abstract:
We proposed an algorithm for solving Hamilton-Jacobi equation associated to an optimal trajectory problem for a vehicle moving inside the pre-specified domain with the speed depending upon the direction of the motion and current position of the vehicle. The dynamics of the vehicle is defined by an ordinary differential equation, the right hand of which is given by product of control(a time depende…
▽ More
We proposed an algorithm for solving Hamilton-Jacobi equation associated to an optimal trajectory problem for a vehicle moving inside the pre-specified domain with the speed depending upon the direction of the motion and current position of the vehicle. The dynamics of the vehicle is defined by an ordinary differential equation, the right hand of which is given by product of control(a time dependent fuction) and a function dependent on trajectory and control. At some unspecified terminal time, the vehicle reaches the boundary of the pre-specified domain and incurs a terminal cost. We also associate the traveling cost with a type of integral to the trajectory followed by vehicle. We are interested in a numerical method for finding a trajectory that minimizes the sum of the traveling cost and terminal cost. We developed an algorithm solving the value function for general trajectory optimization problem. Our algorithm is closely related to the Tsitsiklis's Fast Marching Method and J. A. Sethian's OUM and SLF-LLL[1-4] and is a generalization of them. On the basis of these results, We applied our algorithm to the image processing such as fingerprint verification.
△ Less
Submitted 27 March, 2013; v1 submitted 20 March, 2013;
originally announced March 2013.
