-
Exploring a Multimodal Fusion-based Deep Learning Network for Detecting Facial Palsy
Authors:
Nicole Heng Yim Oo,
Min Hun Lee,
Jeong Hoon Lim
Abstract:
Algorithmic detection of facial palsy offers the potential to improve current practices, which usually involve labor-intensive and subjective assessment by clinicians. In this paper, we present a multimodal fusion-based deep learning model that utilizes unstructured data (i.e. an image frame with facial line segments) and structured data (i.e. features of facial expressions) to detect facial palsy…
▽ More
Algorithmic detection of facial palsy offers the potential to improve current practices, which usually involve labor-intensive and subjective assessment by clinicians. In this paper, we present a multimodal fusion-based deep learning model that utilizes unstructured data (i.e. an image frame with facial line segments) and structured data (i.e. features of facial expressions) to detect facial palsy. We then contribute to a study to analyze the effect of different data modalities and the benefits of a multimodal fusion-based approach using videos of 21 facial palsy patients. Our experimental results show that among various data modalities (i.e. unstructured data - RGB images and images of facial line segments and structured data - coordinates of facial landmarks and features of facial expressions), the feed-forward neural network using features of facial expression achieved the highest precision of 76.22 while the ResNet-based model using images of facial line segments achieved the highest recall of 83.47. When we leveraged both images of facial line segments and features of facial expressions, our multimodal fusion-based deep learning model slightly improved the precision score to 77.05 at the expense of a decrease in the recall score.
△ Less
Submitted 26 May, 2024;
originally announced May 2024.
-
A slice classification neural network for automated classification of axial PET/CT slices from a multi-centric lymphoma dataset
Authors:
Shadab Ahamed,
Yixi Xu,
Ingrid Bloise,
Joo H. O,
Carlos F. Uribe,
Rahul Dodhia,
Juan L. Ferres,
Arman Rahmim
Abstract:
Automated slice classification is clinically relevant since it can be incorporated into medical image segmentation workflows as a preprocessing step that would flag slices with a higher probability of containing tumors, thereby directing physicians attention to the important slices. In this work, we train a ResNet-18 network to classify axial slices of lymphoma PET/CT images (collected from two in…
▽ More
Automated slice classification is clinically relevant since it can be incorporated into medical image segmentation workflows as a preprocessing step that would flag slices with a higher probability of containing tumors, thereby directing physicians attention to the important slices. In this work, we train a ResNet-18 network to classify axial slices of lymphoma PET/CT images (collected from two institutions) depending on whether the slice intercepted a tumor (positive slice) in the 3D image or if the slice did not (negative slice). Various instances of the network were trained on 2D axial datasets created in different ways: (i) slice-level split and (ii) patient-level split; inputs of different types were used: (i) only PET slices and (ii) concatenated PET and CT slices; and different training strategies were employed: (i) center-aware (CAW) and (ii) center-agnostic (CAG). Model performances were compared using the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC), and various binary classification metrics. We observe and describe a performance overestimation in the case of slice-level split as compared to the patient-level split training. The model trained using patient-level split data with the network input containing only PET slices in the CAG training regime was the best performing/generalizing model on a majority of metrics. Our models were additionally more closely compared using the sensitivity metric on the positive slices from their respective test sets.
△ Less
Submitted 11 March, 2024;
originally announced March 2024.
-
SeSame: Simple, Easy 3D Object Detection with Point-Wise Semantics
Authors:
Hayeon O,
Chanuk Yang,
Kunsoo Huh
Abstract:
In autonomous driving, 3D object detection provides more precise information for downstream tasks, including path planning and motion estimation, compared to 2D object detection. In this paper, we propose SeSame: a method aimed at enhancing semantic information in existing LiDAR-only based 3D object detection. This addresses the limitation of existing 3D detectors, which primarily focus on object…
▽ More
In autonomous driving, 3D object detection provides more precise information for downstream tasks, including path planning and motion estimation, compared to 2D object detection. In this paper, we propose SeSame: a method aimed at enhancing semantic information in existing LiDAR-only based 3D object detection. This addresses the limitation of existing 3D detectors, which primarily focus on object presence and classification, thus lacking in capturing relationships between elemental units that constitute the data, akin to semantic segmentation. Experiments demonstrate the effectiveness of our method with performance improvements on the KITTI object detection benchmark. Our code is available at https://github.com/HAMA-DL-dev/SeSame
△ Less
Submitted 8 July, 2024; v1 submitted 11 March, 2024;
originally announced March 2024.
-
Comprehensive Evaluation and Insights into the Use of Deep Neural Networks to Detect and Quantify Lymphoma Lesions in PET/CT Images
Authors:
Shadab Ahamed,
Yixi Xu,
Claire Gowdy,
Joo H. O,
Ingrid Bloise,
Don Wilson,
Patrick Martineau,
François Bénard,
Fereshteh Yousefirizi,
Rahul Dodhia,
Juan M. Lavista,
William B. Weeks,
Carlos F. Uribe,
Arman Rahmim
Abstract:
This study performs comprehensive evaluation of four neural network architectures (UNet, SegResNet, DynUNet, and SwinUNETR) for lymphoma lesion segmentation from PET/CT images. These networks were trained, validated, and tested on a diverse, multi-institutional dataset of 611 cases. Internal testing (88 cases; total metabolic tumor volume (TMTV) range [0.52, 2300] ml) showed SegResNet as the top p…
▽ More
This study performs comprehensive evaluation of four neural network architectures (UNet, SegResNet, DynUNet, and SwinUNETR) for lymphoma lesion segmentation from PET/CT images. These networks were trained, validated, and tested on a diverse, multi-institutional dataset of 611 cases. Internal testing (88 cases; total metabolic tumor volume (TMTV) range [0.52, 2300] ml) showed SegResNet as the top performer with a median Dice similarity coefficient (DSC) of 0.76 and median false positive volume (FPV) of 4.55 ml; all networks had a median false negative volume (FNV) of 0 ml. On the unseen external test set (145 cases with TMTV range: [0.10, 2480] ml), SegResNet achieved the best median DSC of 0.68 and FPV of 21.46 ml, while UNet had the best FNV of 0.41 ml. We assessed reproducibility of six lesion measures, calculated their prediction errors, and examined DSC performance in relation to these lesion measures, offering insights into segmentation accuracy and clinical relevance. Additionally, we introduced three lesion detection criteria, addressing the clinical need for identifying lesions, counting them, and segmenting based on metabolic characteristics. We also performed expert intra-observer variability analysis revealing the challenges in segmenting ``easy'' vs. ``hard'' cases, to assist in the development of more resilient segmentation algorithms. Finally, we performed inter-observer agreement assessment underscoring the importance of a standardized ground truth segmentation protocol involving multiple expert annotators. Code is available at: https://github.com/microsoft/lymphoma-segmentation-dnn
△ Less
Submitted 16 November, 2023;
originally announced November 2023.
-
An examination of applicability of face recognition sensors in public facilities
Authors:
Takuji Takemoto,
Takashi Ota,
Hiroko Oe
Abstract:
This study aimed to explore the usability and applicability of face recognition sensors in public spaces to collect customer footfall data, which could then be analysed and evaluated for facility design and planning. Nine OMRON sensors were provided for the project and installed at five locations in a public facility for three months. The project was carried out by a local consortium with the coop…
▽ More
This study aimed to explore the usability and applicability of face recognition sensors in public spaces to collect customer footfall data, which could then be analysed and evaluated for facility design and planning. Nine OMRON sensors were provided for the project and installed at five locations in a public facility for three months. The project was carried out by a local consortium with the cooperation of local technology-based Small Medium-sized Enterprises (SMEs), business organisations, and a local university. Collected data were analysed to develop a report with diagrams, and reveal issues and potential for practical application in the future.
△ Less
Submitted 19 May, 2020;
originally announced May 2020.
-
Discussion of digital gaming's impact on players' well-being during the COVID-19 lockdown
Authors:
Hiroko Oe
Abstract:
This research discusses how to utilise digital gaming to support the well-being of its users and sustain their physical and mental health during the COVID-19 lockdown in which people's activities are limited. The published academic literature that is written in English and available for access on online databases was reviewed to develop key take-aways and a framework for discussing how to enhance…
▽ More
This research discusses how to utilise digital gaming to support the well-being of its users and sustain their physical and mental health during the COVID-19 lockdown in which people's activities are limited. The published academic literature that is written in English and available for access on online databases was reviewed to develop key take-aways and a framework for discussing how to enhance people's well-being in the COVID-19 lockdown. Interaction with other players in virtual communities has been found to have a positive influence on the mental health of those suffering from a lack of societal connection. A framework for further research has also been developed that focuses on the critical situation of the COVID-19 lockdown,as this is an urgent topic with a huge impact on our health.Some gaming service providers have been proactive in redesigning game programming to be suitable for the lockdown situation, and this enables players to enjoy physical activities even at home.
△ Less
Submitted 1 May, 2020;
originally announced May 2020.
-
An Efficient Bilinear Pairing-Free Certificateless Two-Party Authenticated Key Agreement Protocol in the eCK Model
Authors:
Yong-** Kim,
Yong-Min Kim,
Yong-** Choe,
Hyong-Chol O
Abstract:
Recent study on certificateless authenticated key agreement focuses on bilinear pairing-free certificateless authenticated key agreement protocol. Yet it has got limitations in the aspect of computational amount. So it is important to reduce the number of the scalar multiplication over elliptic curve group in bilinear pairing-free protocols. This paper proposed a new bilinear pairing-free certific…
▽ More
Recent study on certificateless authenticated key agreement focuses on bilinear pairing-free certificateless authenticated key agreement protocol. Yet it has got limitations in the aspect of computational amount. So it is important to reduce the number of the scalar multiplication over elliptic curve group in bilinear pairing-free protocols. This paper proposed a new bilinear pairing-free certificateless two-party authenticated key agreement protocol, providing more efficiency among related work and proof under the random oracle model.
△ Less
Submitted 5 July, 2013; v1 submitted 1 April, 2013;
originally announced April 2013.
-
Equiaffine Structure and Conjugate Ricci-symmetry of a Statistical Manifold
Authors:
Chol-Rim Min,
Won-Hak Ri,
Hyong-Chol O
Abstract:
A condition for a statistical manifold to have an equiaffine structure is studied. The facts that dual flatness and conjugate symmetry of a statistical manifold are sufficient conditions for a statistical manifold to have an equiaffine structure were obtained in [2] and [3]. In this paper, a fact that a statistical manifold, which is conjugate Ricci-symmetric, has an equiaffine structure is given,…
▽ More
A condition for a statistical manifold to have an equiaffine structure is studied. The facts that dual flatness and conjugate symmetry of a statistical manifold are sufficient conditions for a statistical manifold to have an equiaffine structure were obtained in [2] and [3]. In this paper, a fact that a statistical manifold, which is conjugate Ricci-symmetric, has an equiaffine structure is given, where conjugate Ricci-symmetry is weaker condition than conjugate symmetry. A condition for conjugate symmetry and conjugate Ricci-symmetry to coincide is also given.
△ Less
Submitted 13 February, 2013;
originally announced February 2013.
