-
CLARE: Cognitive Load Assessment in REaltime with Multimodal Data
Authors:
Anubhav Bhatti,
Prithila Angkan,
Behnam Behinaein,
Zunayed Mahmud,
Dirk Rodenburg,
Heather Braund,
P. James Mclellan,
Aaron Ruberto,
Geoffery Harrison,
Daryl Wilson,
Adam Szulewski,
Dan Howes,
Ali Etemad,
Paul Hungler
Abstract:
We present a novel multimodal dataset for Cognitive Load Assessment in REaltime (CLARE). The dataset contains physiological and gaze data from 24 participants with self-reported cognitive load scores as ground-truth labels. The dataset consists of four modalities, namely, Electrocardiography (ECG), Electrodermal Activity (EDA), Electroencephalogram (EEG), and Gaze tracking. To map diverse levels o…
▽ More
We present a novel multimodal dataset for Cognitive Load Assessment in REaltime (CLARE). The dataset contains physiological and gaze data from 24 participants with self-reported cognitive load scores as ground-truth labels. The dataset consists of four modalities, namely, Electrocardiography (ECG), Electrodermal Activity (EDA), Electroencephalogram (EEG), and Gaze tracking. To map diverse levels of mental load on participants during experiments, each participant completed four nine-minutes sessions on a computer-based operator performance and mental workload task (the MATB-II software) with varying levels of complexity in one minute segments. During the experiment, participants reported their cognitive load every 10 seconds. For the dataset, we also provide benchmark binary classification results with machine learning and deep learning models on two different evaluation schemes, namely, 10-fold and leave-one-subject-out (LOSO) cross-validation. Benchmark results show that for 10-fold evaluation, the convolutional neural network (CNN) based deep learning model achieves the best classification performance with ECG, EDA, and Gaze. In contrast, for LOSO, the best performance is achieved by the deep learning model with ECG, EDA, and EEG.
△ Less
Submitted 25 April, 2024;
originally announced April 2024.
-
Explainable Deepfake Video Detection using Convolutional Neural Network and CapsuleNet
Authors:
Gazi Hasin Ishrak,
Zalish Mahmud,
MD. Zami Al Zunaed Farabe,
Tahera Khanom Tinni,
Tanzim Reza,
Mohammad Zavid Parvez
Abstract:
Deepfake technology, derived from deep learning, seamlessly inserts individuals into digital media, irrespective of their actual participation. Its foundation lies in machine learning and Artificial Intelligence (AI). Initially, deepfakes served research, industry, and entertainment. While the concept has existed for decades, recent advancements render deepfakes nearly indistinguishable from reali…
▽ More
Deepfake technology, derived from deep learning, seamlessly inserts individuals into digital media, irrespective of their actual participation. Its foundation lies in machine learning and Artificial Intelligence (AI). Initially, deepfakes served research, industry, and entertainment. While the concept has existed for decades, recent advancements render deepfakes nearly indistinguishable from reality. Accessibility has soared, empowering even novices to create convincing deepfakes. However, this accessibility raises security concerns.The primary deepfake creation algorithm, GAN (Generative Adversarial Network), employs machine learning to craft realistic images or videos. Our objective is to utilize CNN (Convolutional Neural Network) and CapsuleNet with LSTM to differentiate between deepfake-generated frames and originals. Furthermore, we aim to elucidate our model's decision-making process through Explainable AI, fostering transparent human-AI relationships and offering practical examples for real-life scenarios.
△ Less
Submitted 19 April, 2024;
originally announced April 2024.
-
Multimodal Brain-Computer Interface for In-Vehicle Driver Cognitive Load Measurement: Dataset and Baselines
Authors:
Prithila Angkan,
Behnam Behinaein,
Zunayed Mahmud,
Anubhav Bhatti,
Dirk Rodenburg,
Paul Hungler,
Ali Etemad
Abstract:
Through this paper, we introduce a novel driver cognitive load assessment dataset, CL-Drive, which contains Electroencephalogram (EEG) signals along with other physiological signals such as Electrocardiography (ECG) and Electrodermal Activity (EDA) as well as eye tracking data. The data was collected from 21 subjects while driving in an immersive vehicle simulator, in various driving conditions, t…
▽ More
Through this paper, we introduce a novel driver cognitive load assessment dataset, CL-Drive, which contains Electroencephalogram (EEG) signals along with other physiological signals such as Electrocardiography (ECG) and Electrodermal Activity (EDA) as well as eye tracking data. The data was collected from 21 subjects while driving in an immersive vehicle simulator, in various driving conditions, to induce different levels of cognitive load in the subjects. The tasks consisted of 9 complexity levels for 3 minutes each. Each driver reported their subjective cognitive load every 10 seconds throughout the experiment. The dataset contains the subjective cognitive load recorded as ground truth. In this paper, we also provide benchmark classification results for different machine learning and deep learning models for both binary and ternary label distributions. We followed 2 evaluation criteria namely 10-fold and leave-one-subject-out (LOSO). We have trained our models on both hand-crafted features as well as on raw data.
△ Less
Submitted 20 December, 2023; v1 submitted 9 April, 2023;
originally announced April 2023.
-
Multistream Gaze Estimation with Anatomical Eye Region Isolation by Synthetic to Real Transfer Learning
Authors:
Zunayed Mahmud,
Paul Hungler,
Ali Etemad
Abstract:
We propose a novel neural pipeline, MSGazeNet, that learns gaze representations by taking advantage of the eye anatomy information through a multistream framework. Our proposed solution comprises two components, first a network for isolating anatomical eye regions, and a second network for multistream gaze estimation. The eye region isolation is performed with a U-Net style network which we train…
▽ More
We propose a novel neural pipeline, MSGazeNet, that learns gaze representations by taking advantage of the eye anatomy information through a multistream framework. Our proposed solution comprises two components, first a network for isolating anatomical eye regions, and a second network for multistream gaze estimation. The eye region isolation is performed with a U-Net style network which we train using a synthetic dataset that contains eye region masks for the visible eyeball and the iris region. The synthetic dataset used in this stage is procured using the UnityEyes simulator, and consists of 80,000 eye images. Successive to training, the eye region isolation network is then transferred to the real domain for generating masks for the real-world eye images. In order to successfully make the transfer, we exploit domain randomization in the training process, which allows for the synthetic images to benefit from a larger variance with the help of augmentations that resemble artifacts. The generated eye region masks along with the raw eye images are then used together as a multistream input to our gaze estimation network, which consists of wide residual blocks. The output embeddings from these encoders are fused in the channel dimension before feeding into the gaze regression layers. We evaluate our framework on three gaze estimation datasets and achieve strong performances. Our method surpasses the state-of-the-art by 7.57% and 1.85% on two datasets, and obtains competitive results on the other. We also study the robustness of our method with respect to the noise in the data and demonstrate that our model is less sensitive to noisy data. Lastly, we perform a variety of experiments including ablation studies to evaluate the contribution of different components and design choices in our solution.
△ Less
Submitted 12 February, 2024; v1 submitted 18 June, 2022;
originally announced June 2022.
-
Gaze Estimation with Eye Region Segmentation and Self-Supervised Multistream Learning
Authors:
Zunayed Mahmud,
Paul Hungler,
Ali Etemad
Abstract:
We present a novel multistream network that learns robust eye representations for gaze estimation. We first create a synthetic dataset containing eye region masks detailing the visible eyeball and iris using a simulator. We then perform eye region segmentation with a U-Net type model which we later use to generate eye region masks for real-world eye images. Next, we pretrain an eye image encoder i…
▽ More
We present a novel multistream network that learns robust eye representations for gaze estimation. We first create a synthetic dataset containing eye region masks detailing the visible eyeball and iris using a simulator. We then perform eye region segmentation with a U-Net type model which we later use to generate eye region masks for real-world eye images. Next, we pretrain an eye image encoder in the real domain with self-supervised contrastive learning to learn generalized eye representations. Finally, this pretrained eye encoder, along with two additional encoders for visible eyeball region and iris, are used in parallel in our multistream framework to extract salient features for gaze estimation from real-world images. We demonstrate the performance of our method on the EYEDIAP dataset in two different evaluation settings and achieve state-of-the-art results, outperforming all the existing benchmarks on this dataset. We also conduct additional experiments to validate the robustness of our self-supervised network with respect to different amounts of labeled data used for training.
△ Less
Submitted 14 December, 2021;
originally announced December 2021.
-
Cyberbullying Detection Using Deep Neural Network from Social Media Comments in Bangla Language
Authors:
Md Faisal Ahmed,
Zalish Mahmud,
Zarin Tasnim Biash,
Ahmed Ann Noor Ryen,
Arman Hossain,
Faisal Bin Ashraf
Abstract:
Cyberbullying or Online harassment detection on social media for various major languages is currently being given a good amount of focus by researchers worldwide. Being the seventh most speaking language in the world and increasing usage of online platform among the Bengali speaking people urge to find effective detection technique to handle the online harassment. In this paper, we have proposed b…
▽ More
Cyberbullying or Online harassment detection on social media for various major languages is currently being given a good amount of focus by researchers worldwide. Being the seventh most speaking language in the world and increasing usage of online platform among the Bengali speaking people urge to find effective detection technique to handle the online harassment. In this paper, we have proposed binary and multiclass classification model using hybrid neural network for bully expression detection in Bengali language. We have used 44,001 users comments from popular public Facebook pages, which fall into five classes - Non-bully, Sexual, Threat, Troll and Religious. We have examined the performance of our proposed models from different perspective. Our binary classification model gives 87.91% accuracy, whereas introducing ensemble technique after neural network for multiclass classification, we got 85% accuracy.
△ Less
Submitted 8 June, 2021;
originally announced June 2021.
-
Bangla Text Dataset and Exploratory Analysis for Online Harassment Detection
Authors:
Md Faisal Ahmed,
Zalish Mahmud,
Zarin Tasnim Biash,
Ahmed Ann Noor Ryen,
Arman Hossain,
Faisal Bin Ashraf
Abstract:
Being the seventh most spoken language in the world, the use of the Bangla language online has increased in recent times. Hence, it has become very important to analyze Bangla text data to maintain a safe and harassment-free online place. The data that has been made accessible in this article has been gathered and marked from the comments of people in public posts by celebrities, government offici…
▽ More
Being the seventh most spoken language in the world, the use of the Bangla language online has increased in recent times. Hence, it has become very important to analyze Bangla text data to maintain a safe and harassment-free online place. The data that has been made accessible in this article has been gathered and marked from the comments of people in public posts by celebrities, government officials, athletes on Facebook. The total amount of collected comments is 44001. The dataset is compiled with the aim of develo** the ability of machines to differentiate whether a comment is a bully expression or not with the help of Natural Language Processing and to what extent it is improper if it is an inappropriate comment. The comments are labeled with different categories of harassment. Exploratory analysis from different perspectives is also included in this paper to have a detailed overview. Due to the scarcity of data collection of categorized Bengali language comments, this dataset can have a significant role for research in detecting bully words, identifying inappropriate comments, detecting different categories of Bengali bullies, etc. The dataset is publicly available at https://data.mendeley.com/datasets/9xjx8twk8p.
△ Less
Submitted 4 February, 2021;
originally announced February 2021.
-
Sequential Operations in LogicWeb
Authors:
Daeseong Kang,
Keehang Kwon,
Zulkarnine Mahmud
Abstract:
Sequential tasks cannot be effectively handled in logic programming based on classical logic or linear logic. This limitation can be addressed by using a fragment of Japaridze'sSequential tasks cannot be effectively handled in logic programming based on classical logic or linear logic. This limitation can be addressed by using a fragment of Japaridze's computability logic. We propose \seqweb, an e…
▽ More
Sequential tasks cannot be effectively handled in logic programming based on classical logic or linear logic. This limitation can be addressed by using a fragment of Japaridze'sSequential tasks cannot be effectively handled in logic programming based on classical logic or linear logic. This limitation can be addressed by using a fragment of Japaridze's computability logic. We propose \seqweb, an extension to LogicWeb with sequential goal formulas. SeqWeb extends the LogicWeb by allowing goals of the form $G\seqand G$ and $G\seqor G$ where $G$ is a goal. These goals allow us to specify both sequential-conjunctive and sequential-disjunctive tasks. computability logic. We propose \seqweb, an extension to LogicWeb with sequential goal formulas. SeqWeb extends the LogicWeb by allowing goals of the form $G\seqand G$ and $G\seqor G$ where $G$ is a goal. These goals allow us to specify both sequential-conjunctive and sequential-disjunctive tasks.
△ Less
Submitted 3 December, 2014;
originally announced December 2014.
