-
3D Gaze Tracking for Studying Collaborative Interactions in Mixed-Reality Environments
Authors:
Eduardo Davalos,
Yike Zhang,
Ashwin T. S.,
Joyce H. Fonteles,
Umesh Timalsina,
Guatam Biswas
Abstract:
This study presents a novel framework for 3D gaze tracking tailored for mixed-reality settings, aimed at enhancing joint attention and collaborative efforts in team-based scenarios. Conventional gaze tracking, often limited by monocular cameras and traditional eye-tracking apparatus, struggles with simultaneous data synchronization and analysis from multiple participants in group contexts. Our pro…
▽ More
This study presents a novel framework for 3D gaze tracking tailored for mixed-reality settings, aimed at enhancing joint attention and collaborative efforts in team-based scenarios. Conventional gaze tracking, often limited by monocular cameras and traditional eye-tracking apparatus, struggles with simultaneous data synchronization and analysis from multiple participants in group contexts. Our proposed framework leverages state-of-the-art computer vision and machine learning techniques to overcome these obstacles, enabling precise 3D gaze estimation without dependence on specialized hardware or complex data fusion. Utilizing facial recognition and deep learning, the framework achieves real-time, tracking of gaze patterns across several individuals, addressing common depth estimation errors, and ensuring spatial and identity consistency within the dataset. Empirical results demonstrate the accuracy and reliability of our method in group environments. This provides mechanisms for significant advances in behavior and interaction analysis in educational and professional training applications in dynamic and unstructured environments.
△ Less
Submitted 16 June, 2024;
originally announced June 2024.
-
A First Step in Using Machine Learning Methods to Enhance Interaction Analysis for Embodied Learning Environments
Authors:
Joyce Fonteles,
Eduardo Davalos,
Ashwin T. S.,
Yike Zhang,
Mengxi Zhou,
Efrat Ayalon,
Alicia Lane,
Selena Steinberg,
Gabriella Anton,
Joshua Danish,
Noel Enyedy,
Gautam Biswas
Abstract:
Investigating children's embodied learning in mixed-reality environments, where they collaboratively simulate scientific processes, requires analyzing complex multimodal data to interpret their learning and coordination behaviors. Learning scientists have developed Interaction Analysis (IA) methodologies for analyzing such data, but this requires researchers to watch hours of videos to extract and…
▽ More
Investigating children's embodied learning in mixed-reality environments, where they collaboratively simulate scientific processes, requires analyzing complex multimodal data to interpret their learning and coordination behaviors. Learning scientists have developed Interaction Analysis (IA) methodologies for analyzing such data, but this requires researchers to watch hours of videos to extract and interpret students' learning patterns. Our study aims to simplify researchers' tasks, using Machine Learning and Multimodal Learning Analytics to support the IA processes. Our study combines machine learning algorithms and multimodal analyses to support and streamline researcher efforts in develo** a comprehensive understanding of students' scientific engagement through their movements, gaze, and affective responses in a simulated scenario. To facilitate an effective researcher-AI partnership, we present an initial case study to determine the feasibility of visually representing students' states, actions, gaze, affect, and movement on a timeline. Our case study focuses on a specific science scenario where students learn about photosynthesis. The timeline allows us to investigate the alignment of critical learning moments identified by multimodal and interaction analysis, and uncover insights into students' temporal learning progressions.
△ Less
Submitted 9 May, 2024;
originally announced May 2024.
-
Gesture Controlled Robot For Human Detection
Authors:
Athira T. S,
Honey Manoj,
R S Vishnu Priya,
Vishnu K Menon,
Srilekshmi M
Abstract:
It is very important to locate survivors from collapsed buildings so that rescue operations can be arranged. Many lives are lost due to lack of competent systems to detect people in these collapsed buildings at the right time. So here we have designed a hand gesture controlled robot which is capable of detecting humans under these collapsed building parts. The proposed work can be used to access s…
▽ More
It is very important to locate survivors from collapsed buildings so that rescue operations can be arranged. Many lives are lost due to lack of competent systems to detect people in these collapsed buildings at the right time. So here we have designed a hand gesture controlled robot which is capable of detecting humans under these collapsed building parts. The proposed work can be used to access specific locations that are not humanly possible, and detect those humans trapped under the rubble of collapsed buildings. This information is then used to notify the rescue team to take adequate measures and initiate rescue operations accordingly.
△ Less
Submitted 31 January, 2024;
originally announced January 2024.
-
Kee** Teams in the Game: Predicting Dropouts in Online Problem-Based Learning Competition
Authors:
Aditya Panwar,
Ashwin T S,
Ramkumar Rajendran,
Kavi Arya
Abstract:
Online learning and MOOCs have become increasingly popular in recent years, and the trend will continue, given the technology boom. There is a dire need to observe learners' behavior in these online courses, similar to what instructors do in a face-to-face classroom. Learners' strategies and activities become crucial to understanding their behavior. One major challenge in online courses is predict…
▽ More
Online learning and MOOCs have become increasingly popular in recent years, and the trend will continue, given the technology boom. There is a dire need to observe learners' behavior in these online courses, similar to what instructors do in a face-to-face classroom. Learners' strategies and activities become crucial to understanding their behavior. One major challenge in online courses is predicting and preventing dropout behavior. While several studies have tried to perform such analysis, there is still a shortage of studies that employ different data streams to understand and predict the drop rates. Moreover, studies rarely use a fully online team-based collaborative environment as their context. Thus, the current study employs an online longitudinal problem-based learning (PBL) collaborative robotics competition as the testbed. Through methodological triangulation, the study aims to predict dropout behavior via the contributions of Discourse discussion forum 'activities' of participating teams, along with a self-reported Online Learning Strategies Questionnaire (OSLQ). The study also uses Qualitative interviews to enhance the ground truth and results. The OSLQ data is collected from more than 4000 participants. Furthermore, the study seeks to establish the reliability of OSLQ to advance research within online environments. Various Machine Learning algorithms are applied to analyze the data. The findings demonstrate the reliability of OSLQ with our substantial sample size and reveal promising results for predicting the dropout rate in online competition.
△ Less
Submitted 26 December, 2023;
originally announced December 2023.
-
CNN based Channel Estimation using NOMA for mmWave Massive MIMO System
Authors:
Anu T S,
Tara Raveendran
Abstract:
Non-Orthogonal Multiple Access (NOMA) schemes are being actively explored to address some of the major challenges in 5th Generation (5G) Wireless communications. Channel estimation is exceptionally challenging in scenarios where NOMA schemes are integrated with millimeter wave (mmWave) massive multiple-input multiple-output (MIMO) systems. An accurate estimation of the channel is essential in expl…
▽ More
Non-Orthogonal Multiple Access (NOMA) schemes are being actively explored to address some of the major challenges in 5th Generation (5G) Wireless communications. Channel estimation is exceptionally challenging in scenarios where NOMA schemes are integrated with millimeter wave (mmWave) massive multiple-input multiple-output (MIMO) systems. An accurate estimation of the channel is essential in exploiting the benefits of the pairing of the duo-NOMA and mmWave. This paper proposes a convolutional neural network (CNN) based approach to estimate the channel for NOMA based millimeter wave (mmWave) massive multiple-input multiple-output (MIMO) systems built on a hybrid architecture. Initially, users are grouped into different clusters based on their channel gains and beamforming technique is performed to maximize the signal in the direction of desired cluster. A coarse estimation of the channel is first made from the received signal and this estimate is given as the input to CNN to fine estimate the channel coefficients. Numerical illustrations show that the proposed method outperforms least square (LS) estimate, minimum mean square error (MMSE) estimate and are close to the Cramer-Rao Bound (CRB).
△ Less
Submitted 1 August, 2021;
originally announced August 2021.
