-
A Machine Learning Approach for Predicting Upper Limb Motion Intentions with Multimodal Data in Virtual Reality
Authors:
Pavan Uttej Ravva,
Pinar Kullu,
Mohammad Fahim Abrar,
Roghayeh Leila Barmaki
Abstract:
Over the last decade, there has been significant progress in the field of interactive virtual rehabilitation. Physical therapy (PT) stands as a highly effective approach for enhancing physical impairments. However, patient motivation and progress tracking in rehabilitation outcomes remain a challenge. This work addresses the gap through a machine learning-based approach to objectively measure outc…
▽ More
Over the last decade, there has been significant progress in the field of interactive virtual rehabilitation. Physical therapy (PT) stands as a highly effective approach for enhancing physical impairments. However, patient motivation and progress tracking in rehabilitation outcomes remain a challenge. This work addresses the gap through a machine learning-based approach to objectively measure outcomes of the upper limb virtual therapy system in a user study with non-clinical participants. In this study, we use virtual reality to perform several tracing tasks while collecting motion and movement data using a KinArm robot and a custom-made wearable sleeve sensor. We introduce a two-step machine learning architecture to predict the motion intention of participants. The first step predicts reaching task segments to which the participant-marked points belonged using gaze, while the second step employs a Long Short-Term Memory (LSTM) model to predict directional movements based on resistance change values from the wearable sensor and the KinArm. We specifically propose to transpose our raw resistance data to the time-domain which significantly improves the accuracy of the models by 34.6%. To evaluate the effectiveness of our model, we compared different classification techniques with various data configurations. The results show that our proposed computational method is exceptional at predicting participant's actions with accuracy values of 96.72% for diamond reaching task, and 97.44% for circle reaching task, which demonstrates the great promise of using multimodal data, including eye-tracking and resistance change, to objectively measure the performance and intention in virtual rehabilitation settings.
△ Less
Submitted 15 May, 2024;
originally announced May 2024.
-
fNIRS Analysis of Interaction Techniques in Touchscreen-Based Educational Gaming
Authors:
Shayla Sharmin,
Elham Bakhshipour,
Behdokht Kiafar,
Md Fahim Abrar,
Pinar Kullu,
Nancy Getchell,
Roghayeh Leila Barmaki
Abstract:
Touchscreens are becoming increasingly widespread in educational games, enhancing the quality of learner experience. Traditional metrics are often used to evaluate various input modalities, including hand and stylus. However, there exists a gap in understanding the cognitive impacts of these modalities during educational gameplay, which can be addressed through brain signal analysis to gain deeper…
▽ More
Touchscreens are becoming increasingly widespread in educational games, enhancing the quality of learner experience. Traditional metrics are often used to evaluate various input modalities, including hand and stylus. However, there exists a gap in understanding the cognitive impacts of these modalities during educational gameplay, which can be addressed through brain signal analysis to gain deeper insights into the underlying cognitive function and necessary brain resources for each condition. This facilitates a more precise comparison between conditions. In this study, we compared the brain signal and user experience of using hands and stylus on touchscreens while playing an educational game by analyzing hemodynamic response and self-reported measures. Participants engaged in a Unity-based educational quiz game using both hand and stylus on a touchscreen in a counterbalanced within-subject design. Oxygenated and deoxygenated hemoglobin data were collected using fNIRS, alongside quiz performance scores and standardized and customized user experience questionnaire ratings. Our findings show almost the same performance level with both input modalities, however, the hand requires less oxygen flow which suggests a lower cognitive effort than using a stylus while playing the educational game. Although the result shows that the stylus condition required more neural involvement than the hand condition, there is no significant difference between the use of both input modalities. However, there is a statistically significant difference in self-reported measures that support the findings mentioned above, favoring the hand that enhances understanding of modality effects in interactive educational environments.
△ Less
Submitted 14 May, 2024;
originally announced May 2024.
-
Biased Over-the-Air Federated Learning under Wireless Heterogeneity
Authors:
Muhammad Faraz Ul Abrar,
Nicolò Michelusi
Abstract:
Recently, Over-the-Air (OTA) computation has emerged as a promising federated learning (FL) paradigm that leverages the waveform superposition properties of the wireless channel to realize fast model updates. Prior work focused on the OTA device ``pre-scaler" design under \emph{homogeneous} wireless conditions, in which devices experience the same average path loss, resulting in zero-bias solution…
▽ More
Recently, Over-the-Air (OTA) computation has emerged as a promising federated learning (FL) paradigm that leverages the waveform superposition properties of the wireless channel to realize fast model updates. Prior work focused on the OTA device ``pre-scaler" design under \emph{homogeneous} wireless conditions, in which devices experience the same average path loss, resulting in zero-bias solutions. Yet, zero-bias designs are limited by the device with the worst average path loss and hence may perform poorly in \emph{heterogeneous} wireless settings. In this scenario, there may be a benefit in designing \emph{biased} solutions, in exchange for a lower variance in the model updates. To optimize this trade-off, we study the design of OTA device pre-scalers by focusing on the OTA-FL convergence. We derive an upper bound on the model ``optimality error", which explicitly captures the effect of bias and variance in terms of the choice of the pre-scalers. Based on this bound, we identify two solutions of interest: minimum noise variance, and minimum noise variance zero-bias solutions. Numerical evaluations show that using OTA device pre-scalers that minimize the variance of FL updates, while allowing a small bias, can provide high gains over existing schemes.
△ Less
Submitted 28 March, 2024;
originally announced March 2024.
-
Analog-digital Scheduling for Federated Learning: A Communication-Efficient Approach
Authors:
Muhammad Faraz Ul Abrar,
Nicolò Michelusi
Abstract:
Over-the-air (OTA) computation has recently emerged as a communication-efficient Federated Learning (FL) paradigm to train machine learning models over wireless networks. However, its performance is limited by the device with the worst SNR, resulting in fast yet noisy updates. On the other hand, allocating orthogonal resource blocks (RB) to individual devices via digital channels mitigates the noi…
▽ More
Over-the-air (OTA) computation has recently emerged as a communication-efficient Federated Learning (FL) paradigm to train machine learning models over wireless networks. However, its performance is limited by the device with the worst SNR, resulting in fast yet noisy updates. On the other hand, allocating orthogonal resource blocks (RB) to individual devices via digital channels mitigates the noise problem, at the cost of increased communication latency. In this paper, we address this discrepancy and present ADFL, a novel Analog-Digital FL scheme: in each round, the parameter server (PS) schedules each device to either upload its gradient via the analog OTA scheme or transmit its quantized gradient over an orthogonal RB using the ``digital" scheme. Focusing on a single FL round, we cast the optimal scheduling problem as the minimization of the mean squared error (MSE) on the estimated global gradient at the PS, subject to a delay constraint, yielding the optimal device scheduling configuration and quantization bits for the digital devices. Our simulation results show that ADFL, by scheduling most of the devices in the OTA scheme while also occasionally employing the digital scheme for a few devices, consistently outperforms OTA-only and digital-only schemes, in both i.i.d. and non-i.i.d. settings.
△ Less
Submitted 2 February, 2024; v1 submitted 31 January, 2024;
originally announced February 2024.
