-
Towards the Understanding of Receptivity and Affect in EMAs using Physiological based Machine Learning Method: Analysis of Receptivity and Affect
Authors:
Zachary D King,
Han Yu,
Thomas Vaessen,
Iniz Myin-Germeys,
Akane Sano
Abstract:
As mobile health (mHealth) studies become increasingly productive due to the advancements in wearable and mobile sensor technology, our ability to monitor and model human behavior will be constrained by participant receptivity. The reliance on subjective responses for health constructs poses challenges, especially in populations with lower receptivity rates. Researchers have proposed machine-learn…
▽ More
As mobile health (mHealth) studies become increasingly productive due to the advancements in wearable and mobile sensor technology, our ability to monitor and model human behavior will be constrained by participant receptivity. The reliance on subjective responses for health constructs poses challenges, especially in populations with lower receptivity rates. Researchers have proposed machine-learning approaches to optimize survey timing and delivery to address this. However, there are concerns regarding potential biases or unintended influences on participant responses. Our study delves into factors impacting receptivity to ecological momentary assessments (EMA) in a 10-day mHealth study, exploring physiological relationships indicative of receptivity and affect. Utilizing data from 45 participants with wearable devices measuring various biometrics, we employ unsupervised (k-means clustering) and supervised (Random Forest and Neural Networks) machine learning methods to infer affect during non-responses. Findings reveal that triggering EMAs based on a receptivity model reduces reported negative affect by over 3 points (0.29 standard deviations). The predicted affect during non-responses exhibits a bimodal distribution, suggesting more frequent initiation during states of higher positive emotions. The study underscores a clear relationship between affect and receptivity, impacting mHealth study efficacy, especially those using machine learning for EMA triggering. Therefore, we propose a smart trigger that promotes EMA receptivity without influencing affect during sampled time points as future work.
△ Less
Submitted 23 November, 2023; v1 submitted 16 March, 2023;
originally announced March 2023.
-
More to Less (M2L): Enhanced Health Recognition in the Wild with Reduced Modality of Wearable Sensors
Authors:
Huiyuan Yang,
Han Yu,
Kusha Sridhar,
Thomas Vaessen,
Inez Myin-Germeys,
Akane Sano
Abstract:
Accurately recognizing health-related conditions from wearable data is crucial for improved healthcare outcomes. To improve the recognition accuracy, various approaches have focused on how to effectively fuse information from multiple sensors. Fusing multiple sensors is a common scenario in many applications, but may not always be feasible in real-world scenarios. For example, although combining b…
▽ More
Accurately recognizing health-related conditions from wearable data is crucial for improved healthcare outcomes. To improve the recognition accuracy, various approaches have focused on how to effectively fuse information from multiple sensors. Fusing multiple sensors is a common scenario in many applications, but may not always be feasible in real-world scenarios. For example, although combining bio-signals from multiple sensors (i.e., a chest pad sensor and a wrist wearable sensor) has been proved effective for improved performance, wearing multiple devices might be impractical in the free-living context. To solve the challenges, we propose an effective more to less (M2L) learning framework to improve testing performance with reduced sensors through leveraging the complementary information of multiple modalities during training. More specifically, different sensors may carry different but complementary information, and our model is designed to enforce collaborations among different modalities, where positive knowledge transfer is encouraged and negative knowledge transfer is suppressed, so that better representation is learned for individual modalities. Our experimental results show that our framework achieves comparable performance when compared with the full modalities. Our code and results will be available at https://github.com/compwell-org/More2Less.git.
△ Less
Submitted 16 February, 2022;
originally announced February 2022.
-
Modality Fusion Network and Personalized Attention in Momentary Stress Detection in the Wild
Authors:
Han Yu,
Thomas Vaessen,
Inez Myin-Germeys,
Akane Sano
Abstract:
Multimodal wearable physiological data in daily life have been used to estimate self-reported stress labels. However, missing data modalities in data collection makes it challenging to leverage all the collected samples. Besides, heterogeneous sensor data and labels among individuals add challenges in building robust stress detection models. In this paper, we proposed a modality fusion network (MF…
▽ More
Multimodal wearable physiological data in daily life have been used to estimate self-reported stress labels. However, missing data modalities in data collection makes it challenging to leverage all the collected samples. Besides, heterogeneous sensor data and labels among individuals add challenges in building robust stress detection models. In this paper, we proposed a modality fusion network (MFN) to train models and infer self-reported binary stress labels under both complete and incomplete modality conditions. In addition, we applied personalized attention (PA) strategy to leverage personalized representation along with the generalized one-size-fits-all model. We evaluated our methods on a multimodal wearable sensor dataset (N=41) including galvanic skin response (GSR) and electrocardiogram (ECG). Compared to the baseline method using the samples with complete modalities, the performance of the MFN improved by 1.6% in f1-scores. On the other hand, the proposed PA strategy showed a 2.3% higher stress detection f1-score and approximately up to 70% reduction in personalized model parameter size (9.1 MB) compared to the previous state-of-the-art transfer learning strategy (29.3 MB).
△ Less
Submitted 21 February, 2022; v1 submitted 19 July, 2021;
originally announced July 2021.
-
Predicting Depressive Symptom Severity through Individuals' Nearby Bluetooth Devices Count Data Collected by Mobile Phones: A Preliminary Longitudinal Study
Authors:
Yuezhou Zhang,
Amos A Folarin,
Shaoxiong Sun,
Nicholas Cummins,
Yatharth Ranjan,
Zulqarnain Rashid,
Pauline Conde,
Callum Stewart,
Petroula Laiou,
Faith Matcham,
Carolin Oetzmann,
Femke Lamers,
Sara Siddi,
Sara Simblett,
Aki Rintala,
David C Mohr,
Inez Myin-Germeys,
Til Wykes,
Josep Maria Haro,
Brenda WJH Pennix,
Vaibhav A Narayan,
Peter Annas,
Matthew Hotopf,
Richard JB Dobson
Abstract:
The Bluetooth sensor embedded in mobile phones provides an unobtrusive, continuous, and cost-efficient means to capture individuals' proximity information, such as the nearby Bluetooth devices count (NBDC). The continuous NBDC data can partially reflect individuals' behaviors and status, such as social connections and interactions, working status, mobility, and social isolation and loneliness, whi…
▽ More
The Bluetooth sensor embedded in mobile phones provides an unobtrusive, continuous, and cost-efficient means to capture individuals' proximity information, such as the nearby Bluetooth devices count (NBDC). The continuous NBDC data can partially reflect individuals' behaviors and status, such as social connections and interactions, working status, mobility, and social isolation and loneliness, which were found to be significantly associated with depression by previous survey-based studies. This paper aims to explore the NBDC data's value in predicting depressive symptom severity as measured via the 8-item Patient Health Questionnaire (PHQ-8). The data used in this paper included 2,886 bi-weekly PHQ-8 records collected from 316 participants recruited from three study sites in the Netherlands, Spain, and the UK as part of the EU RADAR-CNS study. From the NBDC data two weeks prior to each PHQ-8 score, we extracted 49 Bluetooth features, including statistical features and nonlinear features for measuring periodicity and regularity of individuals' life rhythms. Linear mixed-effect models were used to explore associations between Bluetooth features and the PHQ-8 score. We then applied hierarchical Bayesian linear regression models to predict the PHQ-8 score from the extracted Bluetooth features. A number of significant associations were found between Bluetooth features and depressive symptom severity. Compared with commonly used machine learning models, the proposed hierarchical Bayesian linear regression model achieved the best prediction metrics, R2= 0.526, and root mean squared error (RMSE) of 3.891. Bluetooth features can explain an extra 18.8% of the variance in the PHQ-8 score relative to the baseline model without Bluetooth features (R2=0.338, RMSE = 4.547).
△ Less
Submitted 26 April, 2021;
originally announced April 2021.
-
Using smartphones and wearable devices to monitor behavioural changes during COVID-19
Authors:
Shaoxiong Sun,
Amos Folarin,
Yatharth Ranjan,
Zulqarnain Rashid,
Pauline Conde,
Callum Stewart,
Nicholas Cummins,
Faith Matcham,
Gloria Dalla Costa,
Sara Simblett,
Letizia Leocani,
Per Soelberg Sørensen,
Mathias Buron,
Ana Isabel Guerrero,
Ana Zabalza,
Brenda WJH Penninx,
Femke Lamers,
Sara Siddi,
Josep Maria Haro,
Inez Myin-Germeys,
Aki Rintala,
Til Wykes,
Vaibhav A. Narayan,
Giancarlo Comi,
Matthew Hotopf
, et al. (1 additional authors not shown)
Abstract:
We aimed to explore the utility of the recently developed open-source mobile health platform RADAR-base as a toolbox to rapidly test the effect and response to NPIs aimed at limiting the spread of COVID-19. We analysed data extracted from smartphone and wearable devices and managed by the RADAR-base from 1062 participants recruited in Italy, Spain, Denmark, the UK, and the Netherlands. We derived…
▽ More
We aimed to explore the utility of the recently developed open-source mobile health platform RADAR-base as a toolbox to rapidly test the effect and response to NPIs aimed at limiting the spread of COVID-19. We analysed data extracted from smartphone and wearable devices and managed by the RADAR-base from 1062 participants recruited in Italy, Spain, Denmark, the UK, and the Netherlands. We derived nine features on a daily basis including time spent at home, maximum distance travelled from home, maximum number of Bluetooth-enabled nearby devices (as a proxy for physical distancing), step count, average heart rate, sleep duration, bedtime, phone unlock duration, and social app use duration. We performed Kruskal-Wallis tests followed by post-hoc Dunns tests to assess differences in these features among baseline, pre-, and during-lockdown periods. We also studied behavioural differences by age, gender, body mass index (BMI), and educational background. We were able to quantify expected changes in time spent at home, distance travelled, and the number of nearby Bluetooth-enabled devices between pre- and during-lockdown periods. We saw reduced sociality as measured through mobility features, and increased virtual sociality through phone usage. People were more active on their phones, spending more time using social media apps, particularly around major news events. Furthermore, participants had lower heart rate, went to bed later, and slept more. We also found that young people had longer homestay than older people during lockdown and fewer daily steps. Although there was no significant difference between the high and low BMI groups in time spent at home, the low BMI group walked more. RADAR-base can be used to rapidly quantify and provide a holistic view of behavioural changes in response to public health interventions as a result of infectious outbreaks such as COVID-19.
△ Less
Submitted 22 July, 2020; v1 submitted 29 April, 2020;
originally announced April 2020.
