-
Training and Validating a Treatment Recommender with Partial Verification Evidence
Authors:
Vishnu Unnikrishnan,
Clara Puga,
Miro Schleicher,
Uli Niemann,
Berthod Langguth,
Stefan Schoisswohl,
Birgit Mazurek,
Rilana Cima,
Jose Antonio Lopez-Escamez,
Dimitris Kikidis,
Eleftheria Vellidou,
Ruediger Pryss,
Winfried Schlee,
Myra Spiliopoulou
Abstract:
Current clinical decision support systems (DSS) are trained and validated on observational data from the target clinic. This is problematic for treatments validated in a randomized clinical trial (RCT), but not yet introduced in any clinic. In this work, we report on a method for training and validating the DSS using the RCT data. The key challenges we address are of missingness -- missing rationa…
▽ More
Current clinical decision support systems (DSS) are trained and validated on observational data from the target clinic. This is problematic for treatments validated in a randomized clinical trial (RCT), but not yet introduced in any clinic. In this work, we report on a method for training and validating the DSS using the RCT data. The key challenges we address are of missingness -- missing rationale for treatment assignment (the assignment is at random), and missing verification evidence, since the effectiveness of a treatment for a patient can only be verified (ground truth) for treatments what were actually assigned to a patient. We use data from a multi-armed RCT that investigated the effectiveness of single- and combination- treatments for 240+ tinnitus patients recruited and treated in 5 clinical centers.
To deal with the 'missing rationale' challenge, we re-model the target variable (outcome) in order to suppress the effect of the randomly-assigned treatment, and control on the effect of treatment in general. Our methods are also robust to missing values in features and with a small number of patients per RCT arm. We deal with 'missing verification evidence' by using counterfactual treatment verification, which compares the effectiveness of the DSS recommendations to the effectiveness of the RCT assignments when they are aligned v/s not aligned.
We demonstrate that our approach leverages the RCT data for learning and verification, by showing that the DSS suggests treatments that improve the outcome. The results are limited through the small number of patients per treatment; while our ensemble is designed to mitigate this effect, the predictive performance of the methods is affected by the smallness of the data. We provide a basis for the establishment of decision supporting routines on treatments that have been tested in RCTs but have not yet been deployed clinically.
△ Less
Submitted 10 June, 2024;
originally announced June 2024.
-
A cost-based multi-layer network approach for the discovery of patient phenotypes
Authors:
Clara Puga,
Uli Niemann,
Winfried Schlee,
Myra Spiliopoulou
Abstract:
Clinical records frequently include assessments of the characteristics of patients, which may include the completion of various questionnaires. These questionnaires provide a variety of perspectives on a patient's current state of well-being. Not only is it critical to capture the heterogeneity given by these perspectives, but there is also a growing demand for develo** cost-effective technologi…
▽ More
Clinical records frequently include assessments of the characteristics of patients, which may include the completion of various questionnaires. These questionnaires provide a variety of perspectives on a patient's current state of well-being. Not only is it critical to capture the heterogeneity given by these perspectives, but there is also a growing demand for develo** cost-effective technologies for clinical phenoty**. Filling out many questionnaires may be a strain for the patients and therefore costly. In this work, we propose COBALT -- a cost-based layer selector model for detecting phenotypes using a community detection approach. Our goal is to minimize the number of features used to build these phenotypes while preserving its quality. We test our model using questionnaire data from chronic tinnitus patients and represent the data in a multi-layer network structure. The model is then evaluated by predicting post-treatment data using baseline features (age, gender, and pre-treatment data) as well as the identified phenotypes as a feature. For some post-treatment variables, predictors using phenotypes from COBALT as features outperformed those using phenotypes detected by traditional clustering methods. Moreover, using phenotype data to predict post-treatment data proved beneficial in comparison with predictors that were solely trained with baseline features.
△ Less
Submitted 20 September, 2022; v1 submitted 19 September, 2022;
originally announced September 2022.
-
UNITI Mobile -- EMI-Apps for a Large-Scale European Study on Tinnitus
Authors:
Carsten Vogel,
Johannes Schobel,
Winfried Schlee,
Milena Engelke,
Rüdiger Pryss
Abstract:
More and more observational studies exploit the achievements of mobile technology to ease the overall implementation procedure. Many strategies like digital phenoty**, ecological momentary assessments or mobile crowdsensing are used in this context. Recently, an increasing number of intervention studies makes use of mobile technology as well. For the chronic disorder tinnitus, only few long-runn…
▽ More
More and more observational studies exploit the achievements of mobile technology to ease the overall implementation procedure. Many strategies like digital phenoty**, ecological momentary assessments or mobile crowdsensing are used in this context. Recently, an increasing number of intervention studies makes use of mobile technology as well. For the chronic disorder tinnitus, only few long-running intervention studies exist, which use mobile technology in a larger setting. Tinnitus is characterized by its heterogeneous patient's symptom profiles, which complicates the development of general treatments. In the UNITI project, researchers from different European countries try to unify existing treatments and interventions to cope with this heterogeneity. One study arm (UNITI Mobile) exploits mobile technology to investigate newly implemented interventions types, especially within the pan-European setting. The goals are to learn more about the validity and usefulness of mobile technology in this context. Furthermore, differences among the countries shall be investigated. Practically, two native intervention apps have been developed for UNITI and the mobile study arm, which pose features not presented so far in other apps of the authors. Along the implementation procedure, it is discussed whether these features might leverage similar types of studies in future. Since instruments like the mHealth evidence reporting and assessment checklist (mERA), developed by the WHO mHealth technical evidence review group, indicate that aspects shown for UNITI Mobile are important in the context of health interventions using mobile phones, our findings may be of a more general interest and are therefore being discussed in the work at hand.
△ Less
Submitted 22 July, 2021;
originally announced July 2021.
-
Are Non-Experts Able to Comprehend Business Process Models -- Study Insights Involving Novices and Experts
Authors:
Michael Winter,
Rüdiger Pryss,
Thomas Probst,
Winfried Schlee,
Miles Tallon,
Ulrich Frick,
Manfred Reichert
Abstract:
The comprehension of business process models is crucial for enterprises. Prior research has shown that children as well as adolescents perceive and interpret graphical representations in a different manner compared to grown-ups. To evaluate this, observations in the context of business process models are presented in this paper obtained from a study on visual literacy in cultural education. We dem…
▽ More
The comprehension of business process models is crucial for enterprises. Prior research has shown that children as well as adolescents perceive and interpret graphical representations in a different manner compared to grown-ups. To evaluate this, observations in the context of business process models are presented in this paper obtained from a study on visual literacy in cultural education. We demonstrate that adolescents without expertise in process model comprehension are able to correctly interpret business process models expressed in terms of BPMN 2.0. In a comprehensive study, n = 205 learners (i.e., pupils at the age of 15) needed to answer questions related to process models they were confronted with, reflecting different levels of complexity. In addition, process models were created with varying styles of element labels. Study results indicate that an abstract description (i.e., using only alphabetic letters) of process models is understood more easily compared to concrete or pseudo} descriptions. As benchmark, results are compared with the ones of modeling experts (n = 40). Amongst others, study findings suggest using abstract descriptions in order to introduce novices to process modeling notations. With the obtained insights, we highlight that process models can be properly comprehended by novices.
△ Less
Submitted 6 July, 2021; v1 submitted 2 July, 2021;
originally announced July 2021.
-
Corona Health -- A Study- and Sensor-based Mobile App Platform Exploring Aspects of the COVID-19 Pandemic
Authors:
Felix Beierle,
Johannes Schobel,
Carsten Vogel,
Johannes Allgaier,
Lena Mulansky,
Fabian Haug,
Julian Haug,
Winfried Schlee,
Marc Holfelder,
Michael Stach,
Marc Schickler,
Harald Baumeister,
Caroline Cohrdes,
Jürgen Deckert,
Lorenz Deserno,
Johanna-Sophie Edler,
Felizitas A. Eichner,
Helmut Greger,
Grit Hein,
Peter Heuschmann,
Dennis John,
Hans A. Kestler,
Dagmar Krefting,
Berthold Langguth,
Patrick Meybohm
, et al. (7 additional authors not shown)
Abstract:
Physical and mental well-being during the COVID-19 pandemic is typically assessed via surveys, which might make it difficult to conduct longitudinal studies and might lead to data suffering from recall bias. Ecological momentary assessment (EMA) driven smartphone apps can help alleviate such issues, allowing for in situ recordings. Implementing such an app is not trivial, necessitates strict regul…
▽ More
Physical and mental well-being during the COVID-19 pandemic is typically assessed via surveys, which might make it difficult to conduct longitudinal studies and might lead to data suffering from recall bias. Ecological momentary assessment (EMA) driven smartphone apps can help alleviate such issues, allowing for in situ recordings. Implementing such an app is not trivial, necessitates strict regulatory and legal requirements, and requires short development cycles to appropriately react to abrupt changes in the pandemic. Based on an existing app framework, we developed Corona Health, an app that serves as a platform for deploying questionnaire-based studies in combination with recordings of mobile sensors. In this paper, we present the technical details of Corona Health and provide first insights into the collected data. Through collaborative efforts from experts from public health, medicine, psychology, and computer science, we released Corona Health publicly on Google Play and the Apple App Store (in July, 2020) in 8 languages and attracted 7,290 installations so far. Currently, five studies related to physical and mental well-being are deployed and 17,241 questionnaires have been filled out. Corona Health proves to be a viable tool for conducting research related to the COVID-19 pandemic and can serve as a blueprint for future EMA-based studies. The data we collected will substantially improve our knowledge on mental and physical health states, traits and trajectories as well as its risk and protective factors over the course of the COVID-19 pandemic and its diverse prevention measures.
△ Less
Submitted 6 July, 2021; v1 submitted 7 June, 2021;
originally announced June 2021.
-
Medical device regulation efforts for mHealth apps -- An experience report of Corona Check and Corona Health
Authors:
Marc Holfelder,
Lena Mulansky,
Winfried Schlee,
Harald Baumeister,
Johannes Schobel,
Helmut Greger,
Andreas Hoff,
Rüdiger Pryss
Abstract:
Within the healthcare environment, mobile health (mHealth) applications (apps) are more and more important. The number of new mHealth apps has risen steadily in the last years. Especially the Covid-19 pandemic has led to an enormous amount of app releases. Notably, in most countries, mHealth applications have to be already compliant with several regulatory aspects in order to be declared to be a '…
▽ More
Within the healthcare environment, mobile health (mHealth) applications (apps) are more and more important. The number of new mHealth apps has risen steadily in the last years. Especially the Covid-19 pandemic has led to an enormous amount of app releases. Notably, in most countries, mHealth applications have to be already compliant with several regulatory aspects in order to be declared to be a 'medical app'. However, the latest applicable medical device regulation (MDR) does not comment in more detail on the topic of the requirements for mHealth applications. When develo** a medical app, it is essential that all contributors in an interdisciplinary team - especially the software engineers - are aware of the specific regulatory requirements beforehand. The development process, however, should not be stalled too long due to the integration of the MDR. Therefore, a develo** framework, which includes these aspects, is required, to enable a smooth development process. The paper at hand introduces the creation of such a framework on the basis of the Corona Health and Corona Check apps. The relevant regulatory guidelines are listed and summarized to a guidance for medical app developments. In particular, the important stages and faced challenges emerged during the entire development process are highlighted.
△ Less
Submitted 28 April, 2021;
originally announced April 2021.
-
Develo** Apps for Researching the COVID-19 Pandemic with the TrackYourHealth Platform
Authors:
Carsten Vogel,
Rüdiger Pryss,
Johannes Schobel,
Winfried Schlee,
Felix Beierle
Abstract:
Through lockdowns and other severe changes to daily life, almost everyone is affected by the COVID-19 pandemic. Scientists and medical doctors are - among others - mainly interested in researching, monitoring, and improving physical and mental health of the general population. Mobile health apps (mHealth), and apps conducting ecological momentary assessments (EMA) respectively, can help in this co…
▽ More
Through lockdowns and other severe changes to daily life, almost everyone is affected by the COVID-19 pandemic. Scientists and medical doctors are - among others - mainly interested in researching, monitoring, and improving physical and mental health of the general population. Mobile health apps (mHealth), and apps conducting ecological momentary assessments (EMA) respectively, can help in this context. However, develo** such mobile applications poses many challenges like costly software development efforts, strict privacy rules, compliance with ethical guidelines, local laws, and regulations. In this paper, we present TrackYourHealth (TYH), a highly configurable, generic, and modular mobile data collection and EMA platform, which enabled us to develop and release two mobile multi-platform applications related to COVID-19 in just a few weeks. We present TYH and highlight specific challenges researchers and developers of similar apps may also face, especially when develo** apps related to the medical field.
△ Less
Submitted 25 March, 2021;
originally announced March 2021.
-
Context Data Categories and Privacy Model for Mobile Data Collection Apps
Authors:
Felix Beierle,
Vinh Thuy Tran,
Mathias Allemand,
Patrick Neff,
Winfried Schlee,
Thomas Probst,
Rüdiger Pryss,
Johannes Zimmermann
Abstract:
Context-aware applications stemming from diverse fields like mobile health, recommender systems, and mobile commerce potentially benefit from knowing aspects of the user's personality. As filling out personality questionnaires is tedious, we propose the prediction of the user's personality from smartphone sensor and usage data. In order to collect data for researching the relationship between smar…
▽ More
Context-aware applications stemming from diverse fields like mobile health, recommender systems, and mobile commerce potentially benefit from knowing aspects of the user's personality. As filling out personality questionnaires is tedious, we propose the prediction of the user's personality from smartphone sensor and usage data. In order to collect data for researching the relationship between smartphone data and personality, we developed the Android app TYDR (Track Your Daily Routine) which tracks smartphone data and utilizes psychometric personality questionnaires. With TYDR, we track a larger variety of smartphone data than similar existing apps, including metadata on notifications, photos taken, and music played back by the user. For the development of TYDR, we introduce a general context data model consisting of four categories that focus on the user's different types of interactions with the smartphone: physical conditions and activity, device status and usage, core functions usage, and app usage. On top of this, we develop the privacy model PM-MoDaC specifically for apps related to the collection of mobile data, consisting of nine proposed privacy measures. We present the implementation of all of those measures in TYDR. Although the utilization of the user's personality based on the usage of his or her smartphone is a challenging endeavor, it seems to be a promising approach for various types of context-aware mobile applications.
△ Less
Submitted 4 July, 2018;
originally announced July 2018.
-
TYDR - Track Your Daily Routine. Android App for Tracking Smartphone Sensor and Usage Data
Authors:
Felix Beierle,
Vinh Thuy Tran,
Mathias Allemand,
Patrick Neff,
Winfried Schlee,
Thomas Probst,
Rüdiger Pryss,
Johannes Zimmermann
Abstract:
We present the Android app TYDR (Track Your Daily Routine) which tracks smartphone sensor and usage data and utilizes standardized psychometric personality questionnaires. With the app, we aim at collecting data for researching correlations between the tracked smartphone data and the user's personality in order to predict personality from smartphone data. In this paper, we highlight our approaches…
▽ More
We present the Android app TYDR (Track Your Daily Routine) which tracks smartphone sensor and usage data and utilizes standardized psychometric personality questionnaires. With the app, we aim at collecting data for researching correlations between the tracked smartphone data and the user's personality in order to predict personality from smartphone data. In this paper, we highlight our approaches in addressing the challenges in develo** such an app. We optimize the tracking of sensor data by assessing the trade-off of size of data and battery consumption and granularity of the stored information. Our user interface is designed to incentivize users to install the app and fill out questionnaires. TYDR processes and visualizes the tracked sensor and usage data as well as the results of the personality questionnaires. When develo** an app that will be used in psychological studies, requirements posed by ethics commissions / institutional review boards and data protection officials have to be met. We detail our approaches concerning those requirements regarding the anonymized storing of user data, informing the users about the data collection, and enabling an opt-out option. We present our process for anonymized data storing while still being able to identify individual users who successfully completed a psychological study with the app.
△ Less
Submitted 18 March, 2018;
originally announced March 2018.
