-
Deep learning empowered sensor fusion to improve infant movement classification
Authors:
Tomas Kulvicius,
Dajie Zhang,
Luise Poustka,
Sven Bölte,
Lennart Jahn,
Sarah Flügge,
Marc Kraft,
Markus Zweckstetter,
Karin Nielsen-Saines,
Florentin Wörgötter,
Peter B Marschik
Abstract:
There is a recent boom in the development of AI solutions to facilitate and enhance diagnostic procedures for established clinical tools. To assess the integrity of the develo** nervous system, the Prechtl general movement assessment (GMA) is recognized for its clinical value in diagnosing neurological impairments in early infancy. GMA has been increasingly augmented through machine learning app…
▽ More
There is a recent boom in the development of AI solutions to facilitate and enhance diagnostic procedures for established clinical tools. To assess the integrity of the develo** nervous system, the Prechtl general movement assessment (GMA) is recognized for its clinical value in diagnosing neurological impairments in early infancy. GMA has been increasingly augmented through machine learning approaches intending to scale-up its application, circumvent costs in the training of human assessors and further standardize classification of spontaneous motor patterns. Available deep learning tools, all of which are based on single sensor modalities, are however still considerably inferior to that of well-trained human assessors. These approaches are hardly comparable as all models are designed, trained and evaluated on proprietary/silo-data sets. With this study we propose a sensor fusion approach for assessing fidgety movements (FMs) comparing three different sensor modalities (pressure, inertial, and visual sensors). Various combinations and two sensor fusion approaches (late and early fusion) for infant movement classification were tested to evaluate whether a multi-sensor system outperforms single modality assessments. The performance of the three-sensor fusion (classification accuracy of 94.5\%) was significantly higher than that of any single modality evaluated, suggesting the sensor fusion approach is a promising avenue for automated classification of infant motor patterns. The development of a robust sensor fusion system may significantly enhance AI-based early recognition of neurofunctions, ultimately facilitating automated early detection of neurodevelopmental conditions.
△ Less
Submitted 21 June, 2024; v1 submitted 13 June, 2024;
originally announced June 2024.
-
Mobile solutions for clinical surveillance and evaluation in infancy -- General Movement Apps
Authors:
Peter B Marschik,
Amanda KL Kwong,
Nelson Silva,
Joy E Olsen,
Martin Schulte-Ruether,
Sven Bolte,
Maria Ortqvist,
Abbey Eeles,
Luise Poustka,
Christa Einspieler,
Karin Nielsen-Saines,
Dajie Zhang,
Alicia J Spittle
Abstract:
The Prechtl General Movements Assessment (GMA) has become a clinician and researcher tool-box for evaluating neurodevelopment in early infancy. Given it involves observation of infant movements from video recordings, utilising smartphone applications to obtain these recordings seems like the natural progression for the field. In this review, we look back on the development of apps for acquiring ge…
▽ More
The Prechtl General Movements Assessment (GMA) has become a clinician and researcher tool-box for evaluating neurodevelopment in early infancy. Given it involves observation of infant movements from video recordings, utilising smartphone applications to obtain these recordings seems like the natural progression for the field. In this review, we look back on the development of apps for acquiring general movement videos, describe the application and research studies of available apps, and discuss future directions of mobile solutions and their usability in research and clinical practice. We emphasise the importance of understanding the background that has led to these developments while introducing new technologies, including the barriers and facilitators along the pathway. The GMApp and Baby Moves App were the first ones developed to increase accessibility of the GMA, with two further apps, NeuroMotion and InMotion, designed since. The Baby Moves app has been applied most frequently. For the mobile future of GMA, we advocate collaboration to boost the field's progression and to reduce research waste. We propose future collaborative solutions including standardisation of cross-sites data collection, adaption to local context and privacy laws, employment of user feedback, and sustainable IT structures enabling continuous software updating.
△ Less
Submitted 26 March, 2023;
originally announced March 2023.
-
Infant movement classification through pressure distribution analysis
Authors:
Tomas Kulvicius,
Dajie Zhang,
Karin Nielsen-Saines,
Sven Bölte,
Marc Kraft,
Christa Einspieler,
Luise Poustka,
Florentin Wörgötter,
Peter B Marschik
Abstract:
Aiming at objective early detection of neuromotor disorders such as cerebral palsy, we proposed an innovative non-intrusive approach using a pressure sensing device to classify infant general movements (GMs). Here, we tested the feasibility of using pressure data to differentiate typical GM patterns of the ''fidgety period'' (i.e., fidgety movements) vs. the ''pre-fidgety period'' (i.e., writhing…
▽ More
Aiming at objective early detection of neuromotor disorders such as cerebral palsy, we proposed an innovative non-intrusive approach using a pressure sensing device to classify infant general movements (GMs). Here, we tested the feasibility of using pressure data to differentiate typical GM patterns of the ''fidgety period'' (i.e., fidgety movements) vs. the ''pre-fidgety period'' (i.e., writhing movements). Participants (N = 45) were sampled from a typically-develo** infant cohort. Multi-modal sensor data, including pressure data from a 32x32-grid pressure sensing mat with 1024 sensors, were prospectively recorded for each infant in seven succeeding laboratory sessions in biweekly intervals from 4-16 weeks of post-term age. For proof-of-concept, 1776 pressure data snippets, each 5s long, from the two targeted age periods were taken for movement classification. Each snippet was pre-annotated based on corresponding synchronised video data by human assessors as either fidgety present (FM+) or absent (FM-). Multiple neural network architectures were tested to distinguish the FM+ vs. FM- classes, including support vector machines (SVM), feed-forward networks (FFNs), convolutional neural networks (CNNs), and long short-term memory (LSTM) networks. The CNN achieved the highest average classification accuracy (81.4%) for classes FM+ vs. FM-. Comparing the pros and cons of other methods aiming at automated GMA to the pressure sensing approach, we concluded that the pressure sensing approach has great potential for efficient large-scale motion data acquisition and sharing. This will in return enable improvement of the approach that may prove scalable for daily clinical application for evaluating infant neuromotor functions.
△ Less
Submitted 1 July, 2023; v1 submitted 26 July, 2022;
originally announced August 2022.
-
Open video data sharing in developmental and behavioural science
Authors:
Peter B Marschik,
Tomas Kulvicius,
Sarah Flügge,
Claudius Widmann,
Karin Nielsen-Saines,
Martin Schulte-Rüther,
Britta Hüning,
Sven Bölte,
Luise Poustka,
Jeff Sigafoos,
Florentin Wörgötter,
Christa Einspieler,
Dajie Zhang
Abstract:
Video recording is a widely used method for documenting infant and child behaviours in research and clinical practice. Video data has rarely been shared due to ethical concerns of confidentiality, although the need of shared large-scaled datasets remains increasing. This demand is even more imperative when data-driven computer-based approaches are involved, such as screening tools to complement cl…
▽ More
Video recording is a widely used method for documenting infant and child behaviours in research and clinical practice. Video data has rarely been shared due to ethical concerns of confidentiality, although the need of shared large-scaled datasets remains increasing. This demand is even more imperative when data-driven computer-based approaches are involved, such as screening tools to complement clinical assessments. To share data while abiding by privacy protection rules, a critical question arises whether efforts at data de-identification reduce data utility? We addressed this question by showcasing the Prechtl's general movements assessment (GMA), an established and globally practised video-based diagnostic tool in early infancy for detecting neurological deficits, such as cerebral palsy. To date, no shared expert-annotated large data repositories for infant movement analyses exist. Such datasets would massively benefit training and recalibration of human assessors and the development of computer-based approaches. In the current study, sequences from a prospective longitudinal infant cohort with a total of 19451 available general movements video snippets were randomly selected for human clinical reasoning and computer-based analysis. We demonstrated for the first time that pseudonymisation by face-blurring video recordings is a viable approach. The video redaction did not affect classification accuracy for either human assessors or computer vision methods, suggesting an adequate and easy-to-apply solution for sharing movement video data. We call for further explorations into efficient and privacy rule-conforming approaches for deidentifying video data in scientific and clinical fields beyond movement assessments. These approaches shall enable sharing and merging stand-alone video datasets into large data pools to advance science and public health.
△ Less
Submitted 22 July, 2022;
originally announced July 2022.
