-
Multimodal Indoor Localisation in Parkinson's Disease for Detecting Medication Use: Observational Pilot Study in a Free-Living Setting
Authors:
Ferdian Jovan,
Catherine Morgan,
Ryan McConville,
Emma L. Tonkin,
Ian Craddock,
Alan Whone
Abstract:
Parkinson's disease (PD) is a slowly progressive, debilitating neurodegenerative disease which causes motor symptoms including gait dysfunction. Motor fluctuations are alterations between periods with a positive response to levodopa therapy ("on") and periods marked by re-emergency of PD symptoms ("off") as the response to medication wears off. These fluctuations often affect gait speed and they i…
▽ More
Parkinson's disease (PD) is a slowly progressive, debilitating neurodegenerative disease which causes motor symptoms including gait dysfunction. Motor fluctuations are alterations between periods with a positive response to levodopa therapy ("on") and periods marked by re-emergency of PD symptoms ("off") as the response to medication wears off. These fluctuations often affect gait speed and they increase in their disabling impact as PD progresses. To improve the effectiveness of current indoor localisation methods, a transformer-based approach utilising dual modalities which provide complementary views of movement, Received Signal Strength Indicator (RSSI) and accelerometer data from wearable devices, is proposed. A sub-objective aims to evaluate whether indoor localisation, including its in-home gait speed features (i.e. the time taken to walk between rooms), could be used to evaluate motor fluctuations by detecting whether the person with PD is taking levodopa medications or withholding them. To properly evaluate our proposed method, we use a free-living dataset where the movements and mobility are greatly varied and unstructured as expected in real-world conditions. 24 participants lived in pairs (consisting of one person with PD, one control) for five days in a smart home with various sensors. Our evaluation on the resulting dataset demonstrates that our proposed network outperforms other methods for indoor localisation. The sub-objective evaluation shows that precise room-level localisation predictions, transformed into in-home gait speed features, produce accurate predictions on whether the PD participant is taking or withholding their medications.
△ Less
Submitted 3 August, 2023;
originally announced August 2023.
-
When the Ground Truth is not True: Modelling Human Biases in Temporal Annotations
Authors:
Taku Yamagata,
Emma L. Tonkin,
Benjamin Arana Sanchez,
Ian Craddock,
Miquel Perello Nieto,
Raul Santos-Rodriguez,
Weisong Yang,
Peter Flach
Abstract:
In supervised learning, low quality annotations lead to poorly performing classification and detection models, while also rendering evaluation unreliable. This is particularly apparent on temporal data, where annotation quality is affected by multiple factors. For example, in the post-hoc self-reporting of daily activities, cognitive biases are one of the most common ingredients. In particular, re…
▽ More
In supervised learning, low quality annotations lead to poorly performing classification and detection models, while also rendering evaluation unreliable. This is particularly apparent on temporal data, where annotation quality is affected by multiple factors. For example, in the post-hoc self-reporting of daily activities, cognitive biases are one of the most common ingredients. In particular, reporting the start and duration of an activity after its finalisation may incorporate biases introduced by personal time perceptions, as well as the imprecision and lack of granularity due to time rounding. Here we propose a method to model human biases on temporal annotations and argue for the use of soft labels. Experimental results in synthetic data show that soft labels provide a better approximation of the ground truth for several metrics. We showcase the method on a real dataset of daily activities.
△ Less
Submitted 6 February, 2023;
originally announced February 2023.
-
Inertial Hallucinations -- When Wearable Inertial Devices Start Seeing Things
Authors:
Alessandro Masullo,
Toby Perrett,
Tilo Burghardt,
Ian Craddock,
Dima Damen,
Majid Mirmehdi
Abstract:
We propose a novel approach to multimodal sensor fusion for Ambient Assisted Living (AAL) which takes advantage of learning using privileged information (LUPI). We address two major shortcomings of standard multimodal approaches, limited area coverage and reduced reliability. Our new framework fuses the concept of modality hallucination with triplet learning to train a model with different modalit…
▽ More
We propose a novel approach to multimodal sensor fusion for Ambient Assisted Living (AAL) which takes advantage of learning using privileged information (LUPI). We address two major shortcomings of standard multimodal approaches, limited area coverage and reduced reliability. Our new framework fuses the concept of modality hallucination with triplet learning to train a model with different modalities to handle missing sensors at inference time. We evaluate the proposed model on inertial data from a wearable accelerometer device, using RGB videos and skeletons as privileged modalities, and show an improvement of accuracy of an average 6.6% on the UTD-MHAD dataset and an average 5.5% on the Berkeley MHAD dataset, reaching a new state-of-the-art for inertial-only classification accuracy on these datasets. We validate our framework through several ablation studies.
△ Less
Submitted 14 July, 2022;
originally announced July 2022.
-
Multimodal Indoor Localisation for Measuring Mobility in Parkinson's Disease using Transformers
Authors:
Ferdian Jovan,
Ryan McConville,
Catherine Morgan,
Emma Tonkin,
Alan Whone,
Ian Craddock
Abstract:
Parkinson's disease (PD) is a slowly progressive debilitating neurodegenerative disease which is prominently characterised by motor symptoms. Indoor localisation, including number and speed of room to room transitions, provides a proxy outcome which represents mobility and could be used as a digital biomarker to quantify how mobility changes as this disease progresses. We use data collected from 1…
▽ More
Parkinson's disease (PD) is a slowly progressive debilitating neurodegenerative disease which is prominently characterised by motor symptoms. Indoor localisation, including number and speed of room to room transitions, provides a proxy outcome which represents mobility and could be used as a digital biomarker to quantify how mobility changes as this disease progresses. We use data collected from 10 people with Parkinson's, and 10 controls, each of whom lived for five days in a smart home with various sensors. In order to more effectively localise them indoors, we propose a transformer-based approach utilizing two data modalities, Received Signal Strength Indicator (RSSI) and accelerometer data from wearable devices, which provide complementary views of movement. Our approach makes asymmetric and dynamic correlations by a) learning temporal correlations at different scales and levels, and b) utilizing various gating mechanisms to select relevant features within modality and suppress unnecessary modalities. On a dataset with real patients, we demonstrate that our proposed method gives an average accuracy of 89.9%, outperforming competitors. We also show that our model is able to better predict in-home mobility for people with Parkinson's with an average offset of 1.13 seconds to ground truth.
△ Less
Submitted 12 May, 2022;
originally announced May 2022.
-
Wi-Fi Based Passive Human Motion Sensing for In-Home Healthcare Applications
Authors:
Bo Tan,
Alison Burrows,
Robert Piechocki,
Ian Craddock,
Karl Woodbridge,
Kevin Chetty
Abstract:
This paper introduces a Wi-Fi signal based passive wireless sensing system that has the capability to detect diverse indoor human movements, from whole body motions to limb movements and including breathing movements of the chest. The real time signal processing used for human body motion sensing and software defined radio demo system are described and verified in practical experiments scenarios,…
▽ More
This paper introduces a Wi-Fi signal based passive wireless sensing system that has the capability to detect diverse indoor human movements, from whole body motions to limb movements and including breathing movements of the chest. The real time signal processing used for human body motion sensing and software defined radio demo system are described and verified in practical experiments scenarios, which include detection of through-wall human body movement, hand gesture or tremor, and even respiration. The experiment results offer potential for promising healthcare applications using Wi-Fi passive sensing in the home to monitor daily activities, to gather health data and detect emergency situations.
△ Less
Submitted 13 April, 2022;
originally announced April 2022.
-
OPERAnet: A Multimodal Activity Recognition Dataset Acquired from Radio Frequency and Vision-based Sensors
Authors:
Mohammud J. Bocus,
Wenda Li,
Shelly Vishwakarma,
Roget Kou,
Chong Tang,
Karl Woodbridge,
Ian Craddock,
Ryan McConville,
Raul Santos-Rodriguez,
Kevin Chetty,
Robert Piechocki
Abstract:
This paper presents a comprehensive dataset intended to evaluate passive Human Activity Recognition (HAR) and localization techniques with measurements obtained from synchronized Radio-Frequency (RF) devices and vision-based sensors. The dataset consists of RF data including Channel State Information (CSI) extracted from a WiFi Network Interface Card (NIC), Passive WiFi Radar (PWR) built upon a So…
▽ More
This paper presents a comprehensive dataset intended to evaluate passive Human Activity Recognition (HAR) and localization techniques with measurements obtained from synchronized Radio-Frequency (RF) devices and vision-based sensors. The dataset consists of RF data including Channel State Information (CSI) extracted from a WiFi Network Interface Card (NIC), Passive WiFi Radar (PWR) built upon a Software Defined Radio (SDR) platform, and Ultra-Wideband (UWB) signals acquired via commercial off-the-shelf hardware. It also consists of vision/Infra-red based data acquired from Kinect sensors. Approximately 8 hours of annotated measurements are provided, which are collected across two rooms from 6 participants performing 6 daily activities. This dataset can be exploited to advance WiFi and vision-based HAR, for example, using pattern recognition, skeletal representation, deep learning algorithms or other novel approaches to accurately recognize human activities. Furthermore, it can potentially be used to passively track a human in an indoor environment. Such datasets are key tools required for the development of new algorithms and methods in the context of smart homes, elderly care, and surveillance applications.
△ Less
Submitted 8 October, 2021;
originally announced October 2021.
-
Detecting Signatures of Early-stage Dementia with Behavioural Models Derived from Sensor Data
Authors:
Rafael Poyiadzi,
Weisong Yang,
Yoav Ben-Shlomo,
Ian Craddock,
Liz Coulthard,
Raul Santos-Rodriguez,
James Selwood,
Niall Twomey
Abstract:
There is a pressing need to automatically understand the state and progression of chronic neurological diseases such as dementia. The emergence of state-of-the-art sensing platforms offers unprecedented opportunities for indirect and automatic evaluation of disease state through the lens of behavioural monitoring. This paper specifically seeks to characterise behavioural signatures of mild cogniti…
▽ More
There is a pressing need to automatically understand the state and progression of chronic neurological diseases such as dementia. The emergence of state-of-the-art sensing platforms offers unprecedented opportunities for indirect and automatic evaluation of disease state through the lens of behavioural monitoring. This paper specifically seeks to characterise behavioural signatures of mild cognitive impairment (MCI) and Alzheimer's disease (AD) in the \textit{early} stages of the disease. We introduce bespoke behavioural models and analyses of key symptoms and deploy these on a novel dataset of longitudinal sensor data from persons with MCI and AD. We present preliminary findings that show the relationship between levels of sleep quality and wandering can be subtly different between patients in the early stages of dementia and healthy cohabiting controls.
△ Less
Submitted 3 July, 2020;
originally announced July 2020.
-
N2D: (Not Too) Deep Clustering via Clustering the Local Manifold of an Autoencoded Embedding
Authors:
Ryan McConville,
Raul Santos-Rodriguez,
Robert J Piechocki,
Ian Craddock
Abstract:
Deep clustering has increasingly been demonstrating superiority over conventional shallow clustering algorithms. Deep clustering algorithms usually combine representation learning with deep neural networks to achieve this performance, typically optimizing a clustering and non-clustering loss. In such cases, an autoencoder is typically connected with a clustering network, and the final clustering i…
▽ More
Deep clustering has increasingly been demonstrating superiority over conventional shallow clustering algorithms. Deep clustering algorithms usually combine representation learning with deep neural networks to achieve this performance, typically optimizing a clustering and non-clustering loss. In such cases, an autoencoder is typically connected with a clustering network, and the final clustering is jointly learned by both the autoencoder and clustering network. Instead, we propose to learn an autoencoded embedding and then search this further for the underlying manifold. For simplicity, we then cluster this with a shallow clustering algorithm, rather than a deeper network. We study a number of local and global manifold learning methods on both the raw data and autoencoded embedding, concluding that UMAP in our framework is best able to find the most clusterable manifold in the embedding, suggesting local manifold learning on an autoencoded embedding is effective for discovering higher quality discovering clusters. We quantitatively show across a range of image and time-series datasets that our method has competitive performance against the latest deep clustering algorithms, including out-performing current state-of-the-art on several. We postulate that these results show a promising research direction for deep clustering. The code can be found at https://github.com/rymc/n2d
△ Less
Submitted 30 June, 2020; v1 submitted 16 August, 2019;
originally announced August 2019.
-
Online Heart Rate Prediction using Acceleration from a Wrist Worn Wearable
Authors:
Ryan McConville,
Gareth Archer,
Ian Craddock,
Herman ter Horst,
Robert Piechocki,
James Pope,
Raul Santos-Rodriguez
Abstract:
In this paper we study the prediction of heart rate from acceleration using a wrist worn wearable. Although existing photoplethysmography (PPG) heart rate sensors provide reliable measurements, they use considerably more energy than accelerometers and have a major impact on battery life of wearable devices. By using energy-efficient accelerometers to predict heart rate, significant energy savings…
▽ More
In this paper we study the prediction of heart rate from acceleration using a wrist worn wearable. Although existing photoplethysmography (PPG) heart rate sensors provide reliable measurements, they use considerably more energy than accelerometers and have a major impact on battery life of wearable devices. By using energy-efficient accelerometers to predict heart rate, significant energy savings can be made. Further, we are interested in understanding patient recovery after a heart rate intervention, where we expect a variation in heart rate over time. Therefore, we propose an online approach to tackle the concept as time passes. We evaluate the methods on approximately 4 weeks of free living data from three patients over a number of months. We show that our approach can achieve good predictive performance (e.g., 2.89 Mean Absolute Error) while using the PPG heart rate sensor infrequently (e.g., 20.25% of the samples).
△ Less
Submitted 25 June, 2018;
originally announced July 2018.
-
A Guide to the SPHERE 100 Homes Study Dataset
Authors:
Atis Elsts,
Tilo Burghardt,
Dallan Byrne,
Massimo Camplani,
Dima Damen,
Xenofon Fafoutis,
Sion Hannuna,
William Harwin,
Michael Holmes,
Balazs Janko,
Victor Ponce Lopez,
Alessandro Masullo,
Majid Mirmehdi,
George Oikonomou,
Robert Piechocki,
R. Simon Sherratt,
Emma Tonkin,
Niall Twomey,
Antonis Vafeas,
Przemyslaw Woznowski,
Ian Craddock
Abstract:
The SPHERE project has developed a multi-modal sensor platform for health and behavior monitoring in residential environments. So far, the SPHERE platform has been deployed for data collection in approximately 50 homes for duration up to one year. This technical document describes the format and the expected content of the SPHERE dataset(s) under preparation. It includes a list of some data qualit…
▽ More
The SPHERE project has developed a multi-modal sensor platform for health and behavior monitoring in residential environments. So far, the SPHERE platform has been deployed for data collection in approximately 50 homes for duration up to one year. This technical document describes the format and the expected content of the SPHERE dataset(s) under preparation. It includes a list of some data quality problems (both known to exist in the dataset(s) and potential ones), their workarounds, and other information important to people working with the SPHERE data, software, and hardware. This document does not aim to be an exhaustive descriptor of the SPHERE dataset(s); it also does not aim to discuss or validate the potential scientific uses of the SPHERE data.
△ Less
Submitted 30 October, 2018; v1 submitted 30 May, 2018;
originally announced May 2018.
-
Probabilistic Sensor Fusion for Ambient Assisted Living
Authors:
Tom Diethe,
Niall Twomey,
Meelis Kull,
Peter Flach,
Ian Craddock
Abstract:
There is a widely-accepted need to revise current forms of health-care provision, with particular interest in sensing systems in the home. Given a multiple-modality sensor platform with heterogeneous network connectivity, as is under development in the Sensor Platform for HEalthcare in Residential Environment (SPHERE) Interdisciplinary Research Collaboration (IRC), we face specific challenges rela…
▽ More
There is a widely-accepted need to revise current forms of health-care provision, with particular interest in sensing systems in the home. Given a multiple-modality sensor platform with heterogeneous network connectivity, as is under development in the Sensor Platform for HEalthcare in Residential Environment (SPHERE) Interdisciplinary Research Collaboration (IRC), we face specific challenges relating to the fusion of the heterogeneous sensor modalities.
We introduce Bayesian models for sensor fusion, which aims to address the challenges of fusion of heterogeneous sensor modalities. Using this approach we are able to identify the modalities that have most utility for each particular activity, and simultaneously identify which features within that activity are most relevant for a given activity.
We further show how the two separate tasks of location prediction and activity recognition can be fused into a single model, which allows for simultaneous learning an prediction for both tasks.
We analyse the performance of this model on data collected in the SPHERE house, and show its utility. We also compare against some benchmark models which do not have the full structure,and show how the proposed model compares favourably to these methods
△ Less
Submitted 3 February, 2017;
originally announced February 2017.
-
Calorie Counter: RGB-Depth Visual Estimation of Energy Expenditure at Home
Authors:
Lili Tao,
Tilo Burghardt,
Majid Mirmehdi,
Dima Damen,
Ashley Cooper,
Sion Hannuna,
Massimo Camplani,
Adeline Paiement,
Ian Craddock
Abstract:
We present a new framework for vision-based estimation of calorific expenditure from RGB-D data - the first that is validated on physical gas exchange measurements and applied to daily living scenarios. Deriving a person's energy expenditure from sensors is an important tool in tracking physical activity levels for health and lifestyle monitoring. Most existing methods use metabolic lookup tables…
▽ More
We present a new framework for vision-based estimation of calorific expenditure from RGB-D data - the first that is validated on physical gas exchange measurements and applied to daily living scenarios. Deriving a person's energy expenditure from sensors is an important tool in tracking physical activity levels for health and lifestyle monitoring. Most existing methods use metabolic lookup tables (METs) for a manual estimate or systems with inertial sensors which ultimately require users to wear devices. In contrast, the proposed pose-invariant and individual-independent vision framework allows for a remote estimation of calorific expenditure. We introduce, and evaluate our approach on, a new dataset called SPHERE-calorie, for which visual estimates can be compared against simultaneously obtained, indirect calorimetry measures based on gas exchange. % based on per breath gas exchange. We conclude from our experiments that the proposed vision pipeline is suitable for home monitoring in a controlled environment, with calorific expenditure estimates above accuracy levels of commonly used manual estimations via METs. With the dataset released, our work establishes a baseline for future research for this little-explored area of computer vision.
△ Less
Submitted 27 July, 2016;
originally announced July 2016.
-
The SPHERE Challenge: Activity Recognition with Multimodal Sensor Data
Authors:
Niall Twomey,
Tom Diethe,
Meelis Kull,
Hao Song,
Massimo Camplani,
Sion Hannuna,
Xenofon Fafoutis,
Ni Zhu,
Pete Woznowski,
Peter Flach,
Ian Craddock
Abstract:
This paper outlines the Sensor Platform for HEalthcare in Residential Environment (SPHERE) project and details the SPHERE challenge that will take place in conjunction with European Conference on Machine Learning and Principles and Practice of Knowledge Discovery (ECML-PKDD) between March and July 2016. The SPHERE challenge is an activity recognition competition where predictions are made from vid…
▽ More
This paper outlines the Sensor Platform for HEalthcare in Residential Environment (SPHERE) project and details the SPHERE challenge that will take place in conjunction with European Conference on Machine Learning and Principles and Practice of Knowledge Discovery (ECML-PKDD) between March and July 2016. The SPHERE challenge is an activity recognition competition where predictions are made from video, accelerometer and environmental sensors. Monetary prizes will be awarded to the top three entrants, with Euro 1,000 being awarded to the winner, Euro 600 being awarded to the first runner up, and Euro 400 being awarded to the second runner up.
△ Less
Submitted 17 March, 2016; v1 submitted 2 March, 2016;
originally announced March 2016.
-
Sensors for healthcare: Would you want them in your home?
Authors:
Alison Burrows,
Rachel Gooberman-Hill,
Ian Craddock,
David Coyle
Abstract:
This paper describes some of the challenges set within SPHERE, a large-scale Interdisciplinary Research Collaboration that aims to develop sensor systems to monitor people's health and wellbeing in the home. In particular we discuss the dual task facing the User- Centered Design research group, to ensure the development of inclusive and desirable domestic healthcare technology. On the one hand, we…
▽ More
This paper describes some of the challenges set within SPHERE, a large-scale Interdisciplinary Research Collaboration that aims to develop sensor systems to monitor people's health and wellbeing in the home. In particular we discuss the dual task facing the User- Centered Design research group, to ensure the development of inclusive and desirable domestic healthcare technology. On the one hand, we seek to gain a rich understanding of the many envisaged users of the SPHERE system. On the other hand, for the user experience requirements to be translated into tangible outputs, it is crucial that we effectively communicate these findings to the broader team of SPHERE engineers and computer scientists.
△ Less
Submitted 11 August, 2014;
originally announced August 2014.
-
SPHERE: Meaningful and Inclusive Sensor-Based Home Healthcare
Authors:
Alison Burrows,
Rachel Gooberman-Hill,
Ian Craddock,
David Coyle
Abstract:
Given current demographic and health trends, and their economic implications, home healthcare technology has become a fertile area for research and development. Motivated by the need for a radical reform of healthcare provision, SPHERE is a large-scale Interdisciplinary Research Collaboration that aims to develop home sensor systems to monitor people's health and wellbeing in the home. This paper…
▽ More
Given current demographic and health trends, and their economic implications, home healthcare technology has become a fertile area for research and development. Motivated by the need for a radical reform of healthcare provision, SPHERE is a large-scale Interdisciplinary Research Collaboration that aims to develop home sensor systems to monitor people's health and wellbeing in the home. This paper outlines the unique circumstances of designing healthcare technology for the home environment, with a particular focus on how to ensure future systems are meaningful to and desirable for the intended users.
△ Less
Submitted 2 February, 2014;
originally announced February 2014.
