-
Enhancing Food Intake Tracking in Long-Term Care with Automated Food Imaging and Nutrient Intake Tracking (AFINI-T) Technology
Authors:
Kaylen J. Pfisterer,
Robert Amelard,
Jennifer Boger,
Audrey G. Chung,
Heather H. Keller,
Alexander Wong
Abstract:
Half of long-term care (LTC) residents are malnourished increasing hospitalization, mortality, morbidity, with lower quality of life. Current tracking methods are subjective and time consuming. This paper presents the automated food imaging and nutrient intake tracking (AFINI-T) technology designed for LTC. We propose a novel convolutional autoencoder for food classification, trained on an augment…
▽ More
Half of long-term care (LTC) residents are malnourished increasing hospitalization, mortality, morbidity, with lower quality of life. Current tracking methods are subjective and time consuming. This paper presents the automated food imaging and nutrient intake tracking (AFINI-T) technology designed for LTC. We propose a novel convolutional autoencoder for food classification, trained on an augmented UNIMIB2016 dataset and tested on our simulated LTC food intake dataset (12 meal scenarios; up to 15 classes each; top-1 classification accuracy: 88.9%; mean intake error: -0.4 mL$\pm$36.7 mL). Nutrient intake estimation by volume was strongly linearly correlated with nutrient estimates from mass ($r^2$ 0.92 to 0.99) with good agreement between methods ($σ$= -2.7 to -0.01; zero within each of the limits of agreement). The AFINI-T approach is a deep-learning powered computational nutrient sensing system that may provide a novel means for more accurately and objectively tracking LTC resident food intake to support and prevent malnutrition tracking strategies.
△ Less
Submitted 8 December, 2021;
originally announced December 2021.
-
Temporal convolutional networks predict dynamic oxygen uptake response from wearable sensors across exercise intensities
Authors:
Robert Amelard,
Eric T Hedge,
Richard L Hughson
Abstract:
Oxygen consumption (VO$_2$) provides established clinical and physiological indicators of cardiorespiratory function and exercise capacity. However, VO$_2$ monitoring is largely limited to specialized laboratory settings, making its widespread monitoring elusive. Here, we investigate temporal prediction of VO$_2$ from wearable sensors during cycle ergometer exercise using a temporal convolutional…
▽ More
Oxygen consumption (VO$_2$) provides established clinical and physiological indicators of cardiorespiratory function and exercise capacity. However, VO$_2$ monitoring is largely limited to specialized laboratory settings, making its widespread monitoring elusive. Here, we investigate temporal prediction of VO$_2$ from wearable sensors during cycle ergometer exercise using a temporal convolutional network (TCN). Cardiorespiratory signals were acquired from a smart shirt with integrated textile sensors alongside ground-truth VO$_2$ from a metabolic system on twenty-two young healthy adults. Participants performed one ramp-incremental and three pseudorandom binary sequence exercise protocols to assess a range of VO$_2$ dynamics. A TCN model was developed using causal convolutions across an effective history length to model the time-dependent nature of VO$_2$. Optimal history length was determined through minimum validation loss across hyperparameter values. The best performing model encoded 218 s history length (TCN-VO$_2$ A), with 187 s, 97 s, and 76 s yielding less than 3% deviation from the optimal validation loss. TCN-VO$_2$ A showed strong prediction accuracy (mean, 95% CI) across all exercise intensities (-22 ml.min$^{-1}$, [-262, 218]), spanning transitions from low-moderate (-23 ml.min$^{-1}$, [-250, 204]), low-high (14 ml.min$^{-1}$, [-252, 280]), ventilatory threshold-high (-49 ml.min$^{-1}$, [-274, 176]), and maximal (-32 ml.min$^{-1}$, [-261, 197]) exercise. Second-by-second classification of physical activity across 16090 s of predicted VO$_2$ was able to discern between vigorous, moderate, and light activity with high accuracy (94.1%). This system enables quantitative aerobic activity monitoring in non-laboratory settings across a range of exercise intensities using wearable sensors for monitoring exercise prescription adherence and personal fitness.
△ Less
Submitted 13 October, 2021; v1 submitted 20 May, 2021;
originally announced May 2021.
-
Optical Hemodynamic Imaging of Jugular Venous Dynamics During Altered Central Venous Pressure
Authors:
Robert Amelard,
Andrew D Robertson,
Courtney A Patterson,
Hannah Heigold,
Essi Saarikoski,
Richard L Hughson
Abstract:
An optical imaging system is proposed for quantitatively assessing jugular venous response to altered central venous pressure. The proposed system assesses sub-surface optical absorption changes from jugular venous waveforms with a spatial calibration procedure to normalize incident tissue illumination. Widefield frames of the right lateral neck were captured and calibrated using a novel flexible…
▽ More
An optical imaging system is proposed for quantitatively assessing jugular venous response to altered central venous pressure. The proposed system assesses sub-surface optical absorption changes from jugular venous waveforms with a spatial calibration procedure to normalize incident tissue illumination. Widefield frames of the right lateral neck were captured and calibrated using a novel flexible surface calibration method. A hemodynamic optical model was derived to quantify jugular venous optical attenuation (JVA) signals, and generate a spatial jugular venous pulsatility map. JVA was assessed in three cardiovascular protocols that altered central venous pressure: acute central hypovolemia (lower body negative pressure), venous congestion (head-down tilt), and impaired cardiac filling (Valsalva maneuver). JVA waveforms exhibited biphasic wave properties consistent with jugular venous pulse dynamics when time-aligned with an electrocardiogram. JVA correlated strongly (median, interquartile range) with invasive central venous pressure during graded central hypovolemia (r=0.85, [0.72, 0.95]), graded venous congestion (r=0.94, [0.84, 0.99]), and impaired cardiac filling (r=0.94, [0.85, 0.99]). Reduced JVA during graded acute hypovolemia was strongly correlated with reductions in stroke volume (SV) (r=0.85, [0.76, 0.92]) from baseline (SV: 79$\pm$15 mL, JVA: 0.56$\pm$0.10 a.u.) to -40 mmHg suction (SV: 59$\pm$18 mL, JVA: 0.47$\pm$0.05 a.u.; p$<$0.01). The proposed non-contact optical imaging system demonstrated jugular venous dynamics consistent with invasive central venous monitoring during three protocols that altered central venous pressure. This system provides non-invasive monitoring of pressure-induced jugular venous dynamics in clinically relevant conditions where catheterization is traditionally required, enabling monitoring in non-surgical environments.
△ Less
Submitted 24 March, 2021; v1 submitted 22 July, 2020;
originally announced July 2020.
-
When Segmentation is Not Enough: Rectifying Visual-Volume Discordance Through Multisensor Depth-Refined Semantic Segmentation for Food Intake Tracking in Long-Term Care
Authors:
Kaylen J Pfisterer,
Robert Amelard,
Audrey G Chung,
Braeden Syrnyk,
Alexander MacLean,
Heather H Keller,
Alexander Wong
Abstract:
Malnutrition is a multidomain problem affecting 54% of older adults in long-term care (LTC). Monitoring nutritional intake in LTC is laborious and subjective, limiting clinical inference capabilities. Recent advances in automatic image-based food estimation have not yet been evaluated in LTC settings. Here, we describe a fully automatic imaging system for quantifying food intake. We propose a nove…
▽ More
Malnutrition is a multidomain problem affecting 54% of older adults in long-term care (LTC). Monitoring nutritional intake in LTC is laborious and subjective, limiting clinical inference capabilities. Recent advances in automatic image-based food estimation have not yet been evaluated in LTC settings. Here, we describe a fully automatic imaging system for quantifying food intake. We propose a novel deep convolutional encoder-decoder food network with depth-refinement (EDFN-D) using an RGB-D camera for quantifying a plate's remaining food volume relative to reference portions in whole and modified texture foods. We trained and validated the network on the pre-labelled UNIMIB2016 food dataset and tested on our two novel LTC-inspired plate datasets (689 plate images, 36 unique foods). EDFN-D performed comparably to depth-refined graph cut on IOU (0.879 vs. 0.887), with intake errors well below typical 50% (mean percent intake error: -4.2%). We identify how standard segmentation metrics are insufficient due to visual-volume discordance, and include volume disparity analysis to facilitate system trust. This system provides improved transparency, approximates human assessors with enhanced objectivity, accuracy, and precision while avoiding hefty semi-automatic method time requirements. This may help address short-comings currently limiting utility of automated early malnutrition detection in resource-constrained LTC and hospital settings.
△ Less
Submitted 31 March, 2021; v1 submitted 24 October, 2019;
originally announced October 2019.
-
Monocular 3D Sway Tracking for Assessing Postural Instability in Cerebral Hypoperfusion During Quiet Standing
Authors:
Robert Amelard,
Kevin R Murray,
Eric T Hedge,
Taylor W Cleworth,
Mamiko Noguchi,
Andrew Laing,
Richard L Hughson
Abstract:
Postural instability is prevalent in aging and neurodegenerative disease, decreasing quality of life and independence. Quantitatively monitoring balance control is important for assessing treatment efficacy and rehabilitation progress. However, existing technologies for assessing postural sway are complex and expensive, limiting their widespread utility. Here, we propose a monocular imaging system…
▽ More
Postural instability is prevalent in aging and neurodegenerative disease, decreasing quality of life and independence. Quantitatively monitoring balance control is important for assessing treatment efficacy and rehabilitation progress. However, existing technologies for assessing postural sway are complex and expensive, limiting their widespread utility. Here, we propose a monocular imaging system capable of assessing sub-millimeter 3D sway dynamics during quiet standing. Two anatomical targets with known feature geometries were placed on the lumbar and shoulder. Upper and lower trunk 3D kinematic motion was automatically assessed from a set of 2D frames through geometric feature tracking and an inverse motion model. Sway was tracked in 3D and compared between control and hypoperfusion conditions in 14 healthy young adults. The proposed system demonstrated high agreement with a commercial motion capture system (error $1.5 \times 10^{-4}~\text{mm}$, [$-0.52$, $0.52$]). Between-condition differences in sway dynamics were observed in anterior-posterior sway during early and mid stance, and medial-lateral sway during mid stance commensurate with decreased cerebral perfusion, followed by recovered sway dynamics during late stance with cerebral perfusion recovery. This inexpensive single-camera system enables quantitative 3D sway monitoring for assessing neuromuscular balance control in weakly constrained environments.
△ Less
Submitted 5 November, 2019; v1 submitted 11 July, 2019;
originally announced July 2019.
-
Towards computer vision powered color-nutrient assessment of pureed food
Authors:
Kaylen J. Pfisterer,
Robert Amelard,
Braeden Syrnyk,
Alexander Wong
Abstract:
With one in four individuals afflicted with malnutrition, computer vision may provide a way of introducing a new level of automation in the nutrition field to reliably monitor food and nutrient intake. In this study, we present a novel approach to modeling the link between color and vitamin A content using transmittance imaging of a pureed foods dilution series in a computer vision powered nutrien…
▽ More
With one in four individuals afflicted with malnutrition, computer vision may provide a way of introducing a new level of automation in the nutrition field to reliably monitor food and nutrient intake. In this study, we present a novel approach to modeling the link between color and vitamin A content using transmittance imaging of a pureed foods dilution series in a computer vision powered nutrient sensing system via a fine-tuned deep autoencoder network, which in this case was trained to predict the relative concentration of sweet potato purees. Experimental results show the deep autoencoder network can achieve an accuracy of 80% across beginner (6 month) and intermediate (8 month) commercially prepared pureed sweet potato samples. Prediction errors may be explained by fundamental differences in optical properties which are further discussed.
△ Less
Submitted 1 May, 2019;
originally announced May 2019.
-
A new take on measuring relative nutritional density: The feasibility of using a deep neural network to assess commercially-prepared pureed food concentrations
Authors:
Kaylen J. Pfisterer,
Robert Amelard,
Audrey G. Chung,
Alexander Wong
Abstract:
Dysphagia affects 590 million people worldwide and increases risk for malnutrition. Pureed food may reduce choking, however preparation differences impact nutrient density making quality assurance necessary. This paper is the first study to investigate the feasibility of computational pureed food nutritional density analysis using an imaging system. Motivated by a theoretical optical dilution mode…
▽ More
Dysphagia affects 590 million people worldwide and increases risk for malnutrition. Pureed food may reduce choking, however preparation differences impact nutrient density making quality assurance necessary. This paper is the first study to investigate the feasibility of computational pureed food nutritional density analysis using an imaging system. Motivated by a theoretical optical dilution model, a novel deep neural network (DNN) was evaluated using 390 samples from thirteen types of commercially prepared purees at five dilutions. The DNN predicted relative concentration of the puree sample (20%, 40%, 60%, 80%, 100% initial concentration). Data were captured using same-side reflectance of multispectral imaging data at different polarizations at three exposures. Experimental results yielded an average top-1 prediction accuracy of 92.2+/-0.41% with sensitivity and specificity of 83.0+/-15.0% and 95.0+/-4.8%, respectively. This DNN imaging system for nutrient density analysis of pureed food shows promise as a novel tool for nutrient quality assurance.
△ Less
Submitted 3 November, 2017; v1 submitted 23 July, 2017;
originally announced July 2017.
-
Spatial probabilistic pulsatility model for enhancing photoplethysmographic imaging systems
Authors:
Robert Amelard,
David A Clausi,
Alexander Wong
Abstract:
Photolethysmographic imaging (PPGI) is a widefield non-contact biophotonic technology able to remotely monitor cardiovascular function over anatomical areas. Though spatial context can provide increased physiological insight, existing PPGI systems rely on coarse spatial averaging with no anatomical priors for assessing arterial pulsatility. Here, we developed a continuous probabilistic pulsatility…
▽ More
Photolethysmographic imaging (PPGI) is a widefield non-contact biophotonic technology able to remotely monitor cardiovascular function over anatomical areas. Though spatial context can provide increased physiological insight, existing PPGI systems rely on coarse spatial averaging with no anatomical priors for assessing arterial pulsatility. Here, we developed a continuous probabilistic pulsatility model for importance-weighted blood pulse waveform extraction. Using a data-driven approach, the model was constructed using a 23 participant sample with large demographic variation (11/12 female/male, age 11-60 years, BMI 16.4-35.1 kg$\cdot$m$^{-2}$). Using time-synchronized ground-truth waveforms, spatial correlation priors were computed and projected into a co-aligned importance-weighted Cartesian space. A modified Parzen-Rosenblatt kernel density estimation method was used to compute the continuous resolution-agnostic probabilistic pulsatility model. The model identified locations that consistently exhibited pulsatility across the sample. Blood pulse waveform signals extracted with the model exhibited significantly stronger temporal correlation ($W=35,p<0.01$) and spectral SNR ($W=31,p<0.01$) compared to uniform spatial averaging. Heart rate estimation was in strong agreement with true heart rate ($r^2=0.9619$, error $(μ,σ)=(0.52,1.69)$ bpm).
△ Less
Submitted 27 July, 2016;
originally announced July 2016.
-
A spectral-spatial fusion model for robust blood pulse waveform extraction in photoplethysmographic imaging
Authors:
Robert Amelard,
David A Clausi,
Alexander Wong
Abstract:
Photoplethysmographic imaging is a camera-based solution for non-contact cardiovascular monitoring from a distance. This technology enables monitoring in situations where contact-based devices may be problematic or infeasible, such as ambulatory, sleep, and multi-individual monitoring. However, extracting the blood pulse waveform signal is challenging due to the unknown mixture of relevant (pulsat…
▽ More
Photoplethysmographic imaging is a camera-based solution for non-contact cardiovascular monitoring from a distance. This technology enables monitoring in situations where contact-based devices may be problematic or infeasible, such as ambulatory, sleep, and multi-individual monitoring. However, extracting the blood pulse waveform signal is challenging due to the unknown mixture of relevant (pulsatile) and irrelevant pixels in the scene. Here, we design and implement a signal fusion framework, FusionPPG, for extracting a blood pulse waveform signal with strong temporal fidelity from a scene without requiring anatomical priors (e.g., facial tracking). The extraction problem is posed as a Bayesian least squares fusion problem, and solved using a novel probabilistic pulsatility model that incorporates both physiologically derived spectral and spatial waveform priors to identify pulsatility characteristics in the scene. Experimental results show statistically significantly improvements compared to the FaceMeanPPG method ($p<0.001$) and DistancePPG ($p<0.001$) methods. Heart rates predicted using FusionPPG correlated strongly with ground truth measurements ($r^2=0.9952$). FusionPPG was the only method able to assess cardiac arrhythmia via temporal analysis.
△ Less
Submitted 29 June, 2016;
originally announced June 2016.
-
Non-contact hemodynamic imaging reveals the jugular venous pulse waveform
Authors:
Robert Amelard,
Richard L Hughson,
Danielle K Greaves,
Kaylen J Pfisterer,
Jason Leung,
David A Clausi,
Alexander Wong
Abstract:
Cardiovascular monitoring is important to prevent diseases from progressing. The jugular venous pulse (JVP) waveform offers important clinical information about cardiac health, but is not routinely examined due to its invasive catheterisation procedure. Here, we demonstrate for the first time that the JVP can be consistently observed in a non-contact manner using a novel light-based photoplethysmo…
▽ More
Cardiovascular monitoring is important to prevent diseases from progressing. The jugular venous pulse (JVP) waveform offers important clinical information about cardiac health, but is not routinely examined due to its invasive catheterisation procedure. Here, we demonstrate for the first time that the JVP can be consistently observed in a non-contact manner using a novel light-based photoplethysmographic imaging system, coded hemodynamic imaging (CHI). While traditional monitoring methods measure the JVP at a single location, CHI's wide-field imaging capabilities were able to observe the jugular venous pulse's spatial flow profile for the first time. The important inflection points in the JVP were observed, meaning that cardiac abnormalities can be assessed through JVP distortions. CHI provides a new way to assess cardiac health through non-contact light-based JVP monitoring, and can be used in non-surgical environments for cardiac assessment.
△ Less
Submitted 21 April, 2016; v1 submitted 15 April, 2016;
originally announced April 2016.
-
Non-contact transmittance photoplethysmographic imaging (PPGI) for long-distance cardiovascular monitoring
Authors:
Robert Amelard,
Christian Scharfenberger,
Farnoud Kazemzadeh,
Kaylen J. Pfisterer,
Bill S. Lin,
Alexander Wong,
David A. Clausi
Abstract:
Photoplethysmography (PPG) devices are widely used for monitoring cardiovascular function. However, these devices require skin contact, which restrict their use to at-rest short-term monitoring using single-point measurements. Photoplethysmographic imaging (PPGI) has been recently proposed as a non-contact monitoring alternative by measuring blood pulse signals across a spatial region of interest.…
▽ More
Photoplethysmography (PPG) devices are widely used for monitoring cardiovascular function. However, these devices require skin contact, which restrict their use to at-rest short-term monitoring using single-point measurements. Photoplethysmographic imaging (PPGI) has been recently proposed as a non-contact monitoring alternative by measuring blood pulse signals across a spatial region of interest. Existing systems operate in reflectance mode, of which many are limited to short-distance monitoring and are prone to temporal changes in ambient illumination. This paper is the first study to investigate the feasibility of long-distance non-contact cardiovascular monitoring at the supermeter level using transmittance PPGI. For this purpose, a novel PPGI system was designed at the hardware and software level using ambient correction via temporally coded illumination (TCI) and signal processing for PPGI signal extraction. Experimental results show that the processing steps yield a substantially more pulsatile PPGI signal than the raw acquired signal, resulting in statistically significant increases in correlation to ground-truth PPG in both short- ($p \in [<0.0001, 0.040]$) and long-distance ($p \in [<0.0001, 0.056]$) monitoring. The results support the hypothesis that long-distance heart rate monitoring is feasible using transmittance PPGI, allowing for new possibilities of monitoring cardiovascular function in a non-contact manner.
△ Less
Submitted 23 March, 2015;
originally announced March 2015.
