-
TRIP: Trainable Region-of-Interest Prediction for Hardware-Efficient Neuromorphic Processing on Event-based Vision
Authors:
Cina Arjmand,
Yingfu Xu,
Kevin Shidqi,
Alexandra F. Dobrita,
Kanishkan Vadivel,
Paul Detterer,
Manolis Sifalakis,
Amirreza Yousefzadeh,
Guangzhi Tang
Abstract:
Neuromorphic processors are well-suited for efficiently handling sparse events from event-based cameras. However, they face significant challenges in the growth of computing demand and hardware costs as the input resolution increases. This paper proposes the Trainable Region-of-Interest Prediction (TRIP), the first hardware-efficient hard attention framework for event-based vision processing on a…
▽ More
Neuromorphic processors are well-suited for efficiently handling sparse events from event-based cameras. However, they face significant challenges in the growth of computing demand and hardware costs as the input resolution increases. This paper proposes the Trainable Region-of-Interest Prediction (TRIP), the first hardware-efficient hard attention framework for event-based vision processing on a neuromorphic processor. Our TRIP framework actively produces low-resolution Region-of-Interest (ROIs) for efficient and accurate classification. The framework exploits sparse events' inherent low information density to reduce the overhead of ROI prediction. We introduced extensive hardware-aware optimizations for TRIP and implemented the hardware-optimized algorithm on the SENECA neuromorphic processor. We utilized multiple event-based classification datasets for evaluation. Our approach achieves state-of-the-art accuracies in all datasets and produces reasonable ROIs with varying locations and sizes. On the DvsGesture dataset, our solution requires 46x less computation than the state-of-the-art while achieving higher accuracy. Furthermore, TRIP enables more than 2x latency and energy improvements on the SENECA neuromorphic processor compared to the conventional solution.
△ Less
Submitted 25 June, 2024;
originally announced June 2024.
-
Deep Imbalanced Regression to Estimate Vascular Age from PPG Data: a Novel Digital Biomarker for Cardiovascular Health
Authors:
Guangkun Nie,
Qinghao Zhao,
Gongzheng Tang,
Jun Li,
Shenda Hong
Abstract:
Photoplethysmography (PPG) is emerging as a crucial tool for monitoring human hemodynamics, with recent studies highlighting its potential in assessing vascular aging through deep learning. However, real-world age distributions are often imbalanced, posing significant challenges for deep learning models. In this paper, we introduce a novel, simple, and effective loss function named the Dist Loss t…
▽ More
Photoplethysmography (PPG) is emerging as a crucial tool for monitoring human hemodynamics, with recent studies highlighting its potential in assessing vascular aging through deep learning. However, real-world age distributions are often imbalanced, posing significant challenges for deep learning models. In this paper, we introduce a novel, simple, and effective loss function named the Dist Loss to address deep imbalanced regression tasks. We trained a one-dimensional convolutional neural network (Net1D) incorporating the Dist Loss on the extensive UK Biobank dataset (n=502,389) to estimate vascular age from PPG signals and validate its efficacy in characterizing cardiovascular health. The model's performance was validated on a 40% held-out test set, achieving state-of-the-art results, especially in regions with small sample sizes. Furthermore, we divided the population into three subgroups based on the difference between predicted vascular age and chronological age: less than -10 years, between -10 and 10 years, and greater than 10 years. We analyzed the relationship between predicted vascular age and several cardiovascular events over a follow-up period of up to 10 years, including death, coronary heart disease, and heart failure. Our results indicate that the predicted vascular age has significant potential to reflect an individual's cardiovascular health status. Our code will be available at https://github.com/Ngk03/AI-vascular-age.
△ Less
Submitted 21 June, 2024;
originally announced June 2024.
-
A Review of Deep Learning Methods for Photoplethysmography Data
Authors:
Guangkun Nie,
Jiabao Zhu,
Gongzheng Tang,
Deyun Zhang,
Shijia Geng,
Qinghao Zhao,
Shenda Hong
Abstract:
Photoplethysmography (PPG) is a highly promising device due to its advantages in portability, user-friendly operation, and non-invasive capabilities to measure a wide range of physiological information. Recent advancements in deep learning have demonstrated remarkable outcomes by leveraging PPG signals for tasks related to personal health management and other multifaceted applications. In this rev…
▽ More
Photoplethysmography (PPG) is a highly promising device due to its advantages in portability, user-friendly operation, and non-invasive capabilities to measure a wide range of physiological information. Recent advancements in deep learning have demonstrated remarkable outcomes by leveraging PPG signals for tasks related to personal health management and other multifaceted applications. In this review, we systematically reviewed papers that applied deep learning models to process PPG data between January 1st of 2017 and July 31st of 2023 from Google Scholar, PubMed and Dimensions. Each paper is analyzed from three key perspectives: tasks, models, and data. We finally extracted 193 papers where different deep learning frameworks were used to process PPG signals. Based on the tasks addressed in these papers, we categorized them into two major groups: medical-related, and non-medical-related. The medical-related tasks were further divided into seven subgroups, including blood pressure analysis, cardiovascular monitoring and diagnosis, sleep health, mental health, respiratory monitoring and analysis, blood glucose analysis, as well as others. The non-medical-related tasks were divided into four subgroups, which encompass signal processing, biometric identification, electrocardiogram reconstruction, and human activity recognition. In conclusion, significant progress has been made in the field of using deep learning methods to process PPG data recently. This allows for a more thorough exploration and utilization of the information contained in PPG signals. However, challenges remain, such as limited quantity and quality of publicly available databases, a lack of effective validation in real-world scenarios, and concerns about the interpretability, scalability, and complexity of deep learning models. Moreover, there are still emerging research areas that require further investigation.
△ Less
Submitted 23 January, 2024;
originally announced January 2024.
-
Open the box of digital neuromorphic processor: Towards effective algorithm-hardware co-design
Authors:
Guangzhi Tang,
Ali Safa,
Kevin Shidqi,
Paul Detterer,
Stefano Traferro,
Mario Konijnenburg,
Manolis Sifalakis,
Gert-Jan van Schaik,
Amirreza Yousefzadeh
Abstract:
Sparse and event-driven spiking neural network (SNN) algorithms are the ideal candidate solution for energy-efficient edge computing. Yet, with the growing complexity of SNN algorithms, it isn't easy to properly benchmark and optimize their computational cost without hardware in the loop. Although digital neuromorphic processors have been widely adopted to benchmark SNN algorithms, their black-box…
▽ More
Sparse and event-driven spiking neural network (SNN) algorithms are the ideal candidate solution for energy-efficient edge computing. Yet, with the growing complexity of SNN algorithms, it isn't easy to properly benchmark and optimize their computational cost without hardware in the loop. Although digital neuromorphic processors have been widely adopted to benchmark SNN algorithms, their black-box nature is problematic for algorithm-hardware co-optimization. In this work, we open the black box of the digital neuromorphic processor for algorithm designers by presenting the neuron processing instruction set and detailed energy consumption of the SENeCA neuromorphic architecture. For convenient benchmarking and optimization, we provide the energy cost of the essential neuromorphic components in SENeCA, including neuron models and learning rules. Moreover, we exploit the SENeCA's hierarchical memory and exhibit an advantage over existing neuromorphic processors. We show the energy efficiency of SNN algorithms for video processing and online learning, and demonstrate the potential of our work for optimizing algorithm designs. Overall, we present a practical approach to enable algorithm designers to accurately benchmark SNN algorithms and pave the way towards effective algorithm-hardware co-design.
△ Less
Submitted 27 March, 2023;
originally announced March 2023.
-
Parallel Multi-Extended State Observers based {ADRC} with Application to High-Speed Precision Motion Stage
Authors:
Guojie Tang,
Wenchao Xue,
Hao Peng,
Yanlong Zhao,
Zhijun Yang
Abstract:
In this paper, the parallel multi-extended state observers (ESOs) based active disturbance rejection control approach is proposed to achieve desired tracking performance by automatically selecting the estimation values leading to the least tracking error. First, the relationship between the estimation error of ESO and the tracking error of output is quantitatively studied for single ESO with gener…
▽ More
In this paper, the parallel multi-extended state observers (ESOs) based active disturbance rejection control approach is proposed to achieve desired tracking performance by automatically selecting the estimation values leading to the least tracking error. First, the relationship between the estimation error of ESO and the tracking error of output is quantitatively studied for single ESO with general order. In particular, the algorithm for calculating the tracking error caused by single ESO's estimation error is constructed. Moreover, by timely evaluating the least tracking error caused by different ESOs, a novel switching ADRC approach with parallel multi-ESOs is proposed. In addition, the stability of the algorithm is rigorously proved. Furthermore, the proposed ADRC is applied to the high-speed precision motion stage which has large nonlinear uncertainties and elastic deformation disturbances near the dead zone of friction. The experimental results show that the parallel multi-ESOs based ADRC has higher tracking performance than the traditional single ESO based ADRC.
△ Less
Submitted 17 January, 2023;
originally announced January 2023.
-
Separation-Free Spectral Super-Resolution via Convex Optimization
Authors:
Zai Yang,
Yi-Lin Mo,
Gongguo Tang,
Zongben Xu
Abstract:
Atomic norm methods have recently been proposed for spectral super-resolution with flexibility in dealing with missing data and miscellaneous noises. A notorious drawback of these convex optimization methods however is their lower resolution in the high signal-to-noise (SNR) regime as compared to conventional methods such as ESPRIT. In this paper, we devise a simple weighting scheme in existing at…
▽ More
Atomic norm methods have recently been proposed for spectral super-resolution with flexibility in dealing with missing data and miscellaneous noises. A notorious drawback of these convex optimization methods however is their lower resolution in the high signal-to-noise (SNR) regime as compared to conventional methods such as ESPRIT. In this paper, we devise a simple weighting scheme in existing atomic norm methods and show that the resolution of the resulting convex optimization method can be made arbitrarily high in the absence of noise, achieving the so-called separation-free super-resolution. This is proved by a novel, kernel-free construction of the dual certificate whose existence guarantees exact super-resolution using the proposed method. Numerical results corroborating our analysis are provided.
△ Less
Submitted 28 November, 2022;
originally announced November 2022.
-
See Blue Sky: Deep Image Dehaze Using Paired and Unpaired Training Images
Authors:
Xiaoyan Zhang,
Gaoyang Tang,
Yingying Zhu,
Qi Tian
Abstract:
The issue of image haze removal has attracted wide attention in recent years. However, most existing haze removal methods cannot restore the scene with clear blue sky, since the color and texture information of the object in the original haze image is insufficient. To remedy this, we propose a cycle generative adversarial network to construct a novel end-to-end image dehaze model. We adopt outdoor…
▽ More
The issue of image haze removal has attracted wide attention in recent years. However, most existing haze removal methods cannot restore the scene with clear blue sky, since the color and texture information of the object in the original haze image is insufficient. To remedy this, we propose a cycle generative adversarial network to construct a novel end-to-end image dehaze model. We adopt outdoor image datasets to train our model, which includes a set of real-world unpaired image dataset and a set of paired image dataset to ensure that the generated images are close to the real scene. Based on the cycle structure, our model adds four different kinds of loss function to constrain the effect including adversarial loss, cycle consistency loss, photorealism loss and paired L1 loss. These four constraints can improve the overall quality of such degraded images for better visual appeal and ensure reconstruction of images to keep from distortion. The proposed model could remove the haze of images and also restore the sky of images to be clean and blue (like captured in a sunny weather).
△ Less
Submitted 14 October, 2022;
originally announced October 2022.
-
Quadruped Guidance Robot for the Visually Impaired: A Comfort-Based Approach
Authors:
Yanbo Chen,
Zhengzhe Xu,
Zhuozhu Jian,
Gengpan Tang,
Yunong Yangli,
Anxing Xiao,
Xueqian Wang,
Bin Liang
Abstract:
Guidance robots that can guide people and avoid various obstacles, could potentially be owned by more visually impaired people at a fairly low cost. Most of the previous guidance robots for the visually impaired ignored the human response behavior and comfort, treating the human as an appendage dragged by the robot, which can lead to imprecise guidance of the human and sudden changes in the tracti…
▽ More
Guidance robots that can guide people and avoid various obstacles, could potentially be owned by more visually impaired people at a fairly low cost. Most of the previous guidance robots for the visually impaired ignored the human response behavior and comfort, treating the human as an appendage dragged by the robot, which can lead to imprecise guidance of the human and sudden changes in the traction force experienced by the human. In this paper, we propose a novel quadruped guidance robot system with a comfort-based concept. We design a controllable traction device that can adjust the length and force between human and robot to ensure comfort. To allow the human to be guided safely and comfortably to the target position in complex environments, our proposed human motion planner can plan the traction force with the force-based human motion model. To track the planned force, we also propose a robot motion planner that can generate the specific robot motion command and design the force control device. Our system has been deployed on Unitree Laikago quadrupedal platform and validated in real-world scenarios.
△ Less
Submitted 23 June, 2023; v1 submitted 8 March, 2022;
originally announced March 2022.
-
Deep interval prediction model with gradient descend optimization method for short-term wind power prediction
Authors:
Chaoshun Li,
Geng Tang,
Xiaoming Xue,
Xinbiao Chen,
Ruoheng Wang,
Chu Zhang
Abstract:
The application of wind power interval prediction for power systems attempts to give more comprehensive support to dispatchers and operators of the grid. Lower upper bound estimation (LUBE) method is widely applied in interval prediction. However, the existing LUBE approaches are trained by meta-heuristic optimization, which is either time-consuming or show poor effect when the LUBE model is compl…
▽ More
The application of wind power interval prediction for power systems attempts to give more comprehensive support to dispatchers and operators of the grid. Lower upper bound estimation (LUBE) method is widely applied in interval prediction. However, the existing LUBE approaches are trained by meta-heuristic optimization, which is either time-consuming or show poor effect when the LUBE model is complex. In this paper, a deep interval prediction method is designed in the framework of LUBE and an efficient gradient descend (GD) training approach is proposed to train the LUBE model. In this method, the long short-term memory is selected as a representative to show the modelling approach. The architecture of the proposed model consists of three parts, namely the long short-term memory module, the fully connected layers and the rank ordered module. Two loss functions are specially designed for implementing the GD training method based on the root mean square back propagation algorithm. To verify the performance of the proposed model, conventional LUBE models, as well as popular statistic interval prediction models are compared in numerical experiments. The results show that the proposed approach performs best in terms of effectiveness and efficiency with average 45% promotion in quality of prediction interval and 66% reduction of time consumptions compared to traditional LUBE models.
△ Less
Submitted 19 November, 2019;
originally announced November 2019.
-
HeartFit: An Accurate Platform for Heart Murmur Diagnosis Utilizing Deep Learning
Authors:
Ankit Gupta,
George Tang,
Sylesh Suresh
Abstract:
Cardiovascular disease (CD) is the number one leading cause of death worldwide, accounting for more than 17 million deaths in 2015. Critical indicators of CD include heart murmurs, intense sounds emitted by the heart during periods of irregular blood flow. Current diagnosis of heart murmurs relies on echocardiography (ECHO), which costs thousands of dollars and medical professionals to analyze the…
▽ More
Cardiovascular disease (CD) is the number one leading cause of death worldwide, accounting for more than 17 million deaths in 2015. Critical indicators of CD include heart murmurs, intense sounds emitted by the heart during periods of irregular blood flow. Current diagnosis of heart murmurs relies on echocardiography (ECHO), which costs thousands of dollars and medical professionals to analyze the results, making it very unsuitable for areas with inadequate medical facilities. Thus, there is a need for an accessible alternative. Based on a simple interface and deep learning, HeartFit allows users to administer diagnoses themselves. An inexpensive, custom designed stethoscope in conjunction with a mobile application allows users to record and upload audio of their heart to a database. Using a deep learning network architecture, the database classifies the audio and returns the diagnosis to the user. The model consists of a deep recurrent convolutional neural network trained on 300 prelabeled heartbeat audio samples. After the model was validated on a previously unseen set of 100 heartbeat audio samples, it achieved a f beta score of 0.9545 and an accuracy of 95.5 percent. This value exceeds that of clinical examination accuracy, which is around 83 percent to 91 percent and costs orders of magnitude less than ECHO, demonstrating the effectiveness of the HeartFit platform. Through the platform, users can obtain immediate, accurate diagnosis of heart murmurs without any professional medical assistance, revolutionizing how we combat CD.
△ Less
Submitted 30 December, 2020; v1 submitted 24 July, 2019;
originally announced July 2019.
-
Development of a Real-time Indoor Location System using Bluetooth Low Energy Technology and Deep Learning to Facilitate Clinical Applications
Authors:
Guanglin Tang,
Yulong Yan,
Chenyang Shen,
Xun Jia,
Meyer Zinn,
Zipalkumar Trivedi,
Alicia Yingling,
Kenneth Westover,
Steve Jiang
Abstract:
An indoor, real-time location system (RTLS) can benefit both hospitals and patients by improving clinical efficiency through data-driven optimization of procedures. Bluetooth-based RTLS systems are cost-effective but lack accuracy and robustness because Bluetooth signal strength is subject to fluctuation. We developed a machine learning-based solution using a Long Short-Term Memory (LSTM) network…
▽ More
An indoor, real-time location system (RTLS) can benefit both hospitals and patients by improving clinical efficiency through data-driven optimization of procedures. Bluetooth-based RTLS systems are cost-effective but lack accuracy and robustness because Bluetooth signal strength is subject to fluctuation. We developed a machine learning-based solution using a Long Short-Term Memory (LSTM) network followed by a Multilayer Perceptron classifier and a posterior constraint algorithm to improve RTLS performance. Training and validation datasets showed that most machine learning models perform well in classifying individual location zones, although LSTM was most reliable. However, when faced with data indicating cross-zone trajectories, all models showed erratic zone switching. Thus, we implemented a history-based posterior constraint algorithm to reduce the variability in exchange for a slight decrease in responsiveness. This network increases robustness at the expense of latency. When latency is less of a concern, we computed the latency-corrected accuracy which is 100% for our testing data, significantly improved from LSTM without constraint which is 96.2%. The balance between robustness and responsiveness can be considered and adjusted on a case-by-case basis, according to the specific needs of downstream clinical applications. This system was deployed and validated in an academic medical center. Industry best practices enabled system scaling without substantial compromises to performance or cost.
△ Less
Submitted 26 March, 2020; v1 submitted 24 July, 2019;
originally announced July 2019.
-
Vandermonde Factorization of Hankel Matrix for Complex Exponential Signal Recovery -- Application in Fast NMR Spectroscopy
Authors:
Jiaxi Ying,
Jian-Feng Cai,
Di Guo,
Gongguo Tang,
Zhong Chen,
Xiaobo Qu
Abstract:
Many signals are modeled as a superposition of exponential functions in spectroscopy of chemistry, biology and medical imaging. This paper studies the problem of recovering exponential signals from a random subset of samples. We exploit the Vandermonde structure of the Hankel matrix formed by the exponential signal and formulate signal recovery as Hankel matrix completion with Vandermonde factoriz…
▽ More
Many signals are modeled as a superposition of exponential functions in spectroscopy of chemistry, biology and medical imaging. This paper studies the problem of recovering exponential signals from a random subset of samples. We exploit the Vandermonde structure of the Hankel matrix formed by the exponential signal and formulate signal recovery as Hankel matrix completion with Vandermonde factorization (HVaF). A numerical algorithm is developed to solve the proposed model and its sequence convergence is analyzed theoretically. Experiments on synthetic data demonstrate that HVaF succeeds over a wider regime than the state-of-the-art nuclear-normminimization-based Hankel matrix completion method, while has a less restriction on frequency separation than the state-of-the-art atomic norm minimization and fast iterative hard thresholding methods. The effectiveness of HVaF is further validated on biological magnetic resonance spectroscopy data.
△ Less
Submitted 3 September, 2018;
originally announced September 2018.
-
Robust Tracking Guidance for Zero Propellant Maneuver
Authors:
Sheng Zhang,
Qian Zhao,
Hai-bing Huang,
Guo-** Tang
Abstract:
The Zero Propellant Maneuver (ZPM) maneuvers the space station by large angle, utilizing the Control Momentum Gyroscopes (CMGs) only. A robust tracking guidance strategy is proposed to enhance its performance. It is distinguished from the traditional trajectory tracking guidance in that the reference trajectory is adjusted on-line, under the inspiration of eliminating the discrepancy on the total…
▽ More
The Zero Propellant Maneuver (ZPM) maneuvers the space station by large angle, utilizing the Control Momentum Gyroscopes (CMGs) only. A robust tracking guidance strategy is proposed to enhance its performance. It is distinguished from the traditional trajectory tracking guidance in that the reference trajectory is adjusted on-line, under the inspiration of eliminating the discrepancy on the total angular momentum of the space station system. The Lyapunov controller is developed to adjust the attitude trajectory and further redesigned for a better performance based on an interesting physical phenomenon, which is taken advantage of by coupling the components of state vector. The adjusted trajectory is then tracked to reach the target states of maneuver. Simulations results show that the disturbance effects arising from initial state errors, parameter uncertainty and modeling errors are attenuated or even eliminated, which verifies the robustness of the guidance strategy.
△ Less
Submitted 11 June, 2017;
originally announced June 2017.
