-
Automatic Tuberculosis and COVID-19 cough classification using deep learning
Authors:
Madhurananda Pahar,
Marisa Klopper,
Byron Reeve,
Rob Warren,
Grant Theron,
Andreas Diacon,
Thomas Niesler
Abstract:
We present a deep learning based automatic cough classifier which can discriminate tuberculosis (TB) coughs from COVID-19 coughs and healthy coughs. Both TB and COVID-19 are respiratory diseases, contagious, have cough as a predominant symptom and claim thousands of lives each year. The cough audio recordings were collected at both indoor and outdoor settings and also uploaded using smartphones fr…
▽ More
We present a deep learning based automatic cough classifier which can discriminate tuberculosis (TB) coughs from COVID-19 coughs and healthy coughs. Both TB and COVID-19 are respiratory diseases, contagious, have cough as a predominant symptom and claim thousands of lives each year. The cough audio recordings were collected at both indoor and outdoor settings and also uploaded using smartphones from subjects around the globe, thus containing various levels of noise. This cough data include 1.68 hours of TB coughs, 18.54 minutes of COVID-19 coughs and 1.69 hours of healthy coughs from 47 TB patients, 229 COVID-19 patients and 1498 healthy patients and were used to train and evaluate a CNN, LSTM and Resnet50. These three deep architectures were also pre-trained on 2.14 hours of sneeze, 2.91 hours of speech and 2.79 hours of noise for improved performance. The class-imbalance in our dataset was addressed by using SMOTE data balancing technique and using performance metrics such as F1-score and AUC. Our study shows that the highest F1-scores of 0.9259 and 0.8631 have been achieved from a pre-trained Resnet50 for two-class (TB vs COVID-19) and three-class (TB vs COVID-19 vs healthy) cough classification tasks, respectively. The application of deep transfer learning has improved the classifiers' performance and makes them more robust as they generalise better over the cross-validation folds. Their performances exceed the TB triage test requirements set by the world health organisation (WHO). The features producing the best performance contain higher order of MFCCs suggesting that the differences between TB and COVID-19 coughs are not perceivable by the human ear. This type of cough audio classification is non-contact, cost-effective and can easily be deployed on a smartphone, thus it can be an excellent tool for both TB and COVID-19 screening.
△ Less
Submitted 10 September, 2022; v1 submitted 11 May, 2022;
originally announced May 2022.
-
Accelerometer-based Bed Occupancy Detection for Automatic, Non-invasive Long-term Cough Monitoring
Authors:
Madhurananda Pahar,
Igor Miranda,
Andreas Diacon,
Thomas Niesler
Abstract:
We present a new machine learning based bed-occupancy detection system that uses the accelerometer signal captured by a bed-attached consumer smartphone. Automatic bed-occupancy detection is necessary for automatic long-term cough monitoring, since the time which the monitored patient occupies the bed is required to accurately calculate a cough rate. Accelerometer measurements are more cost effect…
▽ More
We present a new machine learning based bed-occupancy detection system that uses the accelerometer signal captured by a bed-attached consumer smartphone. Automatic bed-occupancy detection is necessary for automatic long-term cough monitoring, since the time which the monitored patient occupies the bed is required to accurately calculate a cough rate. Accelerometer measurements are more cost effective and less intrusive than alternatives such as video monitoring or pressure sensors. A 249-hour dataset of manually-labelled acceleration signals gathered from seven patients undergoing treatment for tuberculosis (TB) was compiled for experimentation. These signals are characterised by brief activity bursts interspersed with long periods of little or no activity, even when the bed is occupied. To process them effectively, we propose an architecture consisting of three interconnected components. An occupancy-change detector locates instances at which bed occupancy is likely to have changed, an occupancy-interval detector classifies periods between detected occupancy changes and an occupancy-state detector corrects falsely-identified occupancy changes. Using long short-term memory (LSTM) networks, this architecture was demonstrated to achieve an AUC of 0.94. When integrated into a complete cough monitoring system, the daily cough rate of a patient undergoing TB treatment was determined over a period of 14 days. As the colony forming unit (CFU) counts decreased and the time to positivity (TPP) increased, the measured cough rate decreased, indicating effective TB treatment. This provides a first indication that automatic cough monitoring based on bed-mounted accelerometer measurements may present a non-invasive, non-intrusive and cost-effective means of monitoring long-term recovery of TB patients.
△ Less
Submitted 13 March, 2022; v1 submitted 8 February, 2022;
originally announced February 2022.
-
Wake-Cough: cough spotting and cougher identification for personalised long-term cough monitoring
Authors:
Madhurananda Pahar,
Marisa Klopper,
Byron Reeve,
Rob Warren,
Grant Theron,
Andreas Diacon,
Thomas Niesler
Abstract:
We present `wake-cough', an application of wake-word spotting to coughs using a Resnet50 and the identification of coughers using i-vectors, for the purpose of a long-term, personalised cough monitoring system. Coughs, recorded in a quiet (73$\pm$5 dB) and noisy (34$\pm$17 dB) environment, were used to extract i-vectors, x-vectors and d-vectors, used as features to the classifiers. The system achi…
▽ More
We present `wake-cough', an application of wake-word spotting to coughs using a Resnet50 and the identification of coughers using i-vectors, for the purpose of a long-term, personalised cough monitoring system. Coughs, recorded in a quiet (73$\pm$5 dB) and noisy (34$\pm$17 dB) environment, were used to extract i-vectors, x-vectors and d-vectors, used as features to the classifiers. The system achieves 90.02\% accuracy when using an MLP to discriminate between 51 coughers using 2-sec long cough segments in the noisy environment. When discriminating between 5 and 14 coughers using longer (100 sec) segments in the quiet environment, this accuracy improves to 99.78% and 98.39% respectively. Unlike speech, i-vectors outperform x-vectors and d-vectors in identifying coughers. These coughs were added as an extra class to the Google Speech Commands dataset and features were extracted by preserving the end-to-end time-domain information in a trigger phrase. The highest accuracy of 88.58% is achieved in spotting coughs among 35 other trigger phrases using a Resnet50. Thus, wake-cough represents a personalised, non-intrusive cough monitoring system, which is power-efficient as on-device wake-word detection can keep a smartphone-based monitoring device mostly dormant. This makes wake-cough extremely attractive in multi-bed ward environments to monitor patients' long-term recovery from lung ailments such as tuberculosis (TB) and COVID-19.
△ Less
Submitted 10 September, 2022; v1 submitted 7 October, 2021;
originally announced October 2021.
-
Automatic non-invasive Cough Detection based on Accelerometer and Audio Signals
Authors:
Madhurananda Pahar,
Igor Miranda,
Andreas Diacon,
Thomas Niesler
Abstract:
We present an automatic non-invasive way of detecting cough events based on both accelerometer and audio signals.
The acceleration signals are captured by a smartphone firmly attached to the patient's bed, using its integrated accelerometer.
The audio signals are captured simultaneously by the same smartphone using an external microphone.
We have compiled a manually-annotated dataset contain…
▽ More
We present an automatic non-invasive way of detecting cough events based on both accelerometer and audio signals.
The acceleration signals are captured by a smartphone firmly attached to the patient's bed, using its integrated accelerometer.
The audio signals are captured simultaneously by the same smartphone using an external microphone.
We have compiled a manually-annotated dataset containing such simultaneously-captured acceleration and audio signals for approximately 6000 cough and 68000 non-cough events from 14 adult male patients in a tuberculosis clinic.
LR, SVM and MLP are evaluated as baseline classifiers and compared with deep architectures such as CNN, LSTM, and Resnet50 using a leave-one-out cross-validation scheme.
We find that the studied classifiers can use either acceleration or audio signals to distinguish between coughing and other activities including sneezing, throat-clearing, and movement on the bed with high accuracy.
However, in all cases, the deep neural networks outperform the shallow classifiers by a clear margin and the Resnet50 offers the best performance by achieving an AUC exceeding 0.98 and 0.99 for acceleration and audio signals respectively.
While audio-based classification consistently offers a better performance than acceleration-based classification, we observe that the difference is very small for the best systems.
Since the acceleration signal requires less processing power, and since the need to record audio is sidestepped and thus privacy is inherently secured, and since the recording device is attached to the bed and not worn, an accelerometer-based highly accurate non-invasive cough detector may represent a more convenient and readily accepted method in long-term cough monitoring.
△ Less
Submitted 31 August, 2021;
originally announced September 2021.
-
COVID-19 Detection in Cough, Breath and Speech using Deep Transfer Learning and Bottleneck Features
Authors:
Madhurananda Pahar,
Marisa Klopper,
Robin Warren,
Thomas Niesler
Abstract:
We present an experimental investigation into the effectiveness of transfer learning and bottleneck feature extraction in detecting COVID-19 from audio recordings of cough, breath and speech.
This type of screening is non-contact, does not require specialist medical expertise or laboratory facilities and can be deployed on inexpensive consumer hardware.
We use datasets that contain recordings…
▽ More
We present an experimental investigation into the effectiveness of transfer learning and bottleneck feature extraction in detecting COVID-19 from audio recordings of cough, breath and speech.
This type of screening is non-contact, does not require specialist medical expertise or laboratory facilities and can be deployed on inexpensive consumer hardware.
We use datasets that contain recordings of coughing, sneezing, speech and other noises, but do not contain COVID-19 labels, to pre-train three deep neural networks: a CNN, an LSTM and a Resnet50.
These pre-trained networks are subsequently either fine-tuned using smaller datasets of coughing with COVID-19 labels in the process of transfer learning, or are used as bottleneck feature extractors.
Results show that a Resnet50 classifier trained by this transfer learning process delivers optimal or near-optimal performance across all datasets achieving areas under the receiver operating characteristic (ROC AUC) of 0.98, 0.94 and 0.92 respectively for all three sound classes (coughs, breaths and speech).
This indicates that coughs carry the strongest COVID-19 signature, followed by breath and speech.
Our results also show that applying transfer learning and extracting bottleneck features using the larger datasets without COVID-19 labels led not only to improve performance, but also to minimise the standard deviation of the classifier AUCs among the outer folds of the leave-$p$-out cross-validation, indicating better generalisation.
We conclude that deep transfer learning and bottleneck feature extraction can improve COVID-19 cough, breath and speech audio classification, yielding automatic classifiers with higher accuracy.
△ Less
Submitted 17 August, 2021; v1 submitted 2 April, 2021;
originally announced April 2021.
-
Automatic Cough Classification for Tuberculosis Screening in a Real-World Environment
Authors:
Madhurananda Pahar,
Marisa Klopper,
Byron Reeve,
Grant Theron,
Rob Warren,
Thomas Niesler
Abstract:
Objective: The automatic discrimination between the coughing sounds produced by patients with tuberculosis (TB) and those produced by patients with other lung ailments.
Approach: We present experiments based on a dataset of 1358 forced cough recordings obtained in a develo**-world clinic from 16 patients with confirmed active pulmonary TB and 35 patients suffering from respiratory conditions s…
▽ More
Objective: The automatic discrimination between the coughing sounds produced by patients with tuberculosis (TB) and those produced by patients with other lung ailments.
Approach: We present experiments based on a dataset of 1358 forced cough recordings obtained in a develo**-world clinic from 16 patients with confirmed active pulmonary TB and 35 patients suffering from respiratory conditions suggestive of TB but confirmed to be TB negative. Using nested cross-validation, we have trained and evaluated five machine learning classifiers: logistic regression (LR), support vector machines (SVM), k-nearest neighbour (KNN), multilayer perceptrons (MLP) and convolutional neural networks (CNN).
Main Results: Although classification is possible in all cases, the best performance is achieved using LR. In combination with feature selection by sequential forward selection (SFS), our best LR system achieves an area under the ROC curve (AUC) of 0.94 using 23 features selected from a set of 78 high-resolution mel-frequency cepstral coefficients (MFCCs). This system achieves a sensitivity of 93\% at a specificity of 95\% and thus exceeds the 90\% sensitivity at 70\% specificity specification considered by the World Health Organisation (WHO) as a minimal requirement for a community-based TB triage test.
Significance: The automatic classification of cough audio sounds, when applied to symptomatic patients requiring investigation for TB, can meet the WHO triage specifications for the identification of patients who should undergo expensive molecular downstream testing. This makes it a promising and viable means of low cost, easily deployable frontline screening for TB, which can benefit especially develo** countries with a heavy TB burden.
△ Less
Submitted 17 October, 2021; v1 submitted 23 March, 2021;
originally announced March 2021.
-
Deep Neural Network based Cough Detection using Bed-mounted Accelerometer Measurements
Authors:
Madhurananda Pahar,
Igor Miranda,
Andreas Diacon,
Thomas Niesler
Abstract:
We have performed cough detection based on measurements from an accelerometer attached to the patient's bed. This form of monitoring is less intrusive than body-attached accelerometer sensors, and sidesteps privacy concerns encountered when using audio for cough detection. For our experiments, we have compiled a manually-annotated dataset containing the acceleration signals of approximately 6000 c…
▽ More
We have performed cough detection based on measurements from an accelerometer attached to the patient's bed. This form of monitoring is less intrusive than body-attached accelerometer sensors, and sidesteps privacy concerns encountered when using audio for cough detection. For our experiments, we have compiled a manually-annotated dataset containing the acceleration signals of approximately 6000 cough and 68000 non-cough events from 14 adult male patients in a tuberculosis clinic. As classifiers, we have considered convolutional neural networks (CNN), long-short-term-memory (LSTM) networks, and a residual neural network (Resnet50). We find that all classifiers are able to distinguish between the acceleration signals due to coughing and those due to other activities including sneezing, throat-clearing and movement in the bed with high accuracy. The Resnet50 performs the best, achieving an area under the ROC curve (AUC) exceeding 0.98 in cross-validation experiments. We conclude that high-accuracy cough monitoring based only on measurements from the accelerometer in a consumer smartphone is possible. Since the need to gather audio is avoided and therefore privacy is inherently protected, and since the accelerometer is attached to the bed and not worn, this form of monitoring may represent a more convenient and readily accepted method of long-term patient cough monitoring.
△ Less
Submitted 9 February, 2021;
originally announced February 2021.
-
COVID-19 Cough Classification using Machine Learning and Global Smartphone Recordings
Authors:
Madhurananda Pahar,
Marisa Klopper,
Robin Warren,
Thomas Niesler
Abstract:
We present a machine learning based COVID-19 cough classifier which can discriminate COVID-19 positive coughs from both COVID-19 negative and healthy coughs recorded on a smartphone. This type of screening is non-contact, easy to apply, and can reduce the workload in testing centres as well as limit transmission by recommending early self-isolation to those who have a cough suggestive of COVID-19.…
▽ More
We present a machine learning based COVID-19 cough classifier which can discriminate COVID-19 positive coughs from both COVID-19 negative and healthy coughs recorded on a smartphone. This type of screening is non-contact, easy to apply, and can reduce the workload in testing centres as well as limit transmission by recommending early self-isolation to those who have a cough suggestive of COVID-19. The datasets used in this study include subjects from all six continents and contain both forced and natural coughs, indicating that the approach is widely applicable. The publicly available Coswara dataset contains 92 COVID-19 positive and 1079 healthy subjects, while the second smaller dataset was collected mostly in South Africa and contains 18 COVID-19 positive and 26 COVID-19 negative subjects who have undergone a SARS-CoV laboratory test. Both datasets indicate that COVID-19 positive coughs are 15\%-20\% shorter than non-COVID coughs. Dataset skew was addressed by applying the synthetic minority oversampling technique (SMOTE). A leave-$p$-out cross-validation scheme was used to train and evaluate seven machine learning classifiers: LR, KNN, SVM, MLP, CNN, LSTM and Resnet50. Our results show that although all classifiers were able to identify COVID-19 coughs, the best performance was exhibited by the Resnet50 classifier, which was best able to discriminate between the COVID-19 positive and the healthy coughs with an area under the ROC curve (AUC) of 0.98. An LSTM classifier was best able to discriminate between the COVID-19 positive and COVID-19 negative coughs, with an AUC of 0.94 after selecting the best 13 features from a sequential forward selection (SFS). Since this type of cough audio classification is cost-effective and easy to deploy, it is potentially a useful and viable means of non-contact COVID-19 screening.
△ Less
Submitted 14 June, 2021; v1 submitted 2 December, 2020;
originally announced December 2020.
