-
HarmonICA: Neural non-stationarity correction and source separation for motor neuron interfaces
Authors:
Alexander Kenneth Clarke,
Agnese Grison,
Irene Mendez Guerra,
Pranav Mamidanna,
Shihan Ma,
Silvia Muceli,
Dario Farina
Abstract:
A major outstanding problem when interfacing with spinal motor neurons is how to accurately compensate for non-stationary effects in the signal during source separation routines, particularly when they cannot be estimated in advance. This forces current systems to instead use undifferentiated bulk signal, which limits the potential degrees of freedom for control. In this study we propose a potenti…
▽ More
A major outstanding problem when interfacing with spinal motor neurons is how to accurately compensate for non-stationary effects in the signal during source separation routines, particularly when they cannot be estimated in advance. This forces current systems to instead use undifferentiated bulk signal, which limits the potential degrees of freedom for control. In this study we propose a potential solution, using an unsupervised learning algorithm to blindly correct for the effects of latent processes which drive the signal non-stationarities. We implement this methodology within the theoretical framework of a quasilinear version of independent component analysis (ICA). The proposed design, HarmonICA, sidesteps the identifiability problems of nonlinear ICA, allowing for equivalent predictability to linear ICA whilst retaining the ability to learn complex nonlinear relationships between non-stationary latents and their effects on the signal. We test HarmonICA on both invasive and non-invasive recordings both simulated and real, demonstrating an ability to blindly compensate for the non-stationary effects specific to each, and thus to significantly enhance the quality of a source separation routine.
△ Less
Submitted 27 June, 2024;
originally announced June 2024.
-
The RSNA Abdominal Traumatic Injury CT (RATIC) Dataset
Authors:
Jeffrey D. Rudie,
Hui-Ming Lin,
Robyn L. Ball,
Sabeena Jalal,
Luciano M. Prevedello,
Savvas Nicolaou,
Brett S. Marinelli,
Adam E. Flanders,
Kirti Magudia,
George Shih,
Melissa A. Davis,
John Mongan,
Peter D. Chang,
Ferco H. Berger,
Sebastiaan Hermans,
Meng Law,
Tyler Richards,
Jan-Peter Grunz,
Andreas Steven Kunz,
Shobhit Mathur,
Sandro Galea-Soler,
Andrew D. Chung,
Saif Afat,
Chin-Chi Kuo,
Layal Aweidah
, et al. (15 additional authors not shown)
Abstract:
The RSNA Abdominal Traumatic Injury CT (RATIC) dataset is the largest publicly available collection of adult abdominal CT studies annotated for traumatic injuries. This dataset includes 4,274 studies from 23 institutions across 14 countries. The dataset is freely available for non-commercial use via Kaggle at https://www.kaggle.com/competitions/rsna-2023-abdominal-trauma-detection. Created for the…
▽ More
The RSNA Abdominal Traumatic Injury CT (RATIC) dataset is the largest publicly available collection of adult abdominal CT studies annotated for traumatic injuries. This dataset includes 4,274 studies from 23 institutions across 14 countries. The dataset is freely available for non-commercial use via Kaggle at https://www.kaggle.com/competitions/rsna-2023-abdominal-trauma-detection. Created for the RSNA 2023 Abdominal Trauma Detection competition, the dataset encourages the development of advanced machine learning models for detecting abdominal injuries on CT scans. The dataset encompasses detection and classification of traumatic injuries across multiple organs, including the liver, spleen, kidneys, bowel, and mesentery. Annotations were created by expert radiologists from the American Society of Emergency Radiology (ASER) and Society of Abdominal Radiology (SAR). The dataset is annotated at multiple levels, including the presence of injuries in three solid organs with injury grading, image-level annotations for active extravasations and bowel injury, and voxelwise segmentations of each of the potentially injured organs. With the release of this dataset, we hope to facilitate research and development in machine learning and abdominal trauma that can lead to improved patient care and outcomes.
△ Less
Submitted 29 May, 2024;
originally announced May 2024.
-
Space Physiology and Technology: Musculoskeletal Adaptations, Countermeasures, and the Opportunity for Wearable Robotics
Authors:
Shamas Ul Ebad Khan,
Re** John Varghese,
Panagiotis Kassanos,
Dario Farina,
Etienne Burdet
Abstract:
Space poses significant challenges for human physiology, leading to physiological adaptations in response to an environment vastly different from Earth. While these adaptations can be beneficial, they may not fully counteract the adverse impact of space-related stressors. A comprehensive understanding of these physiological adaptations is needed to devise effective countermeasures to support human…
▽ More
Space poses significant challenges for human physiology, leading to physiological adaptations in response to an environment vastly different from Earth. While these adaptations can be beneficial, they may not fully counteract the adverse impact of space-related stressors. A comprehensive understanding of these physiological adaptations is needed to devise effective countermeasures to support human life in space. This review focuses on the impact of the environment in space on the musculoskeletal system. It highlights the complex interplay between bone and muscle adaptation, the underlying physiological mechanisms, and their implications on astronaut health. Furthermore, the review delves into the deployed and current advances in countermeasures and proposes, as a perspective for future developments, wearable sensing and robotic technologies, such as exoskeletons, as a fitting alternative.
△ Less
Submitted 4 April, 2024;
originally announced April 2024.
-
Design, Fabrication and Evaluation of a Stretchable High-Density Electromyography Array
Authors:
Re** John Varghese,
Matteo Pizzi,
Aritra Kundu,
Agnese Grison,
Etienne Burdet,
Dario Farina
Abstract:
The adoption of high-density electrode systems for human-machine interfaces in real-life applications has been impeded by practical and technical challenges, including noise interference, motion artifacts and the lack of compact electrode interfaces. To overcome some of these challenges, we introduce a wearable and stretchable electromyography (EMG) array, and present its design, fabrication metho…
▽ More
The adoption of high-density electrode systems for human-machine interfaces in real-life applications has been impeded by practical and technical challenges, including noise interference, motion artifacts and the lack of compact electrode interfaces. To overcome some of these challenges, we introduce a wearable and stretchable electromyography (EMG) array, and present its design, fabrication methodology, characterisation, and comprehensive evaluation. Our proposed solution comprises dry-electrodes on flexible printed circuit board (PCB) substrates, eliminating the need for time-consuming skin preparation. The proposed fabrication method allows the manufacturing of stretchable sleeves, with consistent and standardised coverage across subjects. We thoroughly tested our developed prototype, evaluating its potential for application in both research and real-world environments. The results of our study showed that the developed stretchable array matches or outperforms traditional EMG grids and holds promise in furthering the real-world translation of high-density EMG for human-machine interfaces.
△ Less
Submitted 29 March, 2024;
originally announced March 2024.
-
Design and Preliminary Evaluation of a Torso Stabiliser for Individuals with Spinal Cord Injury
Authors:
Re** John Varghese,
Man-Yan Tong,
Isabella Szczech,
Peter Bryan,
Dario Farina,
Etienne Burdet
Abstract:
Spinal cord injuries (SCIs) generally result in sensory and mobility impairments, with torso instability being particularly debilitating. Existing torso stabilisers are often rigid and restrictive. This paper presents an early investigation into a non-restrictive 1 degree-of-freedom (DoF) mechanical torso stabiliser inspired by devices such as centrifugal clutches and seat-belt mechanisms. Firstly…
▽ More
Spinal cord injuries (SCIs) generally result in sensory and mobility impairments, with torso instability being particularly debilitating. Existing torso stabilisers are often rigid and restrictive. This paper presents an early investigation into a non-restrictive 1 degree-of-freedom (DoF) mechanical torso stabiliser inspired by devices such as centrifugal clutches and seat-belt mechanisms. Firstly, the paper presents a motion-capture (MoCap) and OpenSim-based kinematic analysis of the cable-based system to understand requisite device characteristics. The simulated evaluation resulted in the cable-based device to require 55-60cm of unrestricted travel, and to lock at a threshold cable velocity of 80-100cm/sec. Next, the developed 1-DoF device is introduced. The proposed mechanical device is transparent during activities of daily living, and transitions to compliant blocking when incipient fall is detected. Prototype behaviour was then validated using a MoCap-based kinematic analysis to verify non-restrictive movement, reliable transition to blocking, and compliance of the blocking.
△ Less
Submitted 26 March, 2024;
originally announced March 2024.
-
Tackling Electrode Shift In Gesture Recognition with HD-EMG Electrode Subsets
Authors:
Joao Pereira,
Dimitrios Chalatsis,
Balint Hodossy,
Dario Farina
Abstract:
sEMG pattern recognition algorithms have been explored extensively in decoding movement intent, yet are known to be vulnerable to changing recording conditions, exhibiting significant drops in performance across subjects, and even across sessions. Multi-channel surface EMG, also referred to as high-density sEMG (HD-sEMG) systems, have been used to improve performance with the information collected…
▽ More
sEMG pattern recognition algorithms have been explored extensively in decoding movement intent, yet are known to be vulnerable to changing recording conditions, exhibiting significant drops in performance across subjects, and even across sessions. Multi-channel surface EMG, also referred to as high-density sEMG (HD-sEMG) systems, have been used to improve performance with the information collected through the use of additional electrodes. However, a lack of robustness is ever present due to limited datasets and the difficulties in addressing sources of variability, such as electrode placement. In this study, we propose training on a collection of input channel subsets and augmenting our training distribution with data from different electrode locations, simultaneously targeting electrode shift and reducing input dimensionality. Our method increases robustness against electrode shift and results in significantly higher intersession performance across subjects and classification algorithms.
△ Less
Submitted 5 January, 2024;
originally announced January 2024.
-
A Human-Machine Joint Learning Framework to Boost Endogenous BCI Training
Authors:
Hanwen Wang,
Yu Qi,
Lin Yao,
Yueming Wang,
Dario Farina,
Gang Pan
Abstract:
Brain-computer interfaces (BCIs) provide a direct pathway from the brain to external devices and have demonstrated great potential for assistive and rehabilitation technologies. Endogenous BCIs based on electroencephalogram (EEG) signals, such as motor imagery (MI) BCIs, can provide some level of control. However, mastering spontaneous BCI control requires the users to generate discriminative and…
▽ More
Brain-computer interfaces (BCIs) provide a direct pathway from the brain to external devices and have demonstrated great potential for assistive and rehabilitation technologies. Endogenous BCIs based on electroencephalogram (EEG) signals, such as motor imagery (MI) BCIs, can provide some level of control. However, mastering spontaneous BCI control requires the users to generate discriminative and stable brain signal patterns by imagery, which is challenging and is usually achieved over a very long training time (weeks/months). Here, we propose a human-machine joint learning framework to boost the learning process in endogenous BCIs, by guiding the user to generate brain signals towards an optimal distribution estimated by the decoder, given the historical brain signals of the user. To this end, we firstly model the human-machine joint learning process in a uniform formulation. Then a human-machine joint learning framework is proposed: 1) for the human side, we model the learning process in a sequential trial-and-error scenario and propose a novel ``copy/new'' feedback paradigm to help shape the signal generation of the subject toward the optimal distribution; 2) for the machine side, we propose a novel adaptive learning algorithm to learn an optimal signal distribution along with the subject's learning process. Specifically, the decoder reweighs the brain signals generated by the subject to focus more on ``good'' samples to cope with the learning process of the subject. Online and psuedo-online BCI experiments with 18 healthy subjects demonstrated the advantages of the proposed joint learning process over co-adaptive approaches in both learning efficiency and effectiveness.
△ Less
Submitted 24 August, 2023;
originally announced September 2023.
-
Conditional Generative Models for Simulation of EMG During Naturalistic Movements
Authors:
Shihan Ma,
Alexander Kenneth Clarke,
Kostiantyn Maksymenko,
Samuel Deslauriers-Gauthier,
Xinjun Sheng,
Xiangyang Zhu,
Dario Farina
Abstract:
Numerical models of electromyographic (EMG) signals have provided a huge contribution to our fundamental understanding of human neurophysiology and remain a central pillar of motor neuroscience and the development of human-machine interfaces. However, whilst modern biophysical simulations based on finite element methods are highly accurate, they are extremely computationally expensive and thus are…
▽ More
Numerical models of electromyographic (EMG) signals have provided a huge contribution to our fundamental understanding of human neurophysiology and remain a central pillar of motor neuroscience and the development of human-machine interfaces. However, whilst modern biophysical simulations based on finite element methods are highly accurate, they are extremely computationally expensive and thus are generally limited to modelling static systems such as isometrically contracting limbs. As a solution to this problem, we propose a transfer learning approach, in which a conditional generative model is trained to mimic the output of an advanced numerical model. To this end, we present BioMime, a conditional generative neural network trained adversarially to generate motor unit activation potential waveforms under a wide variety of volume conductor parameters. We demonstrate the ability of such a model to predictively interpolate between a much smaller number of numerical model's outputs with a high accuracy. Consequently, the computational load is dramatically reduced, which allows the rapid simulation of EMG signals during truly dynamic and naturalistic movements.
△ Less
Submitted 5 October, 2023; v1 submitted 3 November, 2022;
originally announced November 2022.
-
Toward a Framework for Adaptive Impedance Control of an Upper-limb Prosthesis
Authors:
Laura Ferrante,
Mohan Sridharan,
Claudio Zito,
Dario Farina
Abstract:
Adapting upper-limb impedance (i.e., stiffness, dam**, inertia) is essential for humans interacting with dynamic environments for executing gras** or manipulation tasks. On the other hand, control methods designed for state-of-the-art upper-limb prostheses infer motor intent from surface electromyography (sEMG) signals in terms of joint kinematics, but they fail to infer and use the underlying…
▽ More
Adapting upper-limb impedance (i.e., stiffness, dam**, inertia) is essential for humans interacting with dynamic environments for executing gras** or manipulation tasks. On the other hand, control methods designed for state-of-the-art upper-limb prostheses infer motor intent from surface electromyography (sEMG) signals in terms of joint kinematics, but they fail to infer and use the underlying impedance properties of the limb. We present a framework that allows a human user to simultaneously control the kinematics, stiffness, and dam** of a simulated robot through wrist's flexion-extension. The framework includes muscle-tendon units and a forward dynamics block to estimate the motor intent from sEMG signals, and a variable impedance controller that implements the estimated intent on the robot, allowing the user to adapt the robot's kinematics and dynamics online. We evaluate our framework with 8 able-bodied subjects and an amputee during reaching tasks performed in free space, and in the presence of unexpected external perturbations that require adaptation of the wrist impedance to ensure stable interaction with the environment. We experimentally demonstrate that our approach outperforms a data-driven baseline in terms of its ability to adapt to external perturbations, overall controllability, and feedback from participants.
△ Less
Submitted 17 December, 2022; v1 submitted 11 September, 2022;
originally announced September 2022.
-
Hand Gesture Recognition Using Temporal Convolutions and Attention Mechanism
Authors:
Elahe Rahimian,
Soheil Zabihi,
Amir Asif,
Dario Farina,
S. Farokh Atashzar,
Arash Mohammadi
Abstract:
Advances in biosignal signal processing and machine learning, in particular Deep Neural Networks (DNNs), have paved the way for the development of innovative Human-Machine Interfaces for decoding the human intent and controlling artificial limbs. DNN models have shown promising results with respect to other algorithms for decoding muscle electrical activity, especially for recognition of hand gest…
▽ More
Advances in biosignal signal processing and machine learning, in particular Deep Neural Networks (DNNs), have paved the way for the development of innovative Human-Machine Interfaces for decoding the human intent and controlling artificial limbs. DNN models have shown promising results with respect to other algorithms for decoding muscle electrical activity, especially for recognition of hand gestures. Such data-driven models, however, have been challenged by their need for a large number of trainable parameters and their structural complexity. Here we propose the novel Temporal Convolutions-based Hand Gesture Recognition architecture (TC-HGR) to reduce this computational burden. With this approach, we classified 17 hand gestures via surface Electromyogram (sEMG) signals by the adoption of attention mechanisms and temporal convolutions. The proposed method led to 81.65% and 80.72% classification accuracy for window sizes of 300ms and 200ms, respectively. The number of parameters to train the proposed TC-HGR architecture is 11.9 times less than that of its state-of-the-art counterpart.
△ Less
Submitted 17 October, 2021;
originally announced October 2021.
-
Deep Metric Learning with Locality Sensitive Angular Loss for Self-Correcting Source Separation of Neural Spiking Signals
Authors:
Alexander Kenneth Clarke,
Dario Farina
Abstract:
Neurophysiological time series, such as electromyographic signal and intracortical recordings, are typically composed of many individual spiking sources, the recovery of which can give fundamental insights into the biological system of interest or provide neural information for man-machine interfaces. For this reason, source separation algorithms have become an increasingly important tool in neuro…
▽ More
Neurophysiological time series, such as electromyographic signal and intracortical recordings, are typically composed of many individual spiking sources, the recovery of which can give fundamental insights into the biological system of interest or provide neural information for man-machine interfaces. For this reason, source separation algorithms have become an increasingly important tool in neuroscience and neuroengineering. However, in noisy or highly multivariate recordings these decomposition techniques often make a large number of errors, which degrades human-machine interfacing applications and often requires costly post-hoc manual cleaning of the output label set of spike timestamps. To address both the need for automated post-hoc cleaning and robust separation filters we propose a methodology based on deep metric learning, using a novel loss function which maintains intra-class variance, creating a rich embedding space suitable for both label cleaning and the discovery of new activations. We then validate this method with an artificially corrupted label set based on source-separated high-density surface electromyography recordings, recovering the original timestamps even in extreme degrees of feature and class-dependent label noise. This approach enables a neural network to learn to accurately decode neurophysiological time series using any imperfect method of labelling the signal.
△ Less
Submitted 13 October, 2021;
originally announced October 2021.
-
TEMGNet: Deep Transformer-based Decoding of Upperlimb sEMG for Hand Gestures Recognition
Authors:
Elahe Rahimian,
Soheil Zabihi,
Amir Asif,
Dario Farina,
S. Farokh Atashzar,
Arash Mohammadi
Abstract:
There has been a surge of recent interest in Machine Learning (ML), particularly Deep Neural Network (DNN)-based models, to decode muscle activities from surface Electromyography (sEMG) signals for myoelectric control of neurorobotic systems. DNN-based models, however, require large training sets and, typically, have high structural complexity, i.e., they depend on a large number of trainable para…
▽ More
There has been a surge of recent interest in Machine Learning (ML), particularly Deep Neural Network (DNN)-based models, to decode muscle activities from surface Electromyography (sEMG) signals for myoelectric control of neurorobotic systems. DNN-based models, however, require large training sets and, typically, have high structural complexity, i.e., they depend on a large number of trainable parameters. To address these issues, we developed a framework based on the Transformer architecture for processing sEMG signals. We propose a novel Vision Transformer (ViT)-based neural network architecture (referred to as the TEMGNet) to classify and recognize upperlimb hand gestures from sEMG to be used for myocontrol of prostheses. The proposed TEMGNet architecture is trained with a small dataset without the need for pre-training or fine-tuning. To evaluate the efficacy, following the-recent literature, the second subset (exercise B) of the NinaPro DB2 dataset was utilized, where the proposed TEMGNet framework achieved a recognition accuracy of 82.93% and 82.05% for window sizes of 300ms and 200ms, respectively, outperforming its state-of-the-art counterparts. Moreover, the proposed TEMGNet framework is superior in terms of structural capacity while having seven times fewer trainable parameters. These characteristics and the high performance make DNN-based models promising approaches for myoelectric control of neurorobots.
△ Less
Submitted 25 September, 2021;
originally announced September 2021.
-
Human movement augmentation and how to make it a reality
Authors:
Jonathan Eden,
Mario Bräcklein,
Jaime Ibáñez Pereda,
Deren Yusuf Barsakcioglu,
Giovanni Di Pino,
Dario Farina,
Etienne Burdet,
Carsten Mehring
Abstract:
Augmenting the body with artificial limbs controlled concurrently to the natural limbs has long appeared in science fiction, but recent technological and neuroscientific advances have begun to make this vision possible. By allowing individuals to achieve otherwise impossible actions, this movement augmentation could revolutionize medical and industrial applications and profoundly change the way hu…
▽ More
Augmenting the body with artificial limbs controlled concurrently to the natural limbs has long appeared in science fiction, but recent technological and neuroscientific advances have begun to make this vision possible. By allowing individuals to achieve otherwise impossible actions, this movement augmentation could revolutionize medical and industrial applications and profoundly change the way humans interact with their environment. Here, we construct a movement augmentation taxonomy through what is augmented and how it is achieved. With this framework, we analyze augmentation that extends the number of degrees-of-freedom, discuss critical features of effective augmentation such as physiological control signals, sensory feedback and learning, and propose a vision for the field.
△ Less
Submitted 15 June, 2021;
originally announced June 2021.
-
FS-HGR: Few-shot Learning for Hand Gesture Recognition via ElectroMyography
Authors:
Elahe Rahimian,
Soheil Zabihi,
Amir Asif,
Dario Farina,
Seyed Farokh Atashzar,
Arash Mohammadi
Abstract:
This work is motivated by the recent advances in Deep Neural Networks (DNNs) and their widespread applications in human-machine interfaces. DNNs have been recently used for detecting the intended hand gesture through processing of surface electromyogram (sEMG) signals. The ultimate goal of these approaches is to realize high-performance controllers for prosthetic. However, although DNNs have shown…
▽ More
This work is motivated by the recent advances in Deep Neural Networks (DNNs) and their widespread applications in human-machine interfaces. DNNs have been recently used for detecting the intended hand gesture through processing of surface electromyogram (sEMG) signals. The ultimate goal of these approaches is to realize high-performance controllers for prosthetic. However, although DNNs have shown superior accuracy than conventional methods when large amounts of data are available for training, their performance substantially decreases when data are limited. Collecting large datasets for training may be feasible in research laboratories, but it is not a practical approach for real-life applications. Therefore, there is an unmet need for the design of a modern gesture detection technique that relies on minimal training data while providing high accuracy. Here we propose an innovative and novel "Few-Shot Learning" framework based on the formulation of meta-learning, referred to as the FS-HGR, to address this need. Few-shot learning is a variant of domain adaptation with the goal of inferring the required output based on just one or a few training examples. More specifically, the proposed FS-HGR quickly generalizes after seeing very few examples from each class. The proposed approach led to 85.94% classification accuracy on new repetitions with few-shot observation (5-way 5-shot), 81.29% accuracy on new subjects with few-shot observation (5-way 5-shot), and 73.36% accuracy on new gestures with few-shot observation (5-way 5-shot).
△ Less
Submitted 11 November, 2020;
originally announced November 2020.
-
AutoMat: Accelerated Computational Electrochemical systems Discovery
Authors:
Emil Annevelink,
Rachel Kurchin,
Eric Muckley,
Lance Kavalsky,
Vinay I. Hegde,
Valentin Sulzer,
Shang Zhu,
Jiankun Pu,
David Farina,
Matthew Johnson,
Dhairya Gandhi,
Adarsh Dave,
Hongyi Lin,
Alan Edelman,
Bharath Ramsundar,
James Saal,
Christopher Rackauckas,
Viral Shah,
Bryce Meredig,
Venkatasubramanian Viswanathan
Abstract:
Large-scale electrification is vital to addressing the climate crisis, but several scientific and technological challenges remain to fully electrify both the chemical industry and transportation. In both of these areas, new electrochemical materials will be critical, but their development currently relies heavily on human-time-intensive experimental trial and error and computationally expensive fi…
▽ More
Large-scale electrification is vital to addressing the climate crisis, but several scientific and technological challenges remain to fully electrify both the chemical industry and transportation. In both of these areas, new electrochemical materials will be critical, but their development currently relies heavily on human-time-intensive experimental trial and error and computationally expensive first-principles, meso-scale and continuum simulations. We present an automated workflow, AutoMat, that accelerates these computational steps by introducing both automated input generation and management of simulations across scales from first principles to continuum device modeling. Furthermore, we show how to seamlessly integrate multi-fidelity predictions such as machine learning surrogates or automated robotic experiments "in-the-loop". The automated framework is implemented with design space search techniques to dramatically accelerate the overall materials discovery pipeline by implicitly learning design features that optimize device performance across several metrics. We discuss the benefits of AutoMat using examples in electrocatalysis and energy storage and highlight lessons learned.
△ Less
Submitted 13 May, 2022; v1 submitted 3 November, 2020;
originally announced November 2020.
-
BS-Net: learning COVID-19 pneumonia severity on a large Chest X-Ray dataset
Authors:
Alberto Signoroni,
Mattia Savardi,
Sergio Benini,
Nicola Adami,
Riccardo Leonardi,
Paolo Gibellini,
Filippo Vaccher,
Marco Ravanelli,
Andrea Borghesi,
Roberto Maroldi,
Davide Farina
Abstract:
In this work we design an end-to-end deep learning architecture for predicting, on Chest X-rays images (CXR), a multi-regional score conveying the degree of lung compromise in COVID-19 patients. Such semi-quantitative scoring system, namely Brixia~score, is applied in serial monitoring of such patients, showing significant prognostic value, in one of the hospitals that experienced one of the highe…
▽ More
In this work we design an end-to-end deep learning architecture for predicting, on Chest X-rays images (CXR), a multi-regional score conveying the degree of lung compromise in COVID-19 patients. Such semi-quantitative scoring system, namely Brixia~score, is applied in serial monitoring of such patients, showing significant prognostic value, in one of the hospitals that experienced one of the highest pandemic peaks in Italy. To solve such a challenging visual task, we adopt a weakly supervised learning strategy structured to handle different tasks (segmentation, spatial alignment, and score estimation) trained with a "from-the-part-to-the-whole" procedure involving different datasets. In particular, we exploit a clinical dataset of almost 5,000 CXR annotated images collected in the same hospital. Our BS-Net demonstrates self-attentive behavior and a high degree of accuracy in all processing stages. Through inter-rater agreement tests and a gold standard comparison, we show that our solution outperforms single human annotators in rating accuracy and consistency, thus supporting the possibility of using this tool in contexts of computer-assisted monitoring. Highly resolved (super-pixel level) explainability maps are also generated, with an original technique, to visually help the understanding of the network activity on the lung areas. We also consider other scores proposed in literature and provide a comparison with a recently proposed non-specific approach. We eventually test the performance robustness of our model on an assorted public COVID-19 dataset, for which we also provide Brixia~score annotations, observing good direct generalization and fine-tuning capabilities that highlight the portability of BS-Net in other clinical settings. The CXR dataset along with the source code and the trained model are publicly released for research purposes.
△ Less
Submitted 3 April, 2021; v1 submitted 8 June, 2020;
originally announced June 2020.
