-
Vision-language models for decoding provider attention during neonatal resuscitation
Authors:
Felipe Parodi,
Jordan Matelsky,
Alejandra Regla-Vargas,
Elizabeth Foglia,
Charis Lim,
Danielle Weinberg,
Konrad Kording,
Heidi Herrick,
Michael Platt
Abstract:
Neonatal resuscitations demand an exceptional level of attentiveness from providers, who must process multiple streams of information simultaneously. Gaze strongly influences decision making; thus, understanding where a provider is looking during neonatal resuscitations could inform provider training, enhance real-time decision support, and improve the design of delivery rooms and neonatal intensi…
▽ More
Neonatal resuscitations demand an exceptional level of attentiveness from providers, who must process multiple streams of information simultaneously. Gaze strongly influences decision making; thus, understanding where a provider is looking during neonatal resuscitations could inform provider training, enhance real-time decision support, and improve the design of delivery rooms and neonatal intensive care units (NICUs). Current approaches to quantifying neonatal providers' gaze rely on manual coding or simulations, which limit scalability and utility. Here, we introduce an automated, real-time, deep learning approach capable of decoding provider gaze into semantic classes directly from first-person point-of-view videos recorded during live resuscitations. Combining state-of-the-art, real-time segmentation with vision-language models (CLIP), our low-shot pipeline attains 91\% classification accuracy in identifying gaze targets without training. Upon fine-tuning, the performance of our gaze-guided vision transformer exceeds 98\% accuracy in gaze classification, approaching human-level precision. This system, capable of real-time inference, enables objective quantification of provider attention dynamics during live neonatal resuscitation. Our approach offers a scalable solution that seamlessly integrates with existing infrastructure for data-scarce gaze analysis, thereby offering new opportunities for understanding and refining clinical decision making.
△ Less
Submitted 1 April, 2024;
originally announced April 2024.
-
On the Use of Singular Value Decomposition as a Clutter Filter for Ultrasound Flow Imaging
Authors:
Kai Riemer,
Marcelo Lerendegui,
Matthieu Toulemonde,
Jiaqi Zhu,
Christopher Dunsby,
Peter D. Weinberg,
Meng-Xing Tang
Abstract:
Filtering based on Singular Value Decomposition (SVD) provides substantial separation of clutter, flow and noise in high frame rate ultrasound flow imaging. The use of SVD as a clutter filter has greatly improved techniques such as vector flow imaging, functional ultrasound and super-resolution ultrasound localization microscopy. The removal of clutter and noise relies on the assumption that tissu…
▽ More
Filtering based on Singular Value Decomposition (SVD) provides substantial separation of clutter, flow and noise in high frame rate ultrasound flow imaging. The use of SVD as a clutter filter has greatly improved techniques such as vector flow imaging, functional ultrasound and super-resolution ultrasound localization microscopy. The removal of clutter and noise relies on the assumption that tissue, flow and noise are each represented by different subsets of singular values, so that their signals are uncorrelated and lay on orthogonal sub-spaces. This assumption fails in the presence of tissue motion, for near-wall or microvascular flow, and can be influenced by an incorrect choice of singular value thresholds. Consequently, separation of flow, clutter and noise is imperfect, which can lead to image artefacts not present in the original data. Temporal and spatial fluctuation in intensity are the commonest artefacts, which vary in appearance and strengths. Ghosting and splitting artefacts are observed in the microvasculature where the flow signal is sparsely distributed. Singular value threshold selection, tissue motion, frame rate, flow signal amplitude and acquisition length affect the prevalence of these artefacts. Understanding what causes artefacts due to SVD clutter and noise removal is necessary for their interpretation.
△ Less
Submitted 25 April, 2023;
originally announced April 2023.
-
Communication Trade-offs in Federated Learning of Spiking Neural Networks
Authors:
Soumi Chaki,
David Weinberg,
Ayca Özcelikkale
Abstract:
Spiking Neural Networks (SNNs) are biologically inspired alternatives to conventional Artificial Neural Networks (ANNs). Despite promising preliminary results, the trade-offs in the training of SNNs in a distributed scheme are not well understood. Here, we consider SNNs in a federated learning setting where a high-quality global model is created by aggregating multiple local models from the client…
▽ More
Spiking Neural Networks (SNNs) are biologically inspired alternatives to conventional Artificial Neural Networks (ANNs). Despite promising preliminary results, the trade-offs in the training of SNNs in a distributed scheme are not well understood. Here, we consider SNNs in a federated learning setting where a high-quality global model is created by aggregating multiple local models from the clients without sharing any data. We investigate federated learning for training multiple SNNs at clients when two mechanisms reduce the uplink communication cost: i) random masking of the model updates sent from the clients to the server; and ii) client dropouts where some clients do not send their updates to the server. We evaluated the performance of the SNNs using a subset of the Spiking Heidelberg digits (SHD) dataset. The results show that a trade-off between the random masking and the client drop probabilities is crucial to obtain a satisfactory performance for a fixed number of clients.
△ Less
Submitted 27 February, 2023;
originally announced March 2023.
-
The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification
Authors:
Ujjwal Baid,
Satyam Ghodasara,
Suyash Mohan,
Michel Bilello,
Evan Calabrese,
Errol Colak,
Keyvan Farahani,
Jayashree Kalpathy-Cramer,
Felipe C. Kitamura,
Sarthak Pati,
Luciano M. Prevedello,
Jeffrey D. Rudie,
Chiharu Sako,
Russell T. Shinohara,
Timothy Bergquist,
Rong Chai,
James Eddy,
Julia Elliott,
Walter Reade,
Thomas Schaffter,
Thomas Yu,
Jiaxin Zheng,
Ahmed W. Moawad,
Luiz Otavio Coelho,
Olivia McDonnell
, et al. (78 additional authors not shown)
Abstract:
The BraTS 2021 challenge celebrates its 10th anniversary and is jointly organized by the Radiological Society of North America (RSNA), the American Society of Neuroradiology (ASNR), and the Medical Image Computing and Computer Assisted Interventions (MICCAI) society. Since its inception, BraTS has been focusing on being a common benchmarking venue for brain glioma segmentation algorithms, with wel…
▽ More
The BraTS 2021 challenge celebrates its 10th anniversary and is jointly organized by the Radiological Society of North America (RSNA), the American Society of Neuroradiology (ASNR), and the Medical Image Computing and Computer Assisted Interventions (MICCAI) society. Since its inception, BraTS has been focusing on being a common benchmarking venue for brain glioma segmentation algorithms, with well-curated multi-institutional multi-parametric magnetic resonance imaging (mpMRI) data. Gliomas are the most common primary malignancies of the central nervous system, with varying degrees of aggressiveness and prognosis. The RSNA-ASNR-MICCAI BraTS 2021 challenge targets the evaluation of computational algorithms assessing the same tumor compartmentalization, as well as the underlying tumor's molecular characterization, in pre-operative baseline mpMRI data from 2,040 patients. Specifically, the two tasks that BraTS 2021 focuses on are: a) the segmentation of the histologically distinct brain tumor sub-regions, and b) the classification of the tumor's O[6]-methylguanine-DNA methyltransferase (MGMT) promoter methylation status. The performance evaluation of all participating algorithms in BraTS 2021 will be conducted through the Sage Bionetworks Synapse platform (Task 1) and Kaggle (Task 2), concluding in distributing to the top ranked participants monetary awards of $60,000 collectively.
△ Less
Submitted 12 September, 2021; v1 submitted 5 July, 2021;
originally announced July 2021.
-
Detecting Areas of Potential High Prevalence of Chagas in Argentina
Authors:
Antonio Vazquez Brust,
Tomas Olego,
German Rosati,
Carolina Lang,
Guillermo Bozzoli,
Diego Weinberg,
Roberto Chuit,
Martin A. Minnoni,
Carlos Sarraute
Abstract:
A map of potential prevalence of Chagas disease (ChD) with high spatial disaggregation is presented. It aims to detect areas outside the Gran Chaco ecoregion (hyperendemic for the ChD), characterized by high affinity with ChD and high health vulnerability.
To quantify potential prevalence, we developed several indicators: an Affinity Index which quantifies the degree of linkage between endemic a…
▽ More
A map of potential prevalence of Chagas disease (ChD) with high spatial disaggregation is presented. It aims to detect areas outside the Gran Chaco ecoregion (hyperendemic for the ChD), characterized by high affinity with ChD and high health vulnerability.
To quantify potential prevalence, we developed several indicators: an Affinity Index which quantifies the degree of linkage between endemic areas of ChD and the rest of the country. We also studied favorable habitability conditions for Triatoma infestans, looking for areas where the predominant materials of floors, roofs and internal ceilings favor the presence of the disease vector.
We studied determinants of a more general nature that can be encompassed under the concept of Health Vulnerability Index. These determinants are associated with access to health providers and the socio-economic level of different segments of the population.
Finally we constructed a Chagas Potential Prevalence Index (ChPPI) which combines the affinity index, the health vulnerability index, and the population density. We show and discuss the maps obtained. These maps are intended to assist public health specialists, decision makers of public health policies and public officials in the development of cost-effective strategies to improve access to diagnosis and treatment of ChD.
△ Less
Submitted 2 January, 2020;
originally announced January 2020.
-
Uncovering the Spread of Chagas Disease in Argentina and Mexico
Authors:
Juan de Monasterio,
Alejo Salles,
Carolina Lang,
Diego Weinberg,
Martin Minnoni,
Matias Travizano,
Carlos Sarraute
Abstract:
Chagas disease is a neglected disease, and information about its geographical spread is very scarse. We analyze here mobility and calling patterns in order to identify potential risk zones for the disease, by using public health information and mobile phone records. Geolocalized call records are rich in social and mobility information, which can be used to infer whether an individual has lived in…
▽ More
Chagas disease is a neglected disease, and information about its geographical spread is very scarse. We analyze here mobility and calling patterns in order to identify potential risk zones for the disease, by using public health information and mobile phone records. Geolocalized call records are rich in social and mobility information, which can be used to infer whether an individual has lived in an endemic area. We present two case studies in Latin American countries. Our objective is to generate risk maps which can be used by public health campaign managers to prioritize detection campaigns and target specific areas. Finally, we analyze the value of mobile phone data to infer long-term migrations, which play a crucial role in the geographical spread of Chagas disease.
△ Less
Submitted 9 August, 2018;
originally announced August 2018.
-
Analyzing the Spread of Chagas Disease with Mobile Phone Data
Authors:
Juan de Monasterio,
Alejo Salles,
Carolina Lang,
Diego Weinberg,
Martin Minnoni,
Matias Travizano,
Carlos Sarraute
Abstract:
We use mobile phone records for the analysis of mobility patterns and the detection of possible risk zones of Chagas disease in two Latin American countries. We show that geolocalized call records are rich in social and individual information, which can be used to infer whether an individual has lived in an endemic area. We present two case studies, in Argentina and in Mexico, using data provided…
▽ More
We use mobile phone records for the analysis of mobility patterns and the detection of possible risk zones of Chagas disease in two Latin American countries. We show that geolocalized call records are rich in social and individual information, which can be used to infer whether an individual has lived in an endemic area. We present two case studies, in Argentina and in Mexico, using data provided by mobile phone companies from each country. The risk maps that we generate can be used by health campaign managers to target specific areas and allocate resources more effectively.
△ Less
Submitted 4 July, 2017;
originally announced July 2017.
-
Superresolution of Noisy Remotely Sensed Images Through Directional Representations
Authors:
Wojciech Czaja,
James M. Murphy,
Daniel Weinberg
Abstract:
We develop an algorithm for single-image superresolution of remotely sensed data, based on the discrete shearlet transform. The shearlet transform extracts directional features of signals, and is known to provide near-optimally sparse representations for a broad class of images. This often leads to superior performance in edge detection and image representation when compared to isotropic frames. W…
▽ More
We develop an algorithm for single-image superresolution of remotely sensed data, based on the discrete shearlet transform. The shearlet transform extracts directional features of signals, and is known to provide near-optimally sparse representations for a broad class of images. This often leads to superior performance in edge detection and image representation when compared to isotropic frames. We justify the use of shearlets mathematically, before presenting a denoising single-image superresolution algorithm that combines the shearlet transform with sparse mixing estimators (SME). Our algorithm is compared with a variety of single-image superresolution methods, including wavelet SME superresolution. Our numerical results demonstrate competitive performance in terms of PSNR and SSIM.
△ Less
Submitted 4 September, 2018; v1 submitted 27 February, 2016;
originally announced February 2016.
