-
MrRegNet: Multi-resolution Mask Guided Convolutional Neural Network for Medical Image Registration with Large Deformations
Authors:
Ruizhe Li,
Grazziela Figueredo,
Dorothee Auer,
Christian Wagner,
Xin Chen
Abstract:
Deformable image registration (alignment) is highly sought after in numerous clinical applications, such as computer aided diagnosis and disease progression analysis. Deep Convolutional Neural Network (DCNN)-based image registration methods have demonstrated advantages in terms of registration accuracy and computational speed. However, while most methods excel at global alignment, they often perfo…
▽ More
Deformable image registration (alignment) is highly sought after in numerous clinical applications, such as computer aided diagnosis and disease progression analysis. Deep Convolutional Neural Network (DCNN)-based image registration methods have demonstrated advantages in terms of registration accuracy and computational speed. However, while most methods excel at global alignment, they often perform worse in aligning local regions. To address this challenge, this paper proposes a mask-guided encoder-decoder DCNN-based image registration method, named as MrRegNet. This approach employs a multi-resolution encoder for feature extraction and subsequently estimates multi-resolution displacement fields in the decoder to handle the substantial deformation of images. Furthermore, segmentation masks are employed to direct the model's attention toward aligning local regions. The results show that the proposed method outperforms traditional methods like Demons and a well-known deep learning method, VoxelMorph, on a public 3D brain MRI dataset (OASIS) and a local 2D brain MRI dataset with large deformations. Importantly, the image alignment accuracies are significantly improved at local regions guided by segmentation masks. Github link:https://github.com/ruizhe-l/MrRegNet.
△ Less
Submitted 16 May, 2024;
originally announced May 2024.
-
Federated Learning Enables Big Data for Rare Cancer Boundary Detection
Authors:
Sarthak Pati,
Ujjwal Baid,
Brandon Edwards,
Micah Sheller,
Shih-Han Wang,
G Anthony Reina,
Patrick Foley,
Alexey Gruzdev,
Deepthi Karkada,
Christos Davatzikos,
Chiharu Sako,
Satyam Ghodasara,
Michel Bilello,
Suyash Mohan,
Philipp Vollmuth,
Gianluca Brugnara,
Chandrakanth J Preetha,
Felix Sahm,
Klaus Maier-Hein,
Maximilian Zenk,
Martin Bendszus,
Wolfgang Wick,
Evan Calabrese,
Jeffrey Rudie,
Javier Villanueva-Meyer
, et al. (254 additional authors not shown)
Abstract:
Although machine learning (ML) has shown promise in numerous domains, there are concerns about generalizability to out-of-sample data. This is currently addressed by centrally sharing ample, and importantly diverse, data from multiple sites. However, such centralization is challenging to scale (or even not feasible) due to various limitations. Federated ML (FL) provides an alternative to train acc…
▽ More
Although machine learning (ML) has shown promise in numerous domains, there are concerns about generalizability to out-of-sample data. This is currently addressed by centrally sharing ample, and importantly diverse, data from multiple sites. However, such centralization is challenging to scale (or even not feasible) due to various limitations. Federated ML (FL) provides an alternative to train accurate and generalizable ML models, by only sharing numerical model updates. Here we present findings from the largest FL study to-date, involving data from 71 healthcare institutions across 6 continents, to generate an automatic tumor boundary detector for the rare disease of glioblastoma, utilizing the largest dataset of such patients ever used in the literature (25,256 MRI scans from 6,314 patients). We demonstrate a 33% improvement over a publicly trained model to delineate the surgically targetable tumor, and 23% improvement over the tumor's entire extent. We anticipate our study to: 1) enable more studies in healthcare informed by large and diverse data, ensuring meaningful results for rare diseases and underrepresented populations, 2) facilitate further quantitative analyses for glioblastoma via performance optimization of our consensus model for eventual public release, and 3) demonstrate the effectiveness of FL at such scale and task complexity as a paradigm shift for multi-site collaborations, alleviating the need for data sharing.
△ Less
Submitted 25 April, 2022; v1 submitted 22 April, 2022;
originally announced April 2022.
-
Image Augmentation Using a Task Guided Generative Adversarial Network for Age Estimation on Brain MRI
Authors:
Ruizhe Li,
Matteo Bastiani,
Dorothee Auer,
Christian Wagner,
Xin Chen
Abstract:
Brain age estimation based on magnetic resonance imaging (MRI) is an active research area in early diagnosis of some neurodegenerative diseases (e.g. Alzheimer, Parkinson, Huntington, etc.) for elderly people or brain underdevelopment for the young group. Deep learning methods have achieved the state-of-the-art performance in many medical image analysis tasks, including brain age estimation. Howev…
▽ More
Brain age estimation based on magnetic resonance imaging (MRI) is an active research area in early diagnosis of some neurodegenerative diseases (e.g. Alzheimer, Parkinson, Huntington, etc.) for elderly people or brain underdevelopment for the young group. Deep learning methods have achieved the state-of-the-art performance in many medical image analysis tasks, including brain age estimation. However, the performance and generalisability of the deep learning model are highly dependent on the quantity and quality of the training data set. Both collecting and annotating brain MRI data are extremely time-consuming. In this paper, to overcome the data scarcity problem, we propose a generative adversarial network (GAN) based image synthesis method. Different from the existing GAN-based methods, we integrate a task-guided branch (a regression model for age estimation) to the end of the generator in GAN. By adding a task-guided loss to the conventional GAN loss, the learned low-dimensional latent space and the synthesised images are more task-specific. It helps to boost the performance of the down-stream task by combining the synthesised images and real images for model training. The proposed method was evaluated on a public brain MRI data set for age estimation. Our proposed method outperformed (statistically significant) a deep convolutional neural network based regression model and the GAN-based image synthesis method without the task-guided branch. More importantly, it enables the identification of age-related brain regions in the image space. The code is available on GitHub (https://github.com/ruizhe-l/tgb-gan).
△ Less
Submitted 3 August, 2021;
originally announced August 2021.
-
Control of Scanning Quantum Dot Microscopy
Authors:
Michael Maiworm,
Christian Wagner,
Taner Esat,
Philipp Leinen,
Ruslan Temirov,
F. Stefan Tautz,
Rolf Findeisen
Abstract:
Scanning quantum dot microscopy is a recently developed high-resolution microscopy technique that is based on atomic force microscopy and is capable of imaging the electrostatic potential of nanostructures like molecules or single atoms. Recently, it could be shown that it not only yields qualitatively but also quantitatively cutting edge images even on an atomic level. In this paper we present ho…
▽ More
Scanning quantum dot microscopy is a recently developed high-resolution microscopy technique that is based on atomic force microscopy and is capable of imaging the electrostatic potential of nanostructures like molecules or single atoms. Recently, it could be shown that it not only yields qualitatively but also quantitatively cutting edge images even on an atomic level. In this paper we present how control is a key enabling element to this. The developed control approach consists of a two-degree-of-freedom control framework that comprises a feedforward and a feedback part. For the latter we design two tailored feedback controllers. The feedforward part generates a reference for the current scanned line based on the previously scanned one. We discuss in detail various aspects of the presented control approach and its implications for scanning quantum dot microscopy. We evaluate the influence of the feedforward part and compare the two proposed feedback controllers. The proposed control algorithms speed up scanning quantum dot microscopy by more than a magnitude and enable to scan large sample areas.
△ Less
Submitted 1 March, 2021; v1 submitted 25 February, 2021;
originally announced February 2021.
-
Frequency Sub-Sampling of Ultrasound Non-Destructive Measurements: Acquisition, Reconstruction and Performance
Authors:
Jan Kirchhof,
Sebastian Semper,
Christoph W. Wagner,
Eduardo Pérez,
Florian Römer,
Giovanni Del Galdo
Abstract:
In ultrasound nondestructive testing, a widespread approach is to take synthetic aperture measurements from the surface of a specimen to detect and locate defects within it. Based on these measurements, imaging is usually performed using the Synthetic Aperture Focusing Technique (SAFT). However, SAFT is sub-optimal in terms of resolution and requires oversampling in time domain to obtain a fine gr…
▽ More
In ultrasound nondestructive testing, a widespread approach is to take synthetic aperture measurements from the surface of a specimen to detect and locate defects within it. Based on these measurements, imaging is usually performed using the Synthetic Aperture Focusing Technique (SAFT). However, SAFT is sub-optimal in terms of resolution and requires oversampling in time domain to obtain a fine grid for the Delay-and-Sum (DAS). On the other hand, parametric reconstruction algorithms give better resolution, but their usage for imaging becomes computationally expensive due to the size of the parameter space and the large amount of measurement data in realistic 3-D scenarios. In the literature, the remedies to this are twofold: First, the amount of measurement data can be reduced using state of the art sub-Nyquist sampling approaches to measure Fourier coefficients instead of time domain samples. Second, parametric reconstruction algorithms mostly rely on matrix-vector operations that can be implemented efficiently by exploiting the underlying model structure. In this paper, we propose and compare different strategies to choose the Fourier coefficients to be measured. Their asymptotic performance is compared by numerically evaluating the Cramér-Rao-Bound for the localizability of the defect coordinates. These subsampling strategies are then combined with an $\ell_1$-minimization scheme to compute 3-D reconstructions from the low-rate measurements. Compared to conventional DAS, this allows us to formulate a fully physically motivated forward model. To enable this, the projection operations of the forward model matrix are implemented matrix-free by exploiting the underlying 2-level Toeplitz structure. Finally, we show that high resolution reconstructions from as low as a single Fourier coefficient per scan are possible based on simulated data as well as on measurements.
△ Less
Submitted 8 December, 2020;
originally announced December 2020.
-
Artificial neural networks for 3D cell shape recognition from confocal images
Authors:
G. Simionato,
K. Hinkelmann,
R. Chachanidze,
P. Bianchi,
E. Fermo,
R. van Wijk,
M. Leonetti,
C. Wagner,
L. Kaestner,
S. Quint
Abstract:
We present a dual-stage neural network architecture for analyzing fine shape details from microscopy recordings in 3D. The system, tested on red blood cells, uses training data from both healthy donors and patients with a congenital blood disease. Characteristic shape features are revealed from the spherical harmonics spectrum of each cell and are automatically processed to create a reproducible a…
▽ More
We present a dual-stage neural network architecture for analyzing fine shape details from microscopy recordings in 3D. The system, tested on red blood cells, uses training data from both healthy donors and patients with a congenital blood disease. Characteristic shape features are revealed from the spherical harmonics spectrum of each cell and are automatically processed to create a reproducible and unbiased shape recognition and classification for diagnostic and theragnostic use.
△ Less
Submitted 28 May, 2020; v1 submitted 16 May, 2020;
originally announced May 2020.
-
Fusing Online Gaussian Process-Based Learning and Control for Scanning Quantum Dot Microscopy
Authors:
Maik Pfefferkorn,
Michael Maiworm,
Christian Wagner,
F. Stefan Tautz,
Rolf Findeisen
Abstract:
Elucidating electrostatic surface potentials contributes to a deeper understanding of the nature of matter and its physicochemical properties, which is the basis for a wide field of applications. Scanning quantum dot microscopy, a recently developed technique allows to measure such potentials with atomic resolution. For an efficient deployment in scientific practice, however, it is essential to sp…
▽ More
Elucidating electrostatic surface potentials contributes to a deeper understanding of the nature of matter and its physicochemical properties, which is the basis for a wide field of applications. Scanning quantum dot microscopy, a recently developed technique allows to measure such potentials with atomic resolution. For an efficient deployment in scientific practice, however, it is essential to speed up the scanning process. To this end we employ a two-degree-of-freedom control paradigm, in which a Gaussian process is used as the feedforward part. We present a tailored online learning scheme of the Gaussian process, adapted to scanning quantum dot microscopy, that includes hyperparameter optimization during operation to enable fast and precise scanning of arbitrary surface structures. For the potential application in practice, the accompanying computational cost is reduced evaluating different sparse approximation approaches. The fully independent training conditional approximation, used on a reduced set of active training data, is found to be the most promising approach.
△ Less
Submitted 6 April, 2020;
originally announced April 2020.
