-
Pathology Synthesis of 3D-Consistent Cardiac MR Images using 2D VAEs and GANs
Authors:
Sina Amirrajab,
Cristian Lorenz,
Juergen Weese,
Josien Pluim,
Marcel Breeuwer
Abstract:
We propose a method for synthesizing cardiac magnetic resonance (MR) images with plausible heart pathologies and realistic appearances for the purpose of generating labeled data for the application of supervised deep-learning (DL) training. The image synthesis consists of label deformation and label-to-image translation tasks. The former is achieved via latent space interpolation in a VAE model, w…
▽ More
We propose a method for synthesizing cardiac magnetic resonance (MR) images with plausible heart pathologies and realistic appearances for the purpose of generating labeled data for the application of supervised deep-learning (DL) training. The image synthesis consists of label deformation and label-to-image translation tasks. The former is achieved via latent space interpolation in a VAE model, while the latter is accomplished via a label-conditional GAN model. We devise three approaches for label manipulation in the latent space of the trained VAE model; i) \textbf{intra-subject synthesis} aiming to interpolate the intermediate slices of a subject to increase the through-plane resolution, ii) \textbf{inter-subject synthesis} aiming to interpolate the geometry and appearance of intermediate images between two dissimilar subjects acquired with different scanner vendors, and iii) \textbf{pathology synthesis} aiming to synthesize a series of pseudo-pathological synthetic subjects with characteristics of a desired heart disease. Furthermore, we propose to model the relationship between 2D slices in the latent space of the VAE prior to reconstruction for generating 3D-consistent subjects from stacking up 2D slice-by-slice generations. We demonstrate that such an approach could provide a solution to diversify and enrich an available database of cardiac MR images and to pave the way for the development of generalizable DL-based image analysis algorithms. We quantitatively evaluate the quality of the synthesized data in an augmentation scenario to achieve generalization and robustness to multi-vendor and multi-disease data for image segmentation. Our code is available at https://github.com/sinaamirrajab/CardiacPathologySynthesis.
△ Less
Submitted 30 May, 2023; v1 submitted 9 September, 2022;
originally announced September 2022.
-
sim2real: Cardiac MR Image Simulation-to-Real Translation via Unsupervised GANs
Authors:
Sina Amirrajab,
Yasmina Al Khalil,
Cristian Lorenz,
Jurgen Weese,
Josien Pluim,
Marcel Breeuwer
Abstract:
There has been considerable interest in the MR physics-based simulation of a database of virtual cardiac MR images for the development of deep-learning analysis networks. However, the employment of such a database is limited or shows suboptimal performance due to the realism gap, missing textures, and the simplified appearance of simulated images. In this work we 1) provide image simulation on vir…
▽ More
There has been considerable interest in the MR physics-based simulation of a database of virtual cardiac MR images for the development of deep-learning analysis networks. However, the employment of such a database is limited or shows suboptimal performance due to the realism gap, missing textures, and the simplified appearance of simulated images. In this work we 1) provide image simulation on virtual XCAT subjects with varying anatomies, and 2) propose sim2real translation network to improve image realism. Our usability experiments suggest that sim2real data exhibits a good potential to augment training data and boost the performance of a segmentation algorithm.
△ Less
Submitted 9 August, 2022;
originally announced August 2022.
-
Very large-scale neighborhood search for drone routing with energy replenishment
Authors:
Catherine Lorenz,
Nicola Mimmo,
Alena Otto,
Daniele Vigo
Abstract:
The Drone Routing Problem with Energy replenishment (DRP-E) belongs to a general class of routing problems with intermediate stops and synchronization constraints. In DRP-E, the drone has to visit a set of nodes and routinely requires battery swaps from a (potentially) mobile replenishment station. Contrary to widespread restrictions in the drone routing literature, several destinations may be vis…
▽ More
The Drone Routing Problem with Energy replenishment (DRP-E) belongs to a general class of routing problems with intermediate stops and synchronization constraints. In DRP-E, the drone has to visit a set of nodes and routinely requires battery swaps from a (potentially) mobile replenishment station. Contrary to widespread restrictions in the drone routing literature, several destinations may be visited in between two consecutive battery swaps. In this paper, we propose a nontrivial very large-scale neighbourhood for DRP-E, which synergetically leverages two large-sized polynomially solvable DRP-E SubProblems (SP1 and SP2). The number of feasible solutions in the resulting neighborhood is a multiple of those in SP1 and SP2, and, thus, exponential in the input size of the problem, whereas the computational time to search it remains polynomial. The proposed polynomial two-stage dynamic programming algorithm VLSN to search this neighborhood can be flexibly adjusted to the desired trade-off between accuracy and computational time. For instance, the search procedure can be converted into an exact algorithm of competitive runtime for DRP-E. In computational tests, the developed solution methods outperform current state-of-the art heuristics for DRP-E by a significant margin. A case study based on a search for missing persons demonstrates that VLSN easily accommodates additional practice relevant features and outperforms the state-of-the-art solution in disaster relief by 20%.
△ Less
Submitted 28 July, 2022;
originally announced July 2022.
-
sympy2c: from symbolic expressions to fast C/C++ functions and ODE solvers in Python
Authors:
Uwe Schmitt,
Beatrice Moser,
Christiane S. Lorenz,
Alexandre Refregier
Abstract:
Computer algebra systems play an important role in science as they facilitate the development of new theoretical models. The resulting symbolic equations are often implemented in a compiled programming language in order to provide fast and portable codes for practical applications. We describe sympy2c, a new Python package designed to bridge the gap between the symbolic development and the numeric…
▽ More
Computer algebra systems play an important role in science as they facilitate the development of new theoretical models. The resulting symbolic equations are often implemented in a compiled programming language in order to provide fast and portable codes for practical applications. We describe sympy2c, a new Python package designed to bridge the gap between the symbolic development and the numerical implementation of a theoretical model. sympy2c translates symbolic equations implemented in the SymPy Python package to C/C++ code that is optimized using symbolic transformations. The resulting functions can be conveniently used as an extension module in Python. sympy2c is used within the PyCosmo Python package to solve the Einstein-Boltzmann equations, a large system of ODEs describing the evolution of linear perturbations in the Universe. After reviewing the functionalities and usage of sympy2c, we describe its implementation and optimization strategies. This includes, in particular, a novel approach to generate optimized ODE solvers making use of the sparsity of the symbolic Jacobian matrix. We demonstrate its performance using the Einstein-Boltzmann equations as a test case. sympy2c is widely applicable and may prove useful for various areas of computational physics. sympy2c is publicly available at https://cosmology.ethz.ch/research/software-lab/sympy2c.html
△ Less
Submitted 22 March, 2022;
originally announced March 2022.
-
XCAT-GAN for Synthesizing 3D Consistent Labeled Cardiac MR Images on Anatomically Variable XCAT Phantoms
Authors:
Sina Amirrajab,
Samaneh Abbasi-Sureshjani,
Yasmina Al Khalil,
Cristian Lorenz,
Juergen Weese,
Josien Pluim,
Marcel Breeuwer
Abstract:
Generative adversarial networks (GANs) have provided promising data enrichment solutions by synthesizing high-fidelity images. However, generating large sets of labeled images with new anatomical variations remains unexplored. We propose a novel method for synthesizing cardiac magnetic resonance (CMR) images on a population of virtual subjects with a large anatomical variation, introduced using th…
▽ More
Generative adversarial networks (GANs) have provided promising data enrichment solutions by synthesizing high-fidelity images. However, generating large sets of labeled images with new anatomical variations remains unexplored. We propose a novel method for synthesizing cardiac magnetic resonance (CMR) images on a population of virtual subjects with a large anatomical variation, introduced using the 4D eXtended Cardiac and Torso (XCAT) computerized human phantom. We investigate two conditional image synthesis approaches grounded on a semantically-consistent mask-guided image generation technique: 4-class and 8-class XCAT-GANs. The 4-class technique relies on only the annotations of the heart; while the 8-class technique employs a predicted multi-tissue label map of the heart-surrounding organs and provides better guidance for our conditional image synthesis. For both techniques, we train our conditional XCAT-GAN with real images paired with corresponding labels and subsequently at the inference time, we substitute the labels with the XCAT derived ones. Therefore, the trained network accurately transfers the tissue-specific textures to the new label maps. By creating 33 virtual subjects of synthetic CMR images at the end-diastolic and end-systolic phases, we evaluate the usefulness of such data in the downstream cardiac cavity segmentation task under different augmentation strategies. Results demonstrate that even with only 20% of real images (40 volumes) seen during training, segmentation performance is retained with the addition of synthetic CMR images. Moreover, the improvement in utilizing synthetic images for augmenting the real data is evident through the reduction of Hausdorff distance up to 28% and an increase in the Dice score up to 5%, indicating a higher similarity to the ground truth in all dimensions.
△ Less
Submitted 31 July, 2020; v1 submitted 27 July, 2020;
originally announced July 2020.
-
4D Semantic Cardiac Magnetic Resonance Image Synthesis on XCAT Anatomical Model
Authors:
Samaneh Abbasi-Sureshjani,
Sina Amirrajab,
Cristian Lorenz,
Juergen Weese,
Josien Pluim,
Marcel Breeuwer
Abstract:
We propose a hybrid controllable image generation method to synthesize anatomically meaningful 3D+t labeled Cardiac Magnetic Resonance (CMR) images. Our hybrid method takes the mechanistic 4D eXtended CArdiac Torso (XCAT) heart model as the anatomical ground truth and synthesizes CMR images via a data-driven Generative Adversarial Network (GAN). We employ the state-of-the-art SPatially Adaptive De…
▽ More
We propose a hybrid controllable image generation method to synthesize anatomically meaningful 3D+t labeled Cardiac Magnetic Resonance (CMR) images. Our hybrid method takes the mechanistic 4D eXtended CArdiac Torso (XCAT) heart model as the anatomical ground truth and synthesizes CMR images via a data-driven Generative Adversarial Network (GAN). We employ the state-of-the-art SPatially Adaptive De-normalization (SPADE) technique for conditional image synthesis to preserve the semantic spatial information of ground truth anatomy. Using the parameterized motion model of the XCAT heart, we generate labels for 25 time frames of the heart for one cardiac cycle at 18 locations for the short axis view. Subsequently, realistic images are generated from these labels, with modality-specific features that are learned from real CMR image data. We demonstrate that style transfer from another cardiac image can be accomplished by using a style encoder network. Due to the flexibility of XCAT in creating new heart models, this approach can result in a realistic virtual population to address different challenges the medical image analysis research community is facing such as expensive data collection. Our proposed method has a great potential to synthesize 4D controllable CMR images with annotations and adaptable styles to be used in various supervised multi-site, multi-vendor applications in medical image analysis.
△ Less
Submitted 20 May, 2020; v1 submitted 17 February, 2020;
originally announced February 2020.
-
Open Power System Data - Frictionless data for electricity system modelling
Authors:
Frauke Wiese,
Ingmar Schlecht,
Wolf-Dieter Bunke,
Clemens Gerbaulet,
Lion Hirth,
Martin Jahn,
Friedrich Kunz,
Casimir Lorenz,
Jonathan Mühlenpfordt,
Juliane Reimann,
Wolf-Peter Schill
Abstract:
The quality of electricity system modelling heavily depends on the input data used. Although a lot of data is publicly available, it is often dispersed, tedious to process and partly contains errors. We argue that a central provision of input data for modelling has the character of a public good: it reduces overall societal costs for quantitative energy research as redundant work is avoided, and i…
▽ More
The quality of electricity system modelling heavily depends on the input data used. Although a lot of data is publicly available, it is often dispersed, tedious to process and partly contains errors. We argue that a central provision of input data for modelling has the character of a public good: it reduces overall societal costs for quantitative energy research as redundant work is avoided, and it improves transparency and reproducibility in electricity system modelling. This paper describes the Open Power System Data platform that aims at realising the efficiency and quality gains of centralised data provision by collecting, checking, processing, aggregating, documenting and publishing data required by most modellers. We conclude that the platform can provide substantial benefits to energy system analysis by raising efficiency of data pre-processing, providing a method for making data pre-processing for energy system modelling traceable, flexible and reproducible and improving the quality of original data published by data providers.
△ Less
Submitted 3 January, 2019; v1 submitted 10 December, 2018;
originally announced December 2018.
-
Deep Learning Based Rib Centerline Extraction and Labeling
Authors:
Matthias Lenga,
Tobias Klinder,
Christian Bürger,
Jens von Berg,
Astrid Franz,
Cristian Lorenz
Abstract:
Automated extraction and labeling of rib centerlines is a typically needed prerequisite for more advanced assisted reading tools that help the radiologist to efficiently inspect all 24 ribs in a CT volume. In this paper, we combine a deep learning-based rib detection with a dedicated centerline extraction algorithm applied to the detection result for the purpose of fast, robust and accurate rib ce…
▽ More
Automated extraction and labeling of rib centerlines is a typically needed prerequisite for more advanced assisted reading tools that help the radiologist to efficiently inspect all 24 ribs in a CT volume. In this paper, we combine a deep learning-based rib detection with a dedicated centerline extraction algorithm applied to the detection result for the purpose of fast, robust and accurate rib centerline extraction and labeling from CT volumes. More specifically, we first apply a fully convolutional neural network (FCNN) to generate a probability map for detecting the first rib pair, the twelfth rib pair, and the collection of all intermediate ribs. In a second stage, a newly designed centerline extraction algorithm is applied to this multi-label probability map. Finally, the distinct detection of first and twelfth rib separately, allows to derive individual rib labels by simple sorting and counting the detected centerlines. We applied our method to CT volumes from 116 patients which included a variety of different challenges and achieved a centerline accuracy of 0.787 mm with respect to manual centerline annotations.
This article is a preprint version of: Lenga M., Klinder T., Bürger C., von Berg J., Franz A., Lorenz C. (2019) Deep Learning Based Rib Centerline Extraction and Labeling. In: Vrtovec T., Yao J., Zheng G., Pozo J. (eds) Computational Methods and Clinical Applications in Musculoskeletal Imaging. MSKI 2018. Lecture Notes in Computer Science, vol 11404. Springer, Cham
△ Less
Submitted 14 January, 2019; v1 submitted 19 September, 2018;
originally announced September 2018.
-
Theano: A Python framework for fast computation of mathematical expressions
Authors:
The Theano Development Team,
Rami Al-Rfou,
Guillaume Alain,
Amjad Almahairi,
Christof Angermueller,
Dzmitry Bahdanau,
Nicolas Ballas,
Frédéric Bastien,
Justin Bayer,
Anatoly Belikov,
Alexander Belopolsky,
Yoshua Bengio,
Arnaud Bergeron,
James Bergstra,
Valentin Bisson,
Josh Bleecher Snyder,
Nicolas Bouchard,
Nicolas Boulanger-Lewandowski,
Xavier Bouthillier,
Alexandre de Brébisson,
Olivier Breuleux,
Pierre-Luc Carrier,
Kyunghyun Cho,
Jan Chorowski,
Paul Christiano
, et al. (88 additional authors not shown)
Abstract:
Theano is a Python library that allows to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements. Theano is being actively and continuously developed since 2008, mu…
▽ More
Theano is a Python library that allows to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements. Theano is being actively and continuously developed since 2008, multiple frameworks have been built on top of it and it has been used to produce many state-of-the-art machine learning models.
The present article is structured as follows. Section I provides an overview of the Theano software and its community. Section II presents the principal features of Theano and how to use them, and compares them with other similar projects. Section III focuses on recently-introduced functionalities and improvements. Section IV compares the performance of Theano against Torch7 and TensorFlow on several machine learning models. Section V discusses current limitations of Theano and potential ways of improving it.
△ Less
Submitted 9 May, 2016;
originally announced May 2016.
