-
METGAN: Generative Tumour Inpainting and Modality Synthesis in Light Sheet Microscopy
Authors:
Izabela Horvath,
Johannes C. Paetzold,
Oliver Schoppe,
Rami Al-Maskari,
Ivan Ezhov,
Suprosanna Shit,
Hongwei Li,
Ali Ertuerk,
Bjoern H. Menze
Abstract:
Novel multimodal imaging methods are capable of generating extensive, super high resolution datasets for preclinical research. Yet, a massive lack of annotations prevents the broad use of deep learning to analyze such data. So far, existing generative models fail to mitigate this problem because of frequent labeling errors. In this paper, we introduce a novel generative method which leverages real…
▽ More
Novel multimodal imaging methods are capable of generating extensive, super high resolution datasets for preclinical research. Yet, a massive lack of annotations prevents the broad use of deep learning to analyze such data. So far, existing generative models fail to mitigate this problem because of frequent labeling errors. In this paper, we introduce a novel generative method which leverages real anatomical information to generate realistic image-label pairs of tumours. We construct a dual-pathway generator, for the anatomical image and label, trained in a cycle-consistent setup, constrained by an independent, pretrained segmentor. The generated images yield significant quantitative improvement compared to existing methods. To validate the quality of synthesis, we train segmentation networks on a dataset augmented with the synthetic data, substantially improving the segmentation over baseline.
△ Less
Submitted 23 April, 2021; v1 submitted 22 April, 2021;
originally announced April 2021.
-
Copyspace: Where to Write on Images?
Authors:
Jessica M. Lundin,
Michael Sollami,
Brian Lonsdorf,
Alan Ross,
Owen Schoppe,
David Woodward,
Sönke Rohde
Abstract:
The placement of text over an image is an important part of producing high-quality visual designs. Automating this work by determining appropriate position, orientation, and style for textual elements requires understanding the contents of the background image. We refer to the search for aesthetic parameters of text rendered over images as "copyspace detection", noting that this task is distinct f…
▽ More
The placement of text over an image is an important part of producing high-quality visual designs. Automating this work by determining appropriate position, orientation, and style for textual elements requires understanding the contents of the background image. We refer to the search for aesthetic parameters of text rendered over images as "copyspace detection", noting that this task is distinct from foreground-background separation. We have developed solutions using one and two stage object detection methodologies trained on an expertly labeled data. This workshop will examine such algorithms for copyspace detection and demonstrate their application in generative design models and pipelines such as Einstein Designer.
△ Less
Submitted 3 December, 2020;
originally announced December 2020.
-
Red-GAN: Attacking class imbalance via conditioned generation. Yet another perspective on medical image synthesis for skin lesion dermoscopy and brain tumor MRI
Authors:
Ahmad B Qasim,
Ivan Ezhov,
Suprosanna Shit,
Oliver Schoppe,
Johannes C Paetzold,
Anjany Sekuboyina,
Florian Kofler,
Jana Lipkova,
Hongwei Li,
Bjoern Menze
Abstract:
Exploiting learning algorithms under scarce data regimes is a limitation and a reality of the medical imaging field. In an attempt to mitigate the problem, we propose a data augmentation protocol based on generative adversarial networks. We condition the networks at a pixel-level (segmentation mask) and at a global-level information (acquisition environment or lesion type). Such conditioning provi…
▽ More
Exploiting learning algorithms under scarce data regimes is a limitation and a reality of the medical imaging field. In an attempt to mitigate the problem, we propose a data augmentation protocol based on generative adversarial networks. We condition the networks at a pixel-level (segmentation mask) and at a global-level information (acquisition environment or lesion type). Such conditioning provides immediate access to the image-label pairs while controlling global class specific appearance of the synthesized images. To stimulate synthesis of the features relevant for the segmentation task, an additional passive player in a form of segmentor is introduced into the adversarial game. We validate the approach on two medical datasets: BraTS, ISIC. By controlling the class distribution through injection of synthetic images into the training set we achieve control over the accuracy levels of the datasets' classes.
△ Less
Submitted 27 March, 2021; v1 submitted 22 April, 2020;
originally announced April 2020.
