-
Self2Seg: Single-Image Self-Supervised Joint Segmentation and Denoising
Authors:
Nadja Gruber,
Johannes Schwab,
Noémie Debroux,
Nicolas Papadakis,
Markus Haltmeier
Abstract:
We develop Self2Seg, a self-supervised method for the joint segmentation and denoising of a single image. To this end, we combine the advantages of variational segmentation with self-supervised deep learning. One major benefit of our method lies in the fact, that in contrast to data-driven methods, where huge amounts of labeled samples are necessary, Self2Seg segments an image into meaningful regi…
▽ More
We develop Self2Seg, a self-supervised method for the joint segmentation and denoising of a single image. To this end, we combine the advantages of variational segmentation with self-supervised deep learning. One major benefit of our method lies in the fact, that in contrast to data-driven methods, where huge amounts of labeled samples are necessary, Self2Seg segments an image into meaningful regions without any training database. Moreover, we demonstrate that self-supervised denoising itself is significantly improved through the region-specific learning of Self2Seg. Therefore, we introduce a novel self-supervised energy functional in which denoising and segmentation are coupled in a way that both tasks benefit from each other. We propose a unified optimisation strategy and numerically show that for noisy microscopy images our proposed joint approach outperforms its sequential counterpart as well as alternative methods focused purely on denoising or segmentation.
△ Less
Submitted 29 April, 2024; v1 submitted 19 September, 2023;
originally announced September 2023.
-
Earth Virtualization Engines -- A Technical Perspective
Authors:
Torsten Hoefler,
Bjorn Stevens,
Andreas F. Prein,
Johanna Baehr,
Thomas Schulthess,
Thomas F. Stocker,
John Taylor,
Daniel Klocke,
Pekka Manninen,
Piers M. Forster,
Tobias Kölling,
Nicolas Gruber,
Hartwig Anzt,
Claudia Frauen,
Florian Ziemen,
Milan Klöwer,
Karthik Kashinath,
Christoph Schär,
Oliver Fuhrer,
Bryan N. Lawrence
Abstract:
Participants of the Berlin Summit on Earth Virtualization Engines (EVEs) discussed ideas and concepts to improve our ability to cope with climate change. EVEs aim to provide interactive and accessible climate simulations and data for a wide range of users. They combine high-resolution physics-based models with machine learning techniques to improve the fidelity, efficiency, and interpretability of…
▽ More
Participants of the Berlin Summit on Earth Virtualization Engines (EVEs) discussed ideas and concepts to improve our ability to cope with climate change. EVEs aim to provide interactive and accessible climate simulations and data for a wide range of users. They combine high-resolution physics-based models with machine learning techniques to improve the fidelity, efficiency, and interpretability of climate projections. At their core, EVEs offer a federated data layer that enables simple and fast access to exabyte-sized climate data through simple interfaces. In this article, we summarize the technical challenges and opportunities for develo** EVEs, and argue that they are essential for addressing the consequences of climate change.
△ Less
Submitted 16 September, 2023;
originally announced September 2023.
-
Variational multichannel multiclass segmentation using unsupervised lifting with CNNs
Authors:
Nadja Gruber,
Johannes Schwab,
Sebastien Court,
Elke Gizewski,
Markus Haltmeier
Abstract:
We propose an unsupervised image segmentation approach, that combines a variational energy functional and deep convolutional neural networks. The variational part is based on a recent multichannel multiphase Chan-Vese model, which is capable to extract useful information from multiple input images simultaneously. We implement a flexible multiclass segmentation method that divides a given image int…
▽ More
We propose an unsupervised image segmentation approach, that combines a variational energy functional and deep convolutional neural networks. The variational part is based on a recent multichannel multiphase Chan-Vese model, which is capable to extract useful information from multiple input images simultaneously. We implement a flexible multiclass segmentation method that divides a given image into $K$ different regions. We use convolutional neural networks (CNNs) targeting a pre-decomposition of the image. By subsequently minimising the segmentation functional, the final segmentation is obtained in a fully unsupervised manner. Special emphasis is given to the extraction of informative feature maps serving as a starting point for the segmentation. The initial results indicate that the proposed method is able to decompose and segment the different regions of various types of images, such as texture and medical images and compare its performance with another multiphase segmentation method.
△ Less
Submitted 16 June, 2023; v1 submitted 4 February, 2023;
originally announced February 2023.
-
Lifting-based variational multiclass segmentation algorithm: design, convergence analysis, and implementation with applications in medical imaging
Authors:
Nadja Gruber,
Johannes Schwab,
Sebastien Court,
Elke Gizewski,
Markus Haltmeier
Abstract:
We propose, analyze and realize a variational multiclass segmentation scheme that partitions a given image into multiple regions exhibiting specific properties. Our method determines multiple functions that encode the segmentation regions by minimizing an energy functional combining information from different channels. Multichannel image data can be obtained by lifting the image into a higher dime…
▽ More
We propose, analyze and realize a variational multiclass segmentation scheme that partitions a given image into multiple regions exhibiting specific properties. Our method determines multiple functions that encode the segmentation regions by minimizing an energy functional combining information from different channels. Multichannel image data can be obtained by lifting the image into a higher dimensional feature space using specific multichannel filtering or may already be provided by the imaging modality under consideration, such as an RGB image or multimodal medical data. Experimental results show that the proposed method performs well in various scenarios. In particular, promising results are presented for two medical applications involving classification of brain abscess and tumor growth, respectively. As main theoretical contributions, we prove the existence of global minimizers of the proposed energy functional and show its stability and convergence with respect to noisy inputs. In particular, these results also apply to the special case of binary segmentation, and these results are also novel in this particular situation.
△ Less
Submitted 18 September, 2023; v1 submitted 9 February, 2022;
originally announced February 2022.
-
A Joint Deep Learning Approach for Automated Liver and Tumor Segmentation
Authors:
Nadja Gruber,
Stephan Antholzer,
Werner Jaschke,
Christian Kremser,
Markus Haltmeier
Abstract:
Hepatocellular carcinoma (HCC) is the most common type of primary liver cancer in adults, and the most common cause of death of people suffering from cirrhosis. The segmentation of liver lesions in CT images allows assessment of tumor load, treatment planning, prognosis and monitoring of treatment response. Manual segmentation is a very time-consuming task and in many cases, prone to inaccuracies…
▽ More
Hepatocellular carcinoma (HCC) is the most common type of primary liver cancer in adults, and the most common cause of death of people suffering from cirrhosis. The segmentation of liver lesions in CT images allows assessment of tumor load, treatment planning, prognosis and monitoring of treatment response. Manual segmentation is a very time-consuming task and in many cases, prone to inaccuracies and automatic tools for tumor detection and segmentation are desirable. In this paper, we compare two network architectures, one that is composed of one neural network and manages the segmentation task in one step and one that consists of two consecutive fully convolutional neural networks. The first network segments the liver whereas the second network segments the actual tumor inside the liver. Our networks are trained on a subset of the LiTS (Liver Tumor Segmentation) Challenge and evaluated on data.
△ Less
Submitted 13 March, 2020; v1 submitted 21 February, 2019;
originally announced February 2019.
