-
Robust Tumor Segmentation with Hyperspectral Imaging and Graph Neural Networks
Authors:
Mayar Lotfy,
Anna Alperovich,
Tommaso Giannantonio,
Bjorn Barz,
Xiaohan Zhang,
Felix Holm,
Nassir Navab,
Felix Boehm,
Carolin Schwamborn,
Thomas K. Hoffmann,
Patrick J. Schuler
Abstract:
Segmenting the boundary between tumor and healthy tissue during surgical cancer resection poses a significant challenge. In recent years, Hyperspectral Imaging (HSI) combined with Machine Learning (ML) has emerged as a promising solution. However, due to the extensive information contained within the spectral domain, most ML approaches primarily classify individual HSI (super-)pixels, or tiles, wi…
▽ More
Segmenting the boundary between tumor and healthy tissue during surgical cancer resection poses a significant challenge. In recent years, Hyperspectral Imaging (HSI) combined with Machine Learning (ML) has emerged as a promising solution. However, due to the extensive information contained within the spectral domain, most ML approaches primarily classify individual HSI (super-)pixels, or tiles, without taking into account their spatial context. In this paper, we propose an improved methodology that leverages the spatial context of tiles for more robust and smoother segmentation. To address the irregular shapes of tiles, we utilize Graph Neural Networks (GNNs) to propagate context information across neighboring regions. The features for each tile within the graph are extracted using a Convolutional Neural Network (CNN), which is trained simultaneously with the subsequent GNN. Moreover, we incorporate local image quality metrics into the loss function to enhance the training procedure's robustness against low-quality regions in the training images. We demonstrate the superiority of our proposed method using a clinical ex vivo dataset consisting of 51 HSI images from 30 patients. Despite the limited dataset, the GNN-based model significantly outperforms context-agnostic approaches, accurately distinguishing between healthy and tumor tissues, even in images from previously unseen patients. Furthermore, we show that our carefully designed loss function, accounting for local image quality, results in additional improvements. Our findings demonstrate that context-aware GNN algorithms can robustly find tumor demarcations on HSI images, ultimately contributing to better surgery success and patient outcome.
△ Less
Submitted 20 November, 2023;
originally announced November 2023.
-
Context-based Normalization of Histological Stains using Deep Convolutional Features
Authors:
Daniel Bug,
Steffen Schneider,
Anne Grote,
Eva Oswald,
Friedrich Feuerhake,
Julia Schüler,
Dorit Merhof
Abstract:
While human observers are able to cope with variations in color and appearance of histological stains, digital pathology algorithms commonly require a well-normalized setting to achieve peak performance, especially when a limited amount of labeled data is available. This work provides a fully automated, end-to-end learning-based setup for normalizing histological stains, which considers the textur…
▽ More
While human observers are able to cope with variations in color and appearance of histological stains, digital pathology algorithms commonly require a well-normalized setting to achieve peak performance, especially when a limited amount of labeled data is available. This work provides a fully automated, end-to-end learning-based setup for normalizing histological stains, which considers the texture context of the tissue. We introduce Feature Aware Normalization, which extends the framework of batch normalization in combination with gating elements from Long Short-Term Memory units for normalization among different spatial regions of interest. By incorporating a pretrained deep neural network as a feature extractor steering a pixelwise processing pipeline, we achieve excellent normalization results and ensure a consistent representation of color and texture. The evaluation comprises a comparison of color histogram deviations, structural similarity and measures the color volume obtained by the different methods.
△ Less
Submitted 14 August, 2017;
originally announced August 2017.
-
LightPanel: Active Mobile Platform for Dense 3D Modelling
Authors:
Jonas Schuler,
Reza Sabzevari,
Davide Scaramuzza
Abstract:
In this paper we introduce a novel platform for dense 3D modelling. This platform is an active image acquisition setup assisted with a set of light sources and a distance sensor. The hardware setup is designed for being mounted on a mobile robot which is remotely driven to create accurate dense 3D models from out-of-reach objects. For this reason, the object is actively illuminated by the imaging…
▽ More
In this paper we introduce a novel platform for dense 3D modelling. This platform is an active image acquisition setup assisted with a set of light sources and a distance sensor. The hardware setup is designed for being mounted on a mobile robot which is remotely driven to create accurate dense 3D models from out-of-reach objects. For this reason, the object is actively illuminated by the imaging setup and Photometric Stereo is used to recover the dense 3D model. The proposed image acquisition setup, called LightPanel, is described from design to calibration and discusses the practical challenges of using Photometric Stereo under uncontrolled lighting conditions.
△ Less
Submitted 16 June, 2015;
originally announced June 2015.
-
Learning to Deblur
Authors:
Christian J. Schuler,
Michael Hirsch,
Stefan Harmeling,
Bernhard Schölkopf
Abstract:
We describe a learning-based approach to blind image deconvolution. It uses a deep layered architecture, parts of which are borrowed from recent work on neural network learning, and parts of which incorporate computations that are specific to image deconvolution. The system is trained end-to-end on a set of artificially generated training examples, enabling competitive performance in blind deconvo…
▽ More
We describe a learning-based approach to blind image deconvolution. It uses a deep layered architecture, parts of which are borrowed from recent work on neural network learning, and parts of which incorporate computations that are specific to image deconvolution. The system is trained end-to-end on a set of artificially generated training examples, enabling competitive performance in blind deconvolution, both with respect to quality and runtime.
△ Less
Submitted 28 June, 2014;
originally announced June 2014.
-
Image denoising with multi-layer perceptrons, part 2: training trade-offs and analysis of their mechanisms
Authors:
Harold Christopher Burger,
Christian J. Schuler,
Stefan Harmeling
Abstract:
Image denoising can be described as the problem of map** from a noisy image to a noise-free image. In another paper, we show that multi-layer perceptrons can achieve outstanding image denoising performance for various types of noise (additive white Gaussian noise, mixed Poisson-Gaussian noise, JPEG artifacts, salt-and-pepper noise and noise resembling stripes). In this work we discuss in detail…
▽ More
Image denoising can be described as the problem of map** from a noisy image to a noise-free image. In another paper, we show that multi-layer perceptrons can achieve outstanding image denoising performance for various types of noise (additive white Gaussian noise, mixed Poisson-Gaussian noise, JPEG artifacts, salt-and-pepper noise and noise resembling stripes). In this work we discuss in detail which trade-offs have to be considered during the training procedure. We will show how to achieve good results and which pitfalls to avoid. By analysing the activation patterns of the hidden units we are able to make observations regarding the functioning principle of multi-layer perceptrons trained for image denoising.
△ Less
Submitted 7 November, 2012;
originally announced November 2012.
-
Image denoising with multi-layer perceptrons, part 1: comparison with existing algorithms and with bounds
Authors:
Harold Christopher Burger,
Christian J. Schuler,
Stefan Harmeling
Abstract:
Image denoising can be described as the problem of map** from a noisy image to a noise-free image. The best currently available denoising methods approximate this map** with cleverly engineered algorithms. In this work we attempt to learn this map** directly with plain multi layer perceptrons (MLP) applied to image patches. We will show that by training on large image databases we are able t…
▽ More
Image denoising can be described as the problem of map** from a noisy image to a noise-free image. The best currently available denoising methods approximate this map** with cleverly engineered algorithms. In this work we attempt to learn this map** directly with plain multi layer perceptrons (MLP) applied to image patches. We will show that by training on large image databases we are able to outperform the current state-of-the-art image denoising methods. In addition, our method achieves results that are superior to one type of theoretical bound and goes a large way toward closing the gap with a second type of theoretical bound. Our approach is easily adapted to less extensively studied types of noise, such as mixed Poisson-Gaussian noise, JPEG artifacts, salt-and-pepper noise and noise resembling stripes, for which we achieve excellent results as well. We will show that combining a block-matching procedure with MLPs can further improve the results on certain images. In a second paper, we detail the training trade-offs and the inner mechanisms of our MLPs.
△ Less
Submitted 9 November, 2012; v1 submitted 7 November, 2012;
originally announced November 2012.
-
Entropy Search for Information-Efficient Global Optimization
Authors:
Philipp Hennig,
Christian J. Schuler
Abstract:
Contemporary global optimization algorithms are based on local measures of utility, rather than a probability measure over location and value of the optimum. They thus attempt to collect low function values, not to learn about the optimum. The reason for the absence of probabilistic global optimizers is that the corresponding inference problem is intractable in several ways. This paper develops de…
▽ More
Contemporary global optimization algorithms are based on local measures of utility, rather than a probability measure over location and value of the optimum. They thus attempt to collect low function values, not to learn about the optimum. The reason for the absence of probabilistic global optimizers is that the corresponding inference problem is intractable in several ways. This paper develops desiderata for probabilistic optimization algorithms, then presents a concrete algorithm which addresses each of the computational intractabilities with a sequence of approximations and explicitly adresses the decision problem of maximizing information gain from each evaluation.
△ Less
Submitted 6 December, 2011;
originally announced December 2011.
