-
Semantic segmentation of multispectral photoacoustic images using deep learning
Authors:
Melanie Schellenberg,
Kris Dreher,
Niklas Holzwarth,
Fabian Isensee,
Annika Reinke,
Nicholas Schreck,
Alexander Seitel,
Minu D. Tizabi,
Lena Maier-Hein,
Janek Gröhl
Abstract:
Photoacoustic (PA) imaging has the potential to revolutionize functional medical imaging in healthcare due to the valuable information on tissue physiology contained in multispectral photoacoustic measurements. Clinical translation of the technology requires conversion of the high-dimensional acquired data into clinically relevant and interpretable information. In this work, we present a deep lear…
▽ More
Photoacoustic (PA) imaging has the potential to revolutionize functional medical imaging in healthcare due to the valuable information on tissue physiology contained in multispectral photoacoustic measurements. Clinical translation of the technology requires conversion of the high-dimensional acquired data into clinically relevant and interpretable information. In this work, we present a deep learning-based approach to semantic segmentation of multispectral photoacoustic images to facilitate image interpretability. Manually annotated photoacoustic {and ultrasound} imaging data are used as reference and enable the training of a deep learning-based segmentation algorithm in a supervised manner. Based on a validation study with experimentally acquired data from 16 healthy human volunteers, we show that automatic tissue segmentation can be used to create powerful analyses and visualizations of multispectral photoacoustic images. Due to the intuitive representation of high-dimensional information, such a preprocessing algorithm could be a valuable means to facilitate the clinical translation of photoacoustic imaging.
△ Less
Submitted 16 April, 2022; v1 submitted 20 May, 2021;
originally announced May 2021.
-
Photoacoustic image synthesis with generative adversarial networks
Authors:
Melanie Schellenberg,
Janek Gröhl,
Kris K. Dreher,
Jan-Hinrich Nölke,
Niklas Holzwarth,
Minu D. Tizabi,
Alexander Seitel,
Lena Maier-Hein
Abstract:
Photoacoustic tomography (PAT) has the potential to recover morphological and functional tissue properties with high spatial resolution. However, previous attempts to solve the optical inverse problem with supervised machine learning were hampered by the absence of labeled reference data. While this bottleneck has been tackled by simulating training data, the domain gap between real and simulated…
▽ More
Photoacoustic tomography (PAT) has the potential to recover morphological and functional tissue properties with high spatial resolution. However, previous attempts to solve the optical inverse problem with supervised machine learning were hampered by the absence of labeled reference data. While this bottleneck has been tackled by simulating training data, the domain gap between real and simulated images remains an unsolved challenge. We propose a novel approach to PAT image synthesis that involves subdividing the challenge of generating plausible simulations into two disjoint problems: (1) Probabilistic generation of realistic tissue morphology, and (2) pixel-wise assignment of corresponding optical and acoustic properties. The former is achieved with Generative Adversarial Networks (GANs) trained on semantically annotated medical imaging data. According to a validation study on a downstream task our approach yields more realistic synthetic images than the traditional model-based approach and could therefore become a fundamental step for deep learning-based quantitative PAT (qPAT).
△ Less
Submitted 25 October, 2022; v1 submitted 29 March, 2021;
originally announced March 2021.
-
Tattoo tomography: Freehand 3D photoacoustic image reconstruction with an optical pattern
Authors:
Niklas Holzwarth,
Melanie Schellenberg,
Janek Gröhl,
Kris Dreher,
Jan-Hinrich Nölke,
Alexander Seitel,
Minu D. Tizabi,
Beat P. Müller-Stich,
Lena Maier-Hein
Abstract:
Purpose: Photoacoustic tomography (PAT) is a novel imaging technique that can spatially resolve both morphological and functional tissue properties, such as the vessel topology and tissue oxygenation. While this capacity makes PAT a promising modality for the diagnosis, treatment and follow-up of various diseases, a current drawback is the limited field-of-view (FoV) provided by the conventionally…
▽ More
Purpose: Photoacoustic tomography (PAT) is a novel imaging technique that can spatially resolve both morphological and functional tissue properties, such as the vessel topology and tissue oxygenation. While this capacity makes PAT a promising modality for the diagnosis, treatment and follow-up of various diseases, a current drawback is the limited field-of-view (FoV) provided by the conventionally applied 2D probes.
Methods: In this paper, we present a novel approach to 3D reconstruction of PAT data (Tattoo tomography) that does not require an external tracking system and can smoothly be integrated into clinical workflows. It is based on an optical pattern placed on the region of interest prior to image acquisition. This pattern is designed in a way that a tomographic image of it enables the recovery of the probe pose relative to the coordinate system of the pattern. This allows the transformation of a sequence of acquired PA images into one common global coordinate system and thus the consistent 3D reconstruction of PAT imaging data.
Results: An initial feasibility study conducted with experimental phantom data and in vivo forearm data indicates that the Tattoo approach is well-suited for 3D reconstruction of PAT data with high accuracy and precision.
Conclusion: In contrast to previous approaches to 3D ultrasound (US) or PAT reconstruction, the Tattoo approach neither requires complex external hardware nor training data acquired for a specific application. It could thus become a valuable tool for clinical freehand PAT.
△ Less
Submitted 11 November, 2020; v1 submitted 10 November, 2020;
originally announced November 2020.
-
Photoacoustic monitoring of blood oxygenation during neurosurgical interventions
Authors:
Thomas Kirchner,
Janek Gröhl,
Niklas Holzwarth,
Mildred A. Herrera,
Tim Adler,
Adrián Hernández-Aguilera,
Edgar Santos,
Lena Maier-Hein
Abstract:
Multispectral photoacoustic (PA) imaging is a prime modality to monitor hemodynamics and changes in blood oxygenation (sO2). Although sO2 changes can be an indicator of brain activity both in normal and in pathological conditions, PA imaging of the brain has mainly focused on small animal models with lissencephalic brains. Therefore, the purpose of this work was to investigate the usefulness of mu…
▽ More
Multispectral photoacoustic (PA) imaging is a prime modality to monitor hemodynamics and changes in blood oxygenation (sO2). Although sO2 changes can be an indicator of brain activity both in normal and in pathological conditions, PA imaging of the brain has mainly focused on small animal models with lissencephalic brains. Therefore, the purpose of this work was to investigate the usefulness of multispectral PA imaging in assessing sO2 in a gyrencephalic brain. To this end, we continuously imaged a porcine brain as part of an open neurosurgical intervention with a handheld PA and ultrasonic (US) imaging system in vivo. Throughout the experiment, we varied respiratory oxygen and continuously measured arterial blood gases. The arterial blood oxygenation (SaO2) values derived by the blood gas analyzer were used as a reference to compare the performance of linear spectral unmixing algorithms in this scenario. According to our experiment, PA imaging can be used to monitor sO2 in the porcine cerebral cortex. While linear spectral unmixing algorithms are well-suited for detecting changes in oxygenation, there are limits with respect to the accurate quantification of sO2, especially in depth. Overall, we conclude that multispectral PA imaging can potentially be a valuable tool for change detection of sO2 in the cerebral cortex of a gyrencephalic brain. The spectral unmixing algorithms investigated in this work will be made publicly available as part of the open-source software platform Medical Imaging Interaction Toolkit (MITK).
△ Less
Submitted 26 April, 2019;
originally announced April 2019.
-
Evolutionary optimization of PAW data-sets for accurate high pressure simulations
Authors:
Kanchan Sarkar,
Mehmet Topsakal,
N. A. W. Holzwarth,
Renata M. Wentzcovitch
Abstract:
We examine the challenge of performing accurate electronic structure calculations at high pressures by comparing the results of all-electron full potential linearized augmented-plane-wave calculations with those of the projector augmented wave (PAW) method. In particular, we focus on develo** an automated and consistent way of generating transferable PAW data-sets that can closely produce the al…
▽ More
We examine the challenge of performing accurate electronic structure calculations at high pressures by comparing the results of all-electron full potential linearized augmented-plane-wave calculations with those of the projector augmented wave (PAW) method. In particular, we focus on develo** an automated and consistent way of generating transferable PAW data-sets that can closely produce the all electron equation of state defined from zero to arbitrary high pressures. The technique we propose is an evolutionary search procedure that exploits the ATOMPAW code to generate atomic data-sets and the Quantum ESPRESSO software suite for total energy calculations. We demonstrate different aspects of its workability by optimizing PAW basis functions of some elements relatively abundant in planetary interiors. In addition, we introduce a new measure of atomic data-set goodness by considering their performance uniformity over an enlarged pressure range.
△ Less
Submitted 25 October, 2016;
originally announced October 2016.
-
Generation of Projector Augmented-Wave atomic data: a 71 elements validated table in the XML format
Authors:
François Jollet,
Marc Torrent,
Natalie Holzwarth
Abstract:
A Projector Augmented Wave (PAW) atomic data file is needed to be generated for each element, and plays in the PAW method the role of the pseudopotential file for norm-conserving (NC) or ultra-soft (US) plane wave calculations. In this paper, we present a review on how to obtain these data as well as results concerning their accuracy, their transferability and their efficiency for bulk solids. Fol…
▽ More
A Projector Augmented Wave (PAW) atomic data file is needed to be generated for each element, and plays in the PAW method the role of the pseudopotential file for norm-conserving (NC) or ultra-soft (US) plane wave calculations. In this paper, we present a review on how to obtain these data as well as results concerning their accuracy, their transferability and their efficiency for bulk solids. Following \cite{Lejaeghere}, we propose a new criterium to test PAW atomic data and we provide a new table written in a XML format potentially readable by every PAW electronic structure code.
△ Less
Submitted 1 October, 2013; v1 submitted 27 September, 2013;
originally announced September 2013.
-
Exact Ground State Energy of Hubbard Rings in the Atomic Limit
Authors:
W. B. Hodge,
N. A. W. Holzwarth,
W. C. Kerr
Abstract:
Using a straightforward extension of the analysis of Lieb and Wu, we derive a simple analytic form for the ground state energy of a one-dimensional Hubbard ring in the atomic limit. This result is valid for an \textit{arbitrary} number of lattice sites $L$ and electrons $N \leq L$. Furthermore, our analysis, including an application of the theory of stochastic matrices, provides insight into the…
▽ More
Using a straightforward extension of the analysis of Lieb and Wu, we derive a simple analytic form for the ground state energy of a one-dimensional Hubbard ring in the atomic limit. This result is valid for an \textit{arbitrary} number of lattice sites $L$ and electrons $N \leq L$. Furthermore, our analysis, including an application of the theory of stochastic matrices, provides insight into the degeneracy and spin properties of the ground states in the atomic limit. We give numerical results which illustrate how the atomic limit is approached.
△ Less
Submitted 6 October, 2009; v1 submitted 10 June, 2008;
originally announced June 2008.
