Showing 1–7 of 7 results for author: Bergholt, M S

Synthetic white balancing for intra-operative hyperspectral imaging

Authors: Anisha Bahl, Conor C. Horgan, Mirek Janatka, Oscar J. MacCormac, Philip Noonan, Yi**g Xie, Jianrong Qiu, Nicola Cavalcanti, Philipp Fürnstahl, Michael Ebner, Mads S. Bergholt, Jonathan Shapey, Tom Vercauteren

Abstract: Hyperspectral imaging shows promise for surgical applications to non-invasively provide spatially-resolved, spectral information. For calibration purposes, a white reference image of a highly-reflective Lambertian surface should be obtained under the same imaging conditions. Standard white references are not sterilizable, and so are unsuitable for surgical environments. We demonstrate the necessit… ▽ More Hyperspectral imaging shows promise for surgical applications to non-invasively provide spatially-resolved, spectral information. For calibration purposes, a white reference image of a highly-reflective Lambertian surface should be obtained under the same imaging conditions. Standard white references are not sterilizable, and so are unsuitable for surgical environments. We demonstrate the necessity for in situ white references and address this by proposing a novel, sterile, synthetic reference construction algorithm. The use of references obtained at different distances and lighting conditions to the subject were examined. Spectral and color reconstructions were compared with standard measurements qualitatively and quantitatively, using $ΔE$ and normalised RMSE respectively. The algorithm forms a composite image from a video of a standard sterile ruler, whose imperfect reflectivity is compensated for. The reference is modelled as the product of independent spatial and spectral components, and a scalar factor accounting for gain, exposure, and light intensity. Evaluation of synthetic references against ideal but non-sterile references is performed using the same metrics alongside pixel-by-pixel errors. Finally, intraoperative integration is assessed though cadaveric experiments. Improper white balancing leads to increases in all quantitative and qualitative errors. Synthetic references achieve median pixel-by-pixel errors lower than 6.5% and produce similar reconstructions and errors to an ideal reference. The algorithm integrated well into surgical workflow, achieving median pixel-by-pixel errors of 4.77%, while maintaining good spectral and color reconstruction. △ Less

Submitted 24 July, 2023; originally announced July 2023.

Comments: 22 pages, 10 figures

Deep Reinforcement Learning Based System for Intraoperative Hyperspectral Video Autofocusing

Authors: Charlie Budd, Jianrong Qiu, Oscar MacCormac, Martin Huber, Christopher Mower, Mirek Janatka, Théo Trotouin, Jonathan Shapey, Mads S. Bergholt, Tom Vercauteren

Abstract: Hyperspectral imaging (HSI) captures a greater level of spectral detail than traditional optical imaging, making it a potentially valuable intraoperative tool when precise tissue differentiation is essential. Hardware limitations of current optical systems used for handheld real-time video HSI result in a limited focal depth, thereby posing usability issues for integration of the technology into t… ▽ More Hyperspectral imaging (HSI) captures a greater level of spectral detail than traditional optical imaging, making it a potentially valuable intraoperative tool when precise tissue differentiation is essential. Hardware limitations of current optical systems used for handheld real-time video HSI result in a limited focal depth, thereby posing usability issues for integration of the technology into the operating room. This work integrates a focus-tunable liquid lens into a video HSI exoscope, and proposes novel video autofocusing methods based on deep reinforcement learning. A first-of-its-kind robotic focal-time scan was performed to create a realistic and reproducible testing dataset. We benchmarked our proposed autofocus algorithm against traditional policies, and found our novel approach to perform significantly ($p<0.05$) better than traditional techniques ($0.070\pm.098$ mean absolute focal error compared to $0.146\pm.148$). In addition, we performed a blinded usability trial by having two neurosurgeons compare the system with different autofocus policies, and found our novel approach to be the most favourable, making our system a desirable addition for intraoperative HSI. △ Less

Submitted 21 July, 2023; originally announced July 2023.

Comments: To be presented at MICCAI 2023

Unsupervised denoising of Raman spectra with cycle-consistent generative adversarial networks

Authors: Ciaran Bench, Mads S. Bergholt, Mohamed Ali al-Badri

Abstract: Raman spectroscopy can provide insight into the molecular composition of cells and tissue. Consequently, it can be used as a powerful diagnostic tool, e.g. to help identify changes in molecular contents with the onset of disease. But robust information about sample composition may only be recovered with long acquisition times that produce spectra with a high signal to noise ratio. This acts as a b… ▽ More Raman spectroscopy can provide insight into the molecular composition of cells and tissue. Consequently, it can be used as a powerful diagnostic tool, e.g. to help identify changes in molecular contents with the onset of disease. But robust information about sample composition may only be recovered with long acquisition times that produce spectra with a high signal to noise ratio. This acts as a bottleneck on experimental workflows, driving a desire for effective spectral denoising techniques. Denoising algorithms based on deep neural networks have been shown superior to `classical' approaches, but require the use of bespoke paired datasets (i.e. spectra acquired from the same set of samples acquired with both long and short acquisition times) that require significant effort to generate. Here, we propose an unsupervised denoising approach that does not require paired data. We cast the problem of spectral denoising as a style transfer task and show how cycle-consistent generative adversarial networks can provide significant performance benefits over classical denoising techniques. △ Less

Submitted 1 August, 2023; v1 submitted 2 July, 2023; originally announced July 2023.

Comments: Added HSI evaluation, and changed formatting of some figures

spectrai: A deep learning framework for spectral data

Authors: Conor C. Horgan, Mads S. Bergholt

Abstract: Deep learning computer vision techniques have achieved many successes in recent years across numerous imaging domains. However, the application of deep learning to spectral data remains a complex task due to the need for augmentation routines, specific architectures for spectral data, and significant memory requirements. Here we present spectrai, an open-source deep learning framework designed to… ▽ More Deep learning computer vision techniques have achieved many successes in recent years across numerous imaging domains. However, the application of deep learning to spectral data remains a complex task due to the need for augmentation routines, specific architectures for spectral data, and significant memory requirements. Here we present spectrai, an open-source deep learning framework designed to facilitate the training of neural networks on spectral data and enable comparison between different methods. Spectrai provides numerous built-in spectral data pre-processing and augmentation methods, neural networks for spectral data including spectral (image) denoising, spectral (image) classification, spectral image segmentation, and spectral image super-resolution. Spectrai includes both command line and graphical user interfaces (GUI) designed to guide users through model and hyperparameter decisions for a wide range of applications. △ Less

Submitted 17 August, 2021; originally announced August 2021.

High-throughput molecular imaging via deep learning enabled Raman spectroscopy

Authors: Conor C. Horgan, Magnus Jensen, Anika Nagelkerke, Jean-Phillipe St-Pierre, Tom Vercauteren, Molly M. Stevens, Mads S. Bergholt

Abstract: Raman spectroscopy enables non-destructive, label-free imaging with unprecedented molecular contrast but is limited by slow data acquisition, largely preventing high-throughput imaging applications. Here, we present a comprehensive framework for higher-throughput molecular imaging via deep learning enabled Raman spectroscopy, termed DeepeR, trained on a large dataset of hyperspectral Raman images,… ▽ More Raman spectroscopy enables non-destructive, label-free imaging with unprecedented molecular contrast but is limited by slow data acquisition, largely preventing high-throughput imaging applications. Here, we present a comprehensive framework for higher-throughput molecular imaging via deep learning enabled Raman spectroscopy, termed DeepeR, trained on a large dataset of hyperspectral Raman images, with over 1.5 million spectra (400 hours of acquisition) in total. We firstly perform denoising and reconstruction of low signal-to-noise ratio Raman molecular signatures via deep learning, with a 9x improvement in mean squared error over state-of-the-art Raman filtering methods. Next, we develop a neural network for robust 2-4x super-resolution of hyperspectral Raman images that preserves molecular cellular information. Combining these approaches, we achieve Raman imaging speed-ups of up to 160x, enabling high resolution, high signal-to-noise ratio cellular imaging in under one minute. Finally, transfer learning is applied to extend DeepeR from cell to tissue-scale imaging. DeepeR provides a foundation that will enable a host of higher-throughput Raman spectroscopy and molecular imaging applications across biomedicine. △ Less

Submitted 28 September, 2020; originally announced September 2020.

Fluorescence-Guided Raman Spectroscopy for Tumour Margin Delineation

Authors: Conor C. Horgan, Mads S. Bergholt, May Zaw Thin, Anika Nagelkerke, Robert Kennedy, Tammy L. Kalber, Daniel J. Stuckey, Molly M. Stevens

Abstract: The initial detection and identification of suspicious lesions and the precise delineation of tumour mar-gins are essential for successful tumour resection, with progression-free survival linked to rates of complete resection. However, post-surgical positive margin rates remain high for many cancers and despite numerous advances in intraoperative imaging and diagnostic technologies, there exists n… ▽ More The initial detection and identification of suspicious lesions and the precise delineation of tumour mar-gins are essential for successful tumour resection, with progression-free survival linked to rates of complete resection. However, post-surgical positive margin rates remain high for many cancers and despite numerous advances in intraoperative imaging and diagnostic technologies, there exists no single modality that can adequately perform both tumoural detection and delineation. Here, we demonstrate a multimodal computer vision-based diagnostic system capable of both the gross detection and identification of suspicious lesions and the precise delineation of disease margins. We first show that through visual tracking of a spectroscopic probe, we enable real-time tumour margin delineation both for ex vivo human tumour biopsies and for an in vivo tumour xenograft mouse model. We then demonstrate that the combination of Raman spectroscopic diagnoses with protoporphyrin IX (PPIX) fluorescence imaging enables fluorescence-guided Raman spectroscopic margin delineation. Our fluorescence-guided Raman spectroscopic system achieves superior margin delineation accuracy to fluorescence imaging alone, demonstrating the potential for our system to achieve improved clinical outcomes for tumour resection surgeries. △ Less

Submitted 24 September, 2020; originally announced September 2020.

Integrated Photodynamic Raman Theranostics for Cancer Diagnosis, Treatment, and Post-Treatment Molecular Monitoring

Authors: Conor C. Horgan, Mads S. Bergholt, Anika Nagelkerke, May Zaw Thin, Isaac J. Pence, Ulrike Kauscher, Tammy L. Kalber, Daniel J. Stuckey, Molly M. Stevens

Abstract: Theranostics, the combination of diagnosis and therapy, has long held promise as a means to achieving personalised precision cancer treatments. However, despite its potential, theranostics has yet to realise significant clinical translation, largely due the complexity and overriding toxicity concerns of existing theranostic nanoparticle strategies. Here, we present an alternative nanoparticle-free… ▽ More Theranostics, the combination of diagnosis and therapy, has long held promise as a means to achieving personalised precision cancer treatments. However, despite its potential, theranostics has yet to realise significant clinical translation, largely due the complexity and overriding toxicity concerns of existing theranostic nanoparticle strategies. Here, we present an alternative nanoparticle-free theranostic approach based on simultaneous Raman spectroscopy and photodynamic therapy (PDT) in an integrated clinical platform for cancer theranostics. We detail the compatibility of Raman spectroscopy and PDT for cancer theranostics, whereby Raman spectroscopic diagnosis can be performed on PDT photosensitiser-positive cells and tissues without inadvertent photosensitiser activation/photobleaching or impaired diagnostic capacity. We further demonstrate that our theranostic platform enables in vivo tumour diagnosis, treatment, and post-treatment molecular monitoring in real-time. This system thus achieves effective theranostic performance, providing a promising new avenue towards the clinical realisation of theranostics. △ Less

Submitted 8 September, 2020; originally announced September 2020.