-
Compressive Image Scanning Microscope
Authors:
Ajay Gunalan,
Marco Castello,
Simonluca Piazza,
Shunlei Li,
Alberto Diaspro,
Leonardo S. Mattos,
Paolo Bianchini
Abstract:
We present a novel approach to implement compressive sensing in laser scanning microscopes (LSM), specifically in image scanning microscopy (ISM), using a single-photon avalanche diode (SPAD) array detector. Our method addresses two significant limitations in applying compressive sensing to LSM: the time to compute the sampling matrix and the quality of reconstructed images. We employ a fixed samp…
▽ More
We present a novel approach to implement compressive sensing in laser scanning microscopes (LSM), specifically in image scanning microscopy (ISM), using a single-photon avalanche diode (SPAD) array detector. Our method addresses two significant limitations in applying compressive sensing to LSM: the time to compute the sampling matrix and the quality of reconstructed images. We employ a fixed sampling strategy, skip** alternate rows and columns during data acquisition, which reduces the number of points scanned by a factor of four and eliminates the need to compute different sampling matrices. By exploiting the parallel images generated by the SPAD array, we improve the quality of the reconstructed compressive-ISM images compared to standard compressive confocal LSM images. Our results demonstrate the effectiveness of our approach in producing higher-quality images with reduced data acquisition time and potential benefits in reducing photobleaching.
△ Less
Submitted 19 July, 2023;
originally announced July 2023.
-
Learning-Based Keypoint Registration for Fetoscopic Mosaicking
Authors:
Alessandro Casella,
Sophia Bano,
Francisco Vasconcelos,
Anna L. David,
Dario Paladini,
Jan Deprest,
Elena De Momi,
Leonardo S. Mattos,
Sara Moccia,
Danail Stoyanov
Abstract:
In Twin-to-Twin Transfusion Syndrome (TTTS), abnormal vascular anastomoses in the monochorionic placenta can produce uneven blood flow between the two fetuses. In the current practice, TTTS is treated surgically by closing abnormal anastomoses using laser ablation. This surgery is minimally invasive and relies on fetoscopy. Limited field of view makes anastomosis identification a challenging task…
▽ More
In Twin-to-Twin Transfusion Syndrome (TTTS), abnormal vascular anastomoses in the monochorionic placenta can produce uneven blood flow between the two fetuses. In the current practice, TTTS is treated surgically by closing abnormal anastomoses using laser ablation. This surgery is minimally invasive and relies on fetoscopy. Limited field of view makes anastomosis identification a challenging task for the surgeon. To tackle this challenge, we propose a learning-based framework for in-vivo fetoscopy frame registration for field-of-view expansion. The novelties of this framework relies on a learning-based keypoint proposal network and an encoding strategy to filter (i) irrelevant keypoints based on fetoscopic image segmentation and (ii) inconsistent homographies. We validate of our framework on a dataset of 6 intraoperative sequences from 6 TTTS surgeries from 6 different women against the most recent state of the art algorithm, which relies on the segmentation of placenta vessels. The proposed framework achieves higher performance compared to the state of the art, paving the way for robust mosaicking to provide surgeons with context awareness during TTTS surgery.
△ Less
Submitted 26 July, 2022;
originally announced July 2022.
-
Placental Vessel Segmentation and Registration in Fetoscopy: Literature Review and MICCAI FetReg2021 Challenge Findings
Authors:
Sophia Bano,
Alessandro Casella,
Francisco Vasconcelos,
Abdul Qayyum,
Abdesslam Benzinou,
Moona Mazher,
Fabrice Meriaudeau,
Chiara Lena,
Ilaria Anita Cintorrino,
Gaia Romana De Paolis,
Jessica Biagioli,
Daria Grechishnikova,
**g Jiao,
Bizhe Bai,
Yanyan Qiao,
Binod Bhattarai,
Rebati Raman Gaire,
Ronast Subedi,
Eduard Vazquez,
Szymon Płotka,
Aneta Lisowska,
Arkadiusz Sitek,
George Attilakos,
Ruwan Wimalasundera,
Anna L David
, et al. (6 additional authors not shown)
Abstract:
Fetoscopy laser photocoagulation is a widely adopted procedure for treating Twin-to-Twin Transfusion Syndrome (TTTS). The procedure involves photocoagulation pathological anastomoses to regulate blood exchange among twins. The procedure is particularly challenging due to the limited field of view, poor manoeuvrability of the fetoscope, poor visibility, and variability in illumination. These challe…
▽ More
Fetoscopy laser photocoagulation is a widely adopted procedure for treating Twin-to-Twin Transfusion Syndrome (TTTS). The procedure involves photocoagulation pathological anastomoses to regulate blood exchange among twins. The procedure is particularly challenging due to the limited field of view, poor manoeuvrability of the fetoscope, poor visibility, and variability in illumination. These challenges may lead to increased surgery time and incomplete ablation. Computer-assisted intervention (CAI) can provide surgeons with decision support and context awareness by identifying key structures in the scene and expanding the fetoscopic field of view through video mosaicking. Research in this domain has been hampered by the lack of high-quality data to design, develop and test CAI algorithms. Through the Fetoscopic Placental Vessel Segmentation and Registration (FetReg2021) challenge, which was organized as part of the MICCAI2021 Endoscopic Vision challenge, we released the first largescale multicentre TTTS dataset for the development of generalized and robust semantic segmentation and video mosaicking algorithms. For this challenge, we released a dataset of 2060 images, pixel-annotated for vessels, tool, fetus and background classes, from 18 in-vivo TTTS fetoscopy procedures and 18 short video clips. Seven teams participated in this challenge and their model performance was assessed on an unseen test dataset of 658 pixel-annotated images from 6 fetoscopic procedures and 6 short clips. The challenge provided an opportunity for creating generalized solutions for fetoscopic scene understanding and mosaicking. In this paper, we present the findings of the FetReg2021 challenge alongside reporting a detailed literature review for CAI in TTTS fetoscopy. Through this challenge, its analysis and the release of multi-centre fetoscopic data, we provide a benchmark for future research in this field.
△ Less
Submitted 26 February, 2023; v1 submitted 24 June, 2022;
originally announced June 2022.
-
FetReg: Placental Vessel Segmentation and Registration in Fetoscopy Challenge Dataset
Authors:
Sophia Bano,
Alessandro Casella,
Francisco Vasconcelos,
Sara Moccia,
George Attilakos,
Ruwan Wimalasundera,
Anna L. David,
Dario Paladini,
Jan Deprest,
Elena De Momi,
Leonardo S. Mattos,
Danail Stoyanov
Abstract:
Fetoscopy laser photocoagulation is a widely used procedure for the treatment of Twin-to-Twin Transfusion Syndrome (TTTS), that occur in mono-chorionic multiple pregnancies due to placental vascular anastomoses. This procedure is particularly challenging due to limited field of view, poor manoeuvrability of the fetoscope, poor visibility due to fluid turbidity, variability in light source, and unu…
▽ More
Fetoscopy laser photocoagulation is a widely used procedure for the treatment of Twin-to-Twin Transfusion Syndrome (TTTS), that occur in mono-chorionic multiple pregnancies due to placental vascular anastomoses. This procedure is particularly challenging due to limited field of view, poor manoeuvrability of the fetoscope, poor visibility due to fluid turbidity, variability in light source, and unusual position of the placenta. This may lead to increased procedural time and incomplete ablation, resulting in persistent TTTS. Computer-assisted intervention may help overcome these challenges by expanding the fetoscopic field of view through video mosaicking and providing better visualization of the vessel network. However, the research and development in this domain remain limited due to unavailability of high-quality data to encode the intra- and inter-procedure variability. Through the \textit{Fetoscopic Placental Vessel Segmentation and Registration (FetReg)} challenge, we present a large-scale multi-centre dataset for the development of generalized and robust semantic segmentation and video mosaicking algorithms for the fetal environment with a focus on creating drift-free mosaics from long duration fetoscopy videos. In this paper, we provide an overview of the FetReg dataset, challenge tasks, evaluation metrics and baseline methods for both segmentation and registration. Baseline methods results on the FetReg dataset shows that our dataset poses interesting challenges, offering large opportunity for the creation of novel methods and models through a community effort initiative guided by the FetReg challenge.
△ Less
Submitted 16 June, 2021; v1 submitted 10 June, 2021;
originally announced June 2021.
-
Closed-Loop Control of a Magnetically Actuated Fiber-Coupled Laser for Computer-Assisted Laser Microsurgery
Authors:
Hamed Mohammadbagherpoor,
Alperen Acemoglu,
Leonardo S. Mattos,
Darwin Caldwell,
James E. Johnson,
John Muth,
Edward Grant
Abstract:
Patient outcomes in the medical field are improving through the use and incorporation of robotics technology and laser physics, e.g., the use of optical fibers and lasers in micro-surgery. This paper describes the design and implementation of a new optical fiber laser micro-surgery system, one that is the first to use closed-loop feedback control. In this computerassisted laser scanning tool the l…
▽ More
Patient outcomes in the medical field are improving through the use and incorporation of robotics technology and laser physics, e.g., the use of optical fibers and lasers in micro-surgery. This paper describes the design and implementation of a new optical fiber laser micro-surgery system, one that is the first to use closed-loop feedback control. In this computerassisted laser scanning tool the laser beam is controlled by four magnetic actuators. After attaching permanent magnets to the free end of the optical fiber, it is the control of these four magnets that produces an accurate laser scanning system, one suited to micro-surgery applications. The interaction between the electromagnetic fields generated by the external magnetic coils and the flux of the internal permanent magnets, produces the control torques required to produce the desired movement of the optic fiber. The tracking error of the optic fiber is compensated for by using a photo-detector sensor as the feedback transducer in the control system. The magnetic torque bends the optical fiber and the feedback from the photodetector gives automated control and high-speed laser scanning of the fiber tip. The simulation and the experimental results are accurate and are co-related.
△ Less
Submitted 17 January, 2020; v1 submitted 10 November, 2019;
originally announced November 2019.
-
Quantum Engineered Kondo Lattices
Authors:
Jeremy Figgins,
Laila S. Mattos,
Warren Mar,
Yi-Ting Chen,
Hari C. Manoharan,
Dirk K. Morr
Abstract:
Recent advances in atomic manipulation techniques have provided a novel bottom-up approach to investigating the unconventional properties and complex phases of strongly correlated electron materials. By engineering artificial condensed matter systems containing tens to thousands of atoms with tailored electronic or magnetic properties, it has become possible to explore how quantum many-body effect…
▽ More
Recent advances in atomic manipulation techniques have provided a novel bottom-up approach to investigating the unconventional properties and complex phases of strongly correlated electron materials. By engineering artificial condensed matter systems containing tens to thousands of atoms with tailored electronic or magnetic properties, it has become possible to explore how quantum many-body effects---whose existence lies at the heart of strongly correlated materials---emerge as the size of a system is increased from the nanoscale to the mesoscale. Here we investigate both theoretically and experimentally the quantum engineering of nanoscopic Kondo lattices -- Kondo droplets -- that exemplify nanoscopic replicas of heavy-fermion materials. We demonstrate that by changing a droplet's real space geometry, it is possible to not only create coherently coupled Kondo droplets whose properties asymptotically approach those of a quantum-coherent Kondo lattice, but also to markedly increase or decrease the droplet's Kondo temperature. Furthermore we report on the discovery of a new quantum phenomenon -- the Kondo echo -- a signature of droplets containing Kondo holes functioning as direct probes of spatially extended, quantum-coherent Kondo cloud correlations.
△ Less
Submitted 12 February, 2019;
originally announced February 2019.
-
Towards a Magnetically Actuated Laser Scanner for Endoscopic Microsurgeries
Authors:
Alperen Acemoglu,
Nikhil Deshpande,
Leonardo S. Mattos
Abstract:
This article presents the design and assembly of a novel magnetically actuated endoscopic laser scanner device. The device is designed to perform 2D position control and high speed scanning of a fiber-based laser for operation in narrow workspaces. The device includes laser focusing optics to allow non-contact incisions and tablet-based control interface for intuitive teleoperation. The performanc…
▽ More
This article presents the design and assembly of a novel magnetically actuated endoscopic laser scanner device. The device is designed to perform 2D position control and high speed scanning of a fiber-based laser for operation in narrow workspaces. The device includes laser focusing optics to allow non-contact incisions and tablet-based control interface for intuitive teleoperation. The performance of the proof-of-concept device is analysed through controllability and the usability studies. The computer-controlled high-speed scanning demonstrates repeatable results with 21 um precision and a stable response up to 48 Hz. Teleoperation user trials, were performed for trajectory-following tasks with 12 subjects, show an accuracy of 39 um. The innovative design of the device can be applied to both surgical and diagnostic (imaging) applications in endoscopic systems.
△ Less
Submitted 21 November, 2017;
originally announced November 2017.
-
Uncertainty-Aware Organ Classification for Surgical Data Science Applications in Laparoscopy
Authors:
S. Moccia,
S. J. Wirkert,
H. Kenngott,
A. S. Vemuri,
M. Apitz,
B. Mayer,
E. De Momi,
L. S. Mattos,
L. Maier-Hein
Abstract:
Objective: Surgical data science is evolving into a research field that aims to observe everything occurring within and around the treatment process to provide situation-aware data-driven assistance. In the context of endoscopic video analysis, the accurate classification of organs in the field of view of the camera proffers a technical challenge. Herein, we propose a new approach to anatomical st…
▽ More
Objective: Surgical data science is evolving into a research field that aims to observe everything occurring within and around the treatment process to provide situation-aware data-driven assistance. In the context of endoscopic video analysis, the accurate classification of organs in the field of view of the camera proffers a technical challenge. Herein, we propose a new approach to anatomical structure classification and image tagging that features an intrinsic measure of confidence to estimate its own performance with high reliability and which can be applied to both RGB and multispectral imaging (MI) data. Methods: Organ recognition is performed using a superpixel classification strategy based on textural and reflectance information. Classification confidence is estimated by analyzing the dispersion of class probabilities. Assessment of the proposed technology is performed through a comprehensive in vivo study with seven pigs. Results: When applied to image tagging, mean accuracy in our experiments increased from 65% (RGB) and 80% (MI) to 90% (RGB) and 96% (MI) with the confidence measure. Conclusion: Results showed that the confidence measure had a significant influence on the classification accuracy, and MI data are better suited for anatomical structure labeling than RGB data. Significance: This work significantly enhances the state of art in automatic labeling of endoscopic videos by introducing the use of the confidence metric, and by being the first study to use MI data for in vivo laparoscopic tissue classification. The data of our experiments will be released as the first in vivo MI dataset upon publication of this paper.
△ Less
Submitted 19 October, 2018; v1 submitted 21 June, 2017;
originally announced June 2017.
-
Quantum Holographic Encoding in a Two-dimensional Electron Gas
Authors:
Christopher R. Moon,
Laila S. Mattos,
Brian K. Foster,
Gabriel Zeltzer,
Hari C. Manoharan
Abstract:
The advent of bottom-up atomic manipulation heralded a new horizon for attainable information density, as it allowed a bit of information to be represented by a single atom. The discrete spacing between atoms in condensed matter has thus set a rigid limit on the maximum possible information density. While modern technologies are still far from this scale, all theoretical downscaling of devices t…
▽ More
The advent of bottom-up atomic manipulation heralded a new horizon for attainable information density, as it allowed a bit of information to be represented by a single atom. The discrete spacing between atoms in condensed matter has thus set a rigid limit on the maximum possible information density. While modern technologies are still far from this scale, all theoretical downscaling of devices terminates at this spatial limit. Here, however, we break this barrier with electronic quantum encoding scaled to subatomic densities. We use atomic manipulation to first construct open nanostructures--"molecular holograms"--which in turn concentrate information into a medium free of lattice constraints: the quantum states of a two-dimensional degenerate Fermi gas of electrons. The information embedded in the holograms is transcoded at even smaller length scales into an atomically uniform area of a copper surface, where it is densely projected into both two spatial degrees of freedom and a third holographic dimension mapped to energy. In analogy to optical volume holography, this requires precise amplitude and phase engineering of electron wavefunctions to assemble pages of information volumetrically. This data is read out by map** the energy-resolved electron density of states with a scanning tunnelling microscope. As the projection and readout are both extremely near-field, and because we use native quantum states rather than an external beam, we are not limited by lensing or collimation and can create electronically projected objects with features as small as ~0.3 nm. These techniques reach unprecedented densities exceeding 20 bits/nm2 and place tens of bits into a single fermionic state.
△ Less
Submitted 22 April, 2009;
originally announced April 2009.
-
Quantum Phase Extraction in Isospectral Electronic Nanostructures
Authors:
Christopher R. Moon,
Laila S. Mattos,
Brian K. Foster,
Gabriel Zeltzer,
Wonhee Ko,
Hari C. Manoharan
Abstract:
Quantum phase is not a direct observable and is usually determined by interferometric methods. We present a method to map complete electron wave functions, including internal quantum phase information, from measured single-state probability densities. We harness the mathematical discovery of drum-like manifolds bearing different shapes but identical resonances, and construct quantum isospectral…
▽ More
Quantum phase is not a direct observable and is usually determined by interferometric methods. We present a method to map complete electron wave functions, including internal quantum phase information, from measured single-state probability densities. We harness the mathematical discovery of drum-like manifolds bearing different shapes but identical resonances, and construct quantum isospectral nanostructures possessing matching electronic structure but divergent physical structure. Quantum measurement (scanning tunneling microscopy) of these "quantum drums" [degenerate two-dimensional electron states on the Cu(111) surface confined by individually positioned CO molecules] reveals that isospectrality provides an extra topological degree of freedom enabling robust quantum state transplantation and phase extraction.
△ Less
Submitted 15 March, 2008;
originally announced March 2008.
