Showing 1–9 of 9 results for author: Kaminski, C F

Fast, multicolour optical sectioning over extended fields of view by combining interferometric SIM with machine learning

Authors: Edward N. Ward, Rebecca M. McClelland, Jacob R. Lamb, Roger Rubio-Sánchez, Charles N. Christensen, Bismoy Mazumder, Sofia Kapsiani, Luca Mascheroni, Lorenzo Di Michele, Gabriele S. Kaminski Schierle, Clemens F. Kaminski

Abstract: Structured illumination can reject out-of-focus signal from a sample, enabling high-speed and high-contrast imaging over large areas with widefield detection optics. Currently, this optical-sectioning technique is limited by image reconstruction artefacts and the need for sequential imaging of multiple colour channels. We combine multicolour interferometric pattern generation with machine-learning… ▽ More Structured illumination can reject out-of-focus signal from a sample, enabling high-speed and high-contrast imaging over large areas with widefield detection optics. Currently, this optical-sectioning technique is limited by image reconstruction artefacts and the need for sequential imaging of multiple colour channels. We combine multicolour interferometric pattern generation with machine-learning processing, permitting high-contrast, real-time reconstruction of image data. The method is insensitive to background noise and unevenly phase-stepped illumination patterns. We validate the method in silico and demonstrate its application on diverse specimens, ranging from fixed and live biological cells to synthetic biosystems, imaging at up to 37 Hz across a 44 x 44 $μm^2$ field of view. △ Less

Submitted 31 October, 2023; originally announced November 2023.

Single multimode fiber for in vivo light-field encoded nano-imaging

Authors: Zhong Wen, Zhenyu Dong, Chenlei Pang, Clemens F. Kaminski, Qilin Deng, **ggang Xu, Liqiang Wang, Songguo Liu, Jianbin Tang, Wei Chen, Xu Liu, Qing Yang

Abstract: Super-resolution microscopy normally requiring complex and cumbersome optics is not applicable for in situ imaging through a narrow channel. Here, we demonstrate single hair-thin multimode fiber (MMF) endoscope (less than 250 $μm$) for in vivo light-field nano-imaging, which is called spatial-frequency tracking adaptive beacon light-field encoded nano-endoscopy (STABLE nano-endoscopy) that enables… ▽ More Super-resolution microscopy normally requiring complex and cumbersome optics is not applicable for in situ imaging through a narrow channel. Here, we demonstrate single hair-thin multimode fiber (MMF) endoscope (less than 250 $μm$) for in vivo light-field nano-imaging, which is called spatial-frequency tracking adaptive beacon light-field encoded nano-endoscopy (STABLE nano-endoscopy) that enables three-dimensional (3D) subcellular-scale imaging. Spatial-frequency tracking provides up to $10^3$ Hz disorder tracking that ensures stable imaging in long-haul MMFs (up to 200 m) under various conditions. Full-vector modulation and fluorescence emission difference are combined to enhance the imaging signal-to-noise ratio two times and to improve the resolution to sub-diffraction-limited 250 nm ($λ/3NA$). STABLE nano-endoscopy and white-light endoscopy (WLE) are integrated to achieve cross-scale in vivo imaging inside the lumen. This high-resolution and robust observation in a minimally invasive manner paves the way to gain a deeper understanding of the disease mechanisms and to bridge clinical and biological sciences. △ Less

Submitted 7 July, 2022; originally announced July 2022.

Comments: 14 pages, 5 figures

Journal ref: Published online: 03 July 2023, Nature Photonics

MAI-SIM: interferometric multicolor structured illumination microscopy for everybody

Authors: Edward N. Ward, Lisa Hecker, Charles N. Christensen, Jacob R Lamb, Meng Lu, Luca Mascheroni, Chyi Wei Chung, Anna Wang, Christopher J. Rowlands, Gabriele S. Kaminski Schierle, Clemens F. Kaminski

Abstract: Sub-diffraction resolution, gentle sample illumination, and the possibility to image in multiple colors make Structured Illumination Microscopy (SIM) an imaging technique which is particularly well suited for live cell observations. Here, we present Machine learning Assisted Interferometric-SIM (MAI-SIM), an easy-to-implement method for high speed SIM imaging in multiple colors. The instrument is… ▽ More Sub-diffraction resolution, gentle sample illumination, and the possibility to image in multiple colors make Structured Illumination Microscopy (SIM) an imaging technique which is particularly well suited for live cell observations. Here, we present Machine learning Assisted Interferometric-SIM (MAI-SIM), an easy-to-implement method for high speed SIM imaging in multiple colors. The instrument is based on an interferometer design in which illumination patterns are generated, rotated, and stepped in phase through movement of a single galvanometric mirror element. The design is robust, flexible, and the pattern generation process works for all wavelengths. We complement the unique properties of interferometric SIM with a machine learning toolbox that is simple and efficient to use and is superior to existing methods for the reconstruction of super-resolved images recorded by the instrument. The framework permits real-time SIM reconstructions to be performed in multiple colors, providing the user with instant visualization of the super-resolved images. We demonstrate the capability of MAI-SIM on live biological samples and capture super-resolution images in multiple colors simultaneously over large fields of view. Finally, we embrace a fully open design philosophy to bring the advantages of MAI-SIM to as many users as possible and provide full details on system design and software. △ Less

Submitted 1 June, 2022; originally announced June 2022.

Comments: 31 pages, 7 figures, 9 supporting figures

Spatio-temporal Vision Transformer for Super-resolution Microscopy

Authors: Charles N. Christensen, Meng Lu, Edward N. Ward, Pietro Lio, Clemens F. Kaminski

Abstract: Structured illumination microscopy (SIM) is an optical super-resolution technique that enables live-cell imaging beyond the diffraction limit. Reconstruction of SIM data is prone to artefacts, which becomes problematic when imaging highly dynamic samples because previous methods rely on the assumption that samples are static. We propose a new transformer-based reconstruction method, VSR-SIM, that… ▽ More Structured illumination microscopy (SIM) is an optical super-resolution technique that enables live-cell imaging beyond the diffraction limit. Reconstruction of SIM data is prone to artefacts, which becomes problematic when imaging highly dynamic samples because previous methods rely on the assumption that samples are static. We propose a new transformer-based reconstruction method, VSR-SIM, that uses shifted 3-dimensional window multi-head attention in addition to channel attention mechanism to tackle the problem of video super-resolution (VSR) in SIM. The attention mechanisms are found to capture motion in sequences without the need for common motion estimation techniques such as optical flow. We take an approach to training the network that relies solely on simulated data using videos of natural scenery with a model for SIM image formation. We demonstrate a use case enabled by VSR-SIM referred to as rolling SIM imaging, which increases temporal resolution in SIM by a factor of 9. Our method can be applied to any SIM setup enabling precise recordings of dynamic processes in biomedical research with high temporal resolution. △ Less

Submitted 28 February, 2022; originally announced March 2022.

Comments: 8 pages, 9 figures. Source code: https://github.com/charlesnchr/vsr-sim

Scalable universal tunable virtual-wavevector spatial frequency shift (TVSFS) super-resolution imaging

Authors: Mingwei Tang, Yubing Han, Dehao Ye, Qianwei Zhang, Chenlei Pang, Xiaowei Liu, Weidong Shen, Yaoguang Ma, Clemens F. Kaminski, Xu Liu, Qing Yang

Abstract: Develo** a chip-based super-resolution imaging technique with large field-of-view (FOV), deep subwavelength resolution, and compatibility for both fluorescent and non-fluorescent samples is desired for material science, biomedicine, and life researches, etc. Previous on-chip super-resolution methods focus on either fluorescent or non-fluorescent imaging, putting an urgent requirement on the gene… ▽ More Develo** a chip-based super-resolution imaging technique with large field-of-view (FOV), deep subwavelength resolution, and compatibility for both fluorescent and non-fluorescent samples is desired for material science, biomedicine, and life researches, etc. Previous on-chip super-resolution methods focus on either fluorescent or non-fluorescent imaging, putting an urgent requirement on the general imaging technique compatible with both of them. Here, we introduce a universal super-resolution imaging method based on tunable virtual-wavevector spatial frequency shift (TVSFS), realizing both labeled and label-free super-resolution imaging on a single delicately fabricated scalable photonic chip. Theoretically, with TVSFS, the diffraction limit of a linear optical system can be overcome, and the resolution can be improved more than three times, which is the limitation for most super-resolution imaging based on spatial frequency engineering. Diffractive units were fabricated on the chip's surface to provide a wavevector-variable evanescent wave illumination and induce tunable deep SFS in the samples' Fourier space. A resolution of λ/4.7 for the label-free sample and λ/7.1 for the labeled sample with a large FOV could be achieved with a CMOS-compatible process on a GaP chip. The large FOV, high-compatibility, and high-integration TVSFS chip may advance the fields like cell engineering, precision inspection in the industry, chemical research, etc. △ Less

Submitted 27 December, 2021; v1 submitted 15 March, 2021; originally announced March 2021.

Comments: 13 pages,5 figures

ML-SIM: A deep neural network for reconstruction of structured illumination microscopy images

Authors: Charles N. Christensen, Edward N. Ward, Pietro Lio, Clemens F. Kaminski

Abstract: Structured illumination microscopy (SIM) has become an important technique for optical super-resolution imaging because it allows a doubling of image resolution at speeds compatible for live-cell imaging. However, the reconstruction of SIM images is often slow and prone to artefacts. Here we propose a versatile reconstruction method, ML-SIM, which makes use of machine learning. The model is an end… ▽ More Structured illumination microscopy (SIM) has become an important technique for optical super-resolution imaging because it allows a doubling of image resolution at speeds compatible for live-cell imaging. However, the reconstruction of SIM images is often slow and prone to artefacts. Here we propose a versatile reconstruction method, ML-SIM, which makes use of machine learning. The model is an end-to-end deep residual neural network that is trained on a simulated data set to be free of common SIM artefacts. ML-SIM is thus robust to noise and irregularities in the illumination patterns of the raw SIM input frames. The reconstruction method is widely applicable and does not require the acquisition of experimental training data. Since the training data are generated from simulations of the SIM process on images from generic libraries the method can be efficiently adapted to specific experimental SIM implementations. The reconstruction quality enabled by our method is compared with traditional SIM reconstruction methods, and we demonstrate advantages in terms of noise, reconstruction fidelity and contrast for both simulated and experimental inputs. In addition, reconstruction of one SIM frame typically only takes ~100ms to perform on PCs with modern Nvidia graphics cards, making the technique compatible with real-time imaging. The full implementation and the trained networks are available at http://ML-SIM.com. △ Less

Submitted 24 March, 2020; originally announced March 2020.

Wide-field 3D nanoscopy on chip through large and tunable spatial-frequency-shift effect

Authors: Xiaowei Liu, Chao Meng, Xuechu Xu, Mingwei Tang, Chenlei Pang, Yaoguang Ma, Yaocheng Shi, Qing Yang, Xu Liu, Clemens F. Kaminski

Abstract: Linear super-resolution microscopy via synthesis aperture approach permits fast acquisition because of its wide-field implementations, however, it has been limited in resolution because a missing spatial-frequency band occurs when trying to use a shift magnitude surpassing the cutoff frequency of the detection system beyond a factor of two, which causes ghosting to appear. Here, we propose a metho… ▽ More Linear super-resolution microscopy via synthesis aperture approach permits fast acquisition because of its wide-field implementations, however, it has been limited in resolution because a missing spatial-frequency band occurs when trying to use a shift magnitude surpassing the cutoff frequency of the detection system beyond a factor of two, which causes ghosting to appear. Here, we propose a method of chip-based 3D nanoscopy through large and tunable spatial-frequency-shift effect, capable of covering full extent of the spatial-frequency component within a wide passband. The missing of spatial-frequency can be effectively solved by develo** a spatial-frequency-shift actively tuning approach through wave vector manipulation and operation of optical modes propagating along multiple azimuthal directions on a waveguide chip to interfere. In addition, the method includes a chip-based sectioning capability, which is enabled by saturated absorption of fluorophores. By introducing ultra-large propagation effective refractive index, nanoscale resolution is possible, without sacrificing the temporal resolution and the field-of-view. Imaging on GaP waveguide material demonstrates a lateral resolution of lamda/10, which is 5.4 folds above Abbe diffraction limit, and an axial resolution of lamda/19 using 0.9 NA detection objective. Simulation with an assumed propagation effective refractive index of 10 demonstrates a lateral resolution of lamda/22, in which the huge gap between the directly shifted and the zero-order components is completely filled to ensure the deep-subwavelength resolvability. It means that, a fast wide-field 3D deep-subdiffraction visualization could be realized using a standard microscope by adding a mass-producible and cost-effective spatial-frequency-shift illumination chip. △ Less

Submitted 27 June, 2019; originally announced June 2019.

Comments: 5 figures

A Super-Condenser for Labelfree Nanoscopy

Authors: Florian Ströhl, Ida S. Opstad, Jean-Claude Tinguely, Firehun T. Dullo, Ioanna Mela, Johannes W. M. Osterrieth, Balpreet S. Ahluwalia, Clemens F. Kaminski

Abstract: Labelfree nanoscopy encompasses optical imaging with resolution in the 100 nm range using visible wavelengths. Here, we present a labelfree nanoscopy method that combines Fourier ptychography with waveguide microscopy to realize a 'super-condenser' featuring maximally inclined coherent darkfield illumination with artificially stretched wave vectors due to large refractive indices of the employed S… ▽ More Labelfree nanoscopy encompasses optical imaging with resolution in the 100 nm range using visible wavelengths. Here, we present a labelfree nanoscopy method that combines Fourier ptychography with waveguide microscopy to realize a 'super-condenser' featuring maximally inclined coherent darkfield illumination with artificially stretched wave vectors due to large refractive indices of the employed Si$_3$N$_4$ waveguide material. We produce the required coherent plane wave illumination for Fourier ptychography over imaging areas 400 $\mathrmμ$m$^2$ in size via adiabatically tapered single-mode waveguides and tackle the overlap constraints of the Fourier ptychography phase retrieval algorithm two-fold: firstly, the directionality of the illumination wave vector is changed sequentially via a multiplexed input structure of the waveguide chip layout and secondly, the wave vector modulus is shortend via step-wise increases of the illumination light wavelength over the visible spectrum. We validate the method via in silico and in vitro experiments and provide details on the underlying image formation theory as well as the reconstruction algorithm. △ Less

Submitted 7 May, 2019; originally announced May 2019.

Structured illumination microscopy with extended axial resolution through mirrored illumination

Authors: James D. Manton, Florian Ströhl, Reto Fiolka, Clemens F. Kaminski, Eric J. Rees

Abstract: Wide-field fluorescence microscopy, while much faster than confocal microscopy, suffers from a lack of optical sectioning and poor axial resolution. 3D structured illumination microscopy (SIM) has been demonstrated to provide optical sectioning and to double the achievable resolution both laterally and axially, but even with this the axial resolution is still worse than the lateral resolution of u… ▽ More Wide-field fluorescence microscopy, while much faster than confocal microscopy, suffers from a lack of optical sectioning and poor axial resolution. 3D structured illumination microscopy (SIM) has been demonstrated to provide optical sectioning and to double the achievable resolution both laterally and axially, but even with this the axial resolution is still worse than the lateral resolution of unmodified wide-field detection. Interferometric schemes using two high numerical aperture objectives, such as 4Pi confocal and I5S microscopy, have improved the axial resolution beyond that of the lateral, but at the cost of a significantly more complex optical setup. Here we investigate a simpler dual-objective scheme which we propose can be easily added to an existing 3D-SIM microscope, providing lateral and axial resolutions in excess of 125 nm with conventional fluorophores. △ Less

Submitted 10 October, 2018; originally announced October 2018.