-
Computational 3D topographic microscopy from terabytes of data per sample
Authors:
Kevin C. Zhou,
Mark Harfouche,
Maxwell Zheng,
Joakim Jönsson,
Kyung Chul Lee,
Ron Appel,
Paul Reamey,
Thomas Doman,
Veton Saliu,
Gregor Horstmeyer,
Roarke Horstmeyer
Abstract:
We present a large-scale computational 3D topographic microscope that enables 6-gigapixel profilometric 3D imaging at micron-scale resolution across $>$110 cm$^2$ areas over multi-millimeter axial ranges. Our computational microscope, termed STARCAM (Scanning Topographic All-in-focus Reconstruction with a Computational Array Microscope), features a parallelized, 54-camera architecture with 3-axis…
▽ More
We present a large-scale computational 3D topographic microscope that enables 6-gigapixel profilometric 3D imaging at micron-scale resolution across $>$110 cm$^2$ areas over multi-millimeter axial ranges. Our computational microscope, termed STARCAM (Scanning Topographic All-in-focus Reconstruction with a Computational Array Microscope), features a parallelized, 54-camera architecture with 3-axis translation to capture, for each sample of interest, a multi-dimensional, 2.1-terabyte (TB) dataset, consisting of a total of 224,640 9.4-megapixel images. We developed a self-supervised neural network-based algorithm for 3D reconstruction and stitching that jointly estimates an all-in-focus photometric composite and 3D height map across the entire field of view, using multi-view stereo information and image sharpness as a focal metric. The memory-efficient, compressed differentiable representation offered by the neural network effectively enables joint participation of the entire multi-TB dataset during the reconstruction process. To demonstrate the broad utility of our new computational microscope, we applied STARCAM to a variety of decimeter-scale objects, with applications ranging from cultural heritage to industrial inspection.
△ Less
Submitted 5 June, 2023;
originally announced June 2023.
-
Imaging across multiple spatial scales with the multi-camera array microscope
Authors:
Mark Harfouche,
Kanghyun Kim,
Kevin C. Zhou,
Pavan Chandra Konda,
Sunanda Sharma,
Eric E. Thomson,
Colin Cooke,
Shiqi Xu,
Lucas Kreiss,
Amey Chaware,
Xi Yang,
Xing Yao,
Vinayak Pathak,
Martin Bohlen,
Ron Appel,
Aurélien Bègue,
Clare Cook,
Jed Doman,
John Efromson,
Gregor Horstmeyer,
Jaehee Park,
Paul Reamey,
Veton Saliu,
Eva Naumann,
Roarke Horstmeyer
Abstract:
This article experimentally examines different configurations of a novel multi-camera array microscope (MCAM) imaging technology. The MCAM is based upon a densely packed array of "micro-cameras" to jointly image across a large field-of-view at high resolution. Each micro-camera within the array images a unique area of a sample of interest, and then all acquired data with 54 micro-cameras are digit…
▽ More
This article experimentally examines different configurations of a novel multi-camera array microscope (MCAM) imaging technology. The MCAM is based upon a densely packed array of "micro-cameras" to jointly image across a large field-of-view at high resolution. Each micro-camera within the array images a unique area of a sample of interest, and then all acquired data with 54 micro-cameras are digitally combined into composite frames, whose total pixel counts significantly exceed the pixel counts of standard microscope systems. We present results from three unique MCAM configurations for different use cases. First, we demonstrate a configuration that simultaneously images and estimates the 3D object depth across a 100 x 135 mm^2 field-of-view (FOV) at approximately 20 um resolution, which results in 0.15 gigapixels (GP) per snapshot. Second, we demonstrate an MCAM configuration that records video across a continuous 83 x 123 mm^2 FOV with two-fold increased resolution (0.48 GP per frame). Finally, we report a third high-resolution configuration (2 um resolution) that can rapidly produce 9.8 GP composites of large histopathology specimens.
△ Less
Submitted 28 February, 2023; v1 submitted 30 November, 2022;
originally announced December 2022.
-
Multi-element microscope optimization by a learned sensing network with composite physical layers
Authors:
Kanghyun Kim,
Pavan Chandra Konda,
Colin L. Cooke,
Ron Appel,
Roarke Horstmeyer
Abstract:
Standard microscopes offer a variety of settings to help improve the visibility of different specimens to the end microscope user. Increasingly, however, digital microscopes are used to capture images for automated interpretation by computer algorithms (e.g., for feature classification, detection or segmentation), often without any human involvement. In this work, we investigate an approach to joi…
▽ More
Standard microscopes offer a variety of settings to help improve the visibility of different specimens to the end microscope user. Increasingly, however, digital microscopes are used to capture images for automated interpretation by computer algorithms (e.g., for feature classification, detection or segmentation), often without any human involvement. In this work, we investigate an approach to jointly optimize multiple microscope settings, together with a classification network, for improved performance with such automated tasks. We explore the interplay between optimization of programmable illumination and pupil transmission, using experimentally imaged blood smears for automated malaria parasite detection, to show that multi-element "learned sensing" outperforms its single-element counterpart. While not necessarily ideal for human interpretation, the network's resulting low-resolution microscope images (20X-comparable) offer a machine learning network sufficient contrast to match the classification performance of corresponding high-resolution imagery (100X-comparable), pointing a path towards accurate automation over large fields-of-view.
△ Less
Submitted 27 June, 2020;
originally announced June 2020.
