-
Autonomous Hyperspectral Characterisation Station: Robotically Assisted Characterisation of Polymer Degradation
Authors:
Shayan Azizi,
Ehsan Asadi,
Shaun Howard,
Benjamin W. Muir,
Riley O'Shea,
Alireza Bab-Hadiashar
Abstract:
This paper addresses the gap between the capabilities and utilisation of robotics and automation in laboratory settings and builds upon the concept of Self Driving Labs (SDL). %to significantly impact laboratory operations. We introduce an innovative approach to the temporal characterisation of materials. The article discusses the challenges posed by manual methods involving established laboratory…
▽ More
This paper addresses the gap between the capabilities and utilisation of robotics and automation in laboratory settings and builds upon the concept of Self Driving Labs (SDL). %to significantly impact laboratory operations. We introduce an innovative approach to the temporal characterisation of materials. The article discusses the challenges posed by manual methods involving established laboratory equipment and presents an automated hyperspectral characterisation station. This station integrates robot-aided hyperspectral imaging, complex material characterisation modeling, and automated data analysis, offering a non-destructive and comprehensive approach. This work explains how the proposed assembly can automatically measure the half-life of biodegradable polymers with higher throughput and accuracy than manual methods. The investigation explores the effect of pH, number of average molecular weight (Mn), end groups, and blends on the degradation rate of polylactic acid (PLA). The contributions of the paper lie in introducing an adaptable classification station for novel characterisation methods and presenting an innovative methodology for polymer degradation rate measurement. The proposed system has the potential to accelerate the development of high-throughput screening and characterisation methods in material and chemistry laboratories.
△ Less
Submitted 18 February, 2024;
originally announced February 2024.
-
CoordGate: Efficiently Computing Spatially-Varying Convolutions in Convolutional Neural Networks
Authors:
Sunny Howard,
Peter Norreys,
Andreas Döpp
Abstract:
Optical imaging systems are inherently limited in their resolution due to the point spread function (PSF), which applies a static, yet spatially-varying, convolution to the image. This degradation can be addressed via Convolutional Neural Networks (CNNs), particularly through deblurring techniques. However, current solutions face certain limitations in efficiently computing spatially-varying convo…
▽ More
Optical imaging systems are inherently limited in their resolution due to the point spread function (PSF), which applies a static, yet spatially-varying, convolution to the image. This degradation can be addressed via Convolutional Neural Networks (CNNs), particularly through deblurring techniques. However, current solutions face certain limitations in efficiently computing spatially-varying convolutions. In this paper we propose CoordGate, a novel lightweight module that uses a multiplicative gate and a coordinate encoding network to enable efficient computation of spatially-varying convolutions in CNNs. CoordGate allows for selective amplification or attenuation of filters based on their spatial position, effectively acting like a locally connected neural network. The effectiveness of the CoordGate solution is demonstrated within the context of U-Nets and applied to the challenging problem of image deblurring. The experimental results show that CoordGate outperforms conventional approaches, offering a more robust and spatially aware solution for CNNs in various computer vision applications.
△ Less
Submitted 9 January, 2024;
originally announced January 2024.
-
Deconvolution in Fluorescence Lifetime imaging microscopy (FLIM)
Authors:
Varun Mannam,
Xiaotong Yuan,
Scott Howard
Abstract:
Fluorescence lifetime imaging microscopy (FLIM) is an important technique to understand the chemical micro-environment in cells and tissues since it provides additional contrast compared to conventional fluorescence imaging. When two fluorophores within a diffraction limit are excited, the resulting emission leads to non-linear spatial distortion and localization effects in intensity (magnitude) a…
▽ More
Fluorescence lifetime imaging microscopy (FLIM) is an important technique to understand the chemical micro-environment in cells and tissues since it provides additional contrast compared to conventional fluorescence imaging. When two fluorophores within a diffraction limit are excited, the resulting emission leads to non-linear spatial distortion and localization effects in intensity (magnitude) and lifetime (phase) components. To address this issue, in this work, we provide a theoretical model for convolution in FLIM to describe how the resulting behavior differs from conventional fluorescence microscopy. We then present a Richardson-Lucy (RL) based deconvolution including total variation (TV) regularization method to correct for the distortions in FLIM measurements due to optical convolution, and experimentally demonstrate this FLIM deconvolution method on a multi-photon microscopy (MPM)-FLIM images of fluorescent-labeled fixed bovine pulmonary arterial endothelial (BPAE) cells.
△ Less
Submitted 16 January, 2022;
originally announced January 2022.
-
Low dosage 3D volume fluorescence microscopy imaging using compressive sensing
Authors:
Varun Mannam,
Jacob Brandt,
Cody J. Smith,
Scott Howard
Abstract:
Fluorescence microscopy has been a significant tool to observe long-term imaging of embryos (in vivo) growth over time. However, cumulative exposure is phototoxic to such sensitive live samples. While techniques like light-sheet fluorescence microscopy (LSFM) allow for reduced exposure, it is not well suited for deep imaging models. Other computational techniques are computationally expensive and…
▽ More
Fluorescence microscopy has been a significant tool to observe long-term imaging of embryos (in vivo) growth over time. However, cumulative exposure is phototoxic to such sensitive live samples. While techniques like light-sheet fluorescence microscopy (LSFM) allow for reduced exposure, it is not well suited for deep imaging models. Other computational techniques are computationally expensive and often lack restoration quality. To address this challenge, one can use various low-dosage imaging techniques that are developed to achieve the 3D volume reconstruction using a few slices in the axial direction (z-axis); however, they often lack restoration quality. Also, acquiring dense images (with small steps) in the axial direction is computationally expensive. To address this challenge, we present a compressive sensing (CS) based approach to fully reconstruct 3D volumes with the same signal-to-noise ratio (SNR) with less than half of the excitation dosage. We present the theory and experimentally validate the approach. To demonstrate our technique, we capture a 3D volume of the RFP labeled neurons in the zebrafish embryo spinal cord (30um thickness) with the axial sampling of 0.1um using a confocal microscope. From the results, we observe the CS-based approach achieves accurate 3D volume reconstruction from less than 20% of the entire stack optical sections. The developed CS-based methodology in this work can be easily applied to other deep imaging modalities such as two-photon and light-sheet microscopy, where reducing sample photo-toxicity is a critical challenge.
△ Less
Submitted 3 January, 2022;
originally announced January 2022.
-
Convolutional Neural Network Denoising in Fluorescence Lifetime Imaging Microscopy (FLIM)
Authors:
Varun Mannam,
Yide Zhang,
Xiaotong Yuan,
Takashi Hato,
Pierre C. Dagher,
Evan L. Nichols,
Cody J. Smith,
Kenneth W. Dunn,
Scott Howard
Abstract:
Fluorescence lifetime imaging microscopy (FLIM) systems are limited by their slow processing speed, low signal-to-noise ratio (SNR), and expensive and challenging hardware setups. In this work, we demonstrate applying a denoising convolutional network to improve FLIM SNR. The network will be integrated with an instant FLIM system with fast data acquisition based on analog signal processing, high S…
▽ More
Fluorescence lifetime imaging microscopy (FLIM) systems are limited by their slow processing speed, low signal-to-noise ratio (SNR), and expensive and challenging hardware setups. In this work, we demonstrate applying a denoising convolutional network to improve FLIM SNR. The network will be integrated with an instant FLIM system with fast data acquisition based on analog signal processing, high SNR using high-efficiency pulse-modulation, and cost-effective implementation utilizing off-the-shelf radio-frequency components. Our instant FLIM system simultaneously provides the intensity, lifetime, and phasor plots \textit{in vivo} and \textit{ex vivo}. By integrating image denoising using the trained deep learning model on the FLIM data, provide accurate FLIM phasor measurements are obtained. The enhanced phasor is then passed through the K-means clustering segmentation method, an unbiased and unsupervised machine learning technique to separate different fluorophores accurately. Our experimental \textit{in vivo} mouse kidney results indicate that introducing the deep learning image denoising model before the segmentation effectively removes the noise in the phasor compared to existing methods and provides clearer segments. Hence, the proposed deep learning-based workflow provides fast and accurate automatic segmentation of fluorescence images using instant FLIM. The denoising operation is effective for the segmentation if the FLIM measurements are noisy. The clustering can effectively enhance the detection of biological structures of interest in biomedical imaging applications.
△ Less
Submitted 6 March, 2021;
originally announced March 2021.
-
Deep learning-based super-resolution fluorescence microscopy on small datasets
Authors:
Varun Mannam,
Yide Zhang,
Xiaotong Yuan,
Scott Howard
Abstract:
Fluorescence microscopy has enabled a dramatic development in modern biology by visualizing biological organisms with micrometer scale resolution. However, due to the diffraction limit, sub-micron/nanometer features are difficult to resolve. While various super-resolution techniques are developed to achieve nanometer-scale resolution, they often either require expensive optical setup or specialize…
▽ More
Fluorescence microscopy has enabled a dramatic development in modern biology by visualizing biological organisms with micrometer scale resolution. However, due to the diffraction limit, sub-micron/nanometer features are difficult to resolve. While various super-resolution techniques are developed to achieve nanometer-scale resolution, they often either require expensive optical setup or specialized fluorophores. In recent years, deep learning has shown the potentials to reduce the technical barrier and obtain super-resolution from diffraction-limited images. For accurate results, conventional deep learning techniques require thousands of images as a training dataset. Obtaining large datasets from biological samples is not often feasible due to the photobleaching of fluorophores, phototoxicity, and dynamic processes occurring within the organism. Therefore, achieving deep learning-based super-resolution using small datasets is challenging. We address this limitation with a new convolutional neural network-based approach that is successfully trained with small datasets and achieves super-resolution images. We captured 750 images in total from 15 different field-of-views as the training dataset to demonstrate the technique. In each FOV, a single target image is generated using the super-resolution radial fluctuation method. As expected, this small dataset failed to produce a usable model using traditional super-resolution architecture. However, using the new approach, a network can be trained to achieve super-resolution images from this small dataset. This deep learning model can be applied to other biomedical imaging modalities such as MRI and X-ray imaging, where obtaining large training datasets is challenging.
△ Less
Submitted 6 March, 2021;
originally announced March 2021.
-
Packet Compressed Sensing Imaging (PCSI): Robust Image Transmission over Noisy Channels
Authors:
Scott Howard,
Grant Barthelmes,
Cara Ravasio,
Lisa Huang,
Benjamin Poag,
Varun Mannam
Abstract:
Packet Compressed Sensing Imaging (PCSI) is digital unconnected image transmission method resilient to packet loss. The goal is to develop a robust image transmission method that is computationally trivial to transmit (e.g., compatible with low-power 8-bit microcontrollers) and well suited for weak signal environments where packets are likely to be lost. In other image transmission techniques, noi…
▽ More
Packet Compressed Sensing Imaging (PCSI) is digital unconnected image transmission method resilient to packet loss. The goal is to develop a robust image transmission method that is computationally trivial to transmit (e.g., compatible with low-power 8-bit microcontrollers) and well suited for weak signal environments where packets are likely to be lost. In other image transmission techniques, noise and packet loss leads to parts of the image being distorted or missing. In PCSI, every packet contains random pixel information from the entire image, and each additional packet received (in any order) simply enhances image quality. Satisfactory SSTV resolution (320x240 pixel) images can be received in ~1-2 minutes when transmitted at 1200 baud AFSK, which is on par with analog SSTV transmission time. Image transmission and reception can occur simultaneously on a computer, and multiple images can be received from multiple stations simultaneously - allowing for the creation of "image nets." This paper presents a simple computer application for Windows, Mac, and Linux that implements PCSI transmission and reception on any KISS compatible hardware or software modem on any band and digital mode.
△ Less
Submitted 23 September, 2020;
originally announced September 2020.
-
Machine learning for faster and smarter fluorescence lifetime imaging microscopy
Authors:
Varun Mannam,
Yide Zhang,
Xiaotong Yuan,
Cara Ravasio,
Scott S. Howard
Abstract:
Fluorescence lifetime imaging microscopy (FLIM) is a powerful technique in biomedical research that uses the fluorophore decay rate to provide additional contrast in fluorescence microscopy. However, at present, the calculation, analysis, and interpretation of FLIM is a complex, slow, and computationally expensive process. Machine learning (ML) techniques are well suited to extract and interpret m…
▽ More
Fluorescence lifetime imaging microscopy (FLIM) is a powerful technique in biomedical research that uses the fluorophore decay rate to provide additional contrast in fluorescence microscopy. However, at present, the calculation, analysis, and interpretation of FLIM is a complex, slow, and computationally expensive process. Machine learning (ML) techniques are well suited to extract and interpret measurements from multi-dimensional FLIM data sets with substantial improvement in speed over conventional methods. In this topical review, we first discuss the basics of FILM and ML. Second, we provide a summary of lifetime extraction strategies using ML and its applications in classifying and segmenting FILM images with higher accuracy compared to conventional methods. Finally, we discuss two potential directions to improve FLIM with ML with proof of concept demonstrations.
△ Less
Submitted 5 August, 2020;
originally announced August 2020.
-
Instant Image Denoising Plugin for ImageJ using Convolutional Neural Networks
Authors:
Varun Mannam,
Yide Zhang,
Yinhao Zhu,
Scott Howard
Abstract:
We present a new convolutional neural network (CNN) based ImageJ plugin for fluorescence microscopy image denoising with an average improvement of 7.5 dB in peak signal-to-noise ratio (PSNR) and denoising instantly within 80 msec.
We present a new convolutional neural network (CNN) based ImageJ plugin for fluorescence microscopy image denoising with an average improvement of 7.5 dB in peak signal-to-noise ratio (PSNR) and denoising instantly within 80 msec.
△ Less
Submitted 23 June, 2020;
originally announced June 2020.
-
Bayesian Recursive Estimation on the Rotation Group
Authors:
Sofia Suvorova,
Stephen D. Howard,
Bill Moran
Abstract:
Tracking on the rotation group is a key component of many modern systems for estimation of the motion of rigid bodies. To address this problem, here we describe a Bayesian algorithm that relies on directional measurements for tracking on the special orthogonal (rotation) group. Its novelty lies in the use of maximum entropy distributions on these groups as models for the priors, and justifiable ap…
▽ More
Tracking on the rotation group is a key component of many modern systems for estimation of the motion of rigid bodies. To address this problem, here we describe a Bayesian algorithm that relies on directional measurements for tracking on the special orthogonal (rotation) group. Its novelty lies in the use of maximum entropy distributions on these groups as models for the priors, and justifiable approximation algorithms that permit recursive implementation of such a model. We provide the solutions in a recursive closed form. In the two-dimensional case the parameters of the prior and posterior distributions can be computed exactly and the solution has low complexity. Adoption of this approach eliminates the problem of angle wrap**. In higher dimensions the exact solution cannot be computed, and it is necessary to make (very close) approximations, which is done here. We demonstrate in simulations that, in contrast with some other approaches, our algorithm produces very accurate and statistically meaningful outputs.
△ Less
Submitted 25 March, 2020; v1 submitted 21 March, 2020;
originally announced March 2020.
-
A Poisson-Gaussian Denoising Dataset with Real Fluorescence Microscopy Images
Authors:
Yide Zhang,
Yinhao Zhu,
Evan Nichols,
Qingfei Wang,
Siyuan Zhang,
Cody Smith,
Scott Howard
Abstract:
Fluorescence microscopy has enabled a dramatic development in modern biology. Due to its inherently weak signal, fluorescence microscopy is not only much noisier than photography, but also presented with Poisson-Gaussian noise where Poisson noise, or shot noise, is the dominating noise source. To get clean fluorescence microscopy images, it is highly desirable to have effective denoising algorithm…
▽ More
Fluorescence microscopy has enabled a dramatic development in modern biology. Due to its inherently weak signal, fluorescence microscopy is not only much noisier than photography, but also presented with Poisson-Gaussian noise where Poisson noise, or shot noise, is the dominating noise source. To get clean fluorescence microscopy images, it is highly desirable to have effective denoising algorithms and datasets that are specifically designed to denoise fluorescence microscopy images. While such algorithms exist, no such datasets are available. In this paper, we fill this gap by constructing a dataset - the Fluorescence Microscopy Denoising (FMD) dataset - that is dedicated to Poisson-Gaussian denoising. The dataset consists of 12,000 real fluorescence microscopy images obtained with commercial confocal, two-photon, and wide-field microscopes and representative biological samples such as cells, zebrafish, and mouse brain tissues. We use image averaging to effectively obtain ground truth images and 60,000 noisy images with different noise levels. We use this dataset to benchmark 10 representative denoising algorithms and find that deep learning methods have the best performance. To our knowledge, this is the first real microscopy image dataset for Poisson-Gaussian denoising purposes and it could be an important tool for high-quality, real-time denoising applications in biomedical research.
△ Less
Submitted 5 April, 2019; v1 submitted 26 December, 2018;
originally announced December 2018.
-
Bounds on Multiple Sensor Fusion
Authors:
Bill Moran,
Fred Cohen,
Zengfu Wang,
Sofia Suvorova,
Douglas Cochran,
Tom Taylor,
Peter Farrell,
Stephen Howard
Abstract:
We consider the problem of fusing measurements from multiple sensors, where the sensing regions overlap and data are non-negative---possibly resulting from a count of indistinguishable discrete entities. Because of overlaps, it is, in general, impossible to fuse this information to arrive at an accurate estimate of the overall amount or count of material present in the union of the sensing regions…
▽ More
We consider the problem of fusing measurements from multiple sensors, where the sensing regions overlap and data are non-negative---possibly resulting from a count of indistinguishable discrete entities. Because of overlaps, it is, in general, impossible to fuse this information to arrive at an accurate estimate of the overall amount or count of material present in the union of the sensing regions. Here we study the range of overall values consistent with the data. Posed as a linear programming problem, this leads to interesting questions associated with the geometry of the sensor regions, specifically, the arrangement of their non-empty intersections. We define a computational tool called the fusion polytope and derive a condition for this to be in the positive orthant thus simplifying calculations. We show that, in two dimensions, inflated tiling schemes based on rectangular regions fail to satisfy this condition, whereas inflated tiling schemes based on hexagons do.
△ Less
Submitted 12 October, 2014;
originally announced October 2014.
