Showing 1–50 of 56 results for author: Bouman, A

Total Variation Regularization for Tomographic Reconstruction of Cylindrically Symmetric Objects

Authors: Maliha Hossain, Charles A. Bouman, Brendt Wohlberg

Abstract: Flash X-ray computed tomography (CT) is an important imaging modality for characterization of high-speed dynamic events, such as Kolsky bar impact experiments for the study of mechanical properties of materials subjected to impulsive forces. Due to experimental constraints, the number of X-ray views that can be obtained is typically very sparse in both space and time, requiring strong priors in or… ▽ More Flash X-ray computed tomography (CT) is an important imaging modality for characterization of high-speed dynamic events, such as Kolsky bar impact experiments for the study of mechanical properties of materials subjected to impulsive forces. Due to experimental constraints, the number of X-ray views that can be obtained is typically very sparse in both space and time, requiring strong priors in order to enable a CT reconstruction. In this paper, we propose an effective method for exploiting the cylindrical symmetry inherent in the experiment via a variant of total variation (TV) regularization that operates in cylindrical coordinates, and demonstrate that it outperforms competing approaches. △ Less

Submitted 25 June, 2024; originally announced June 2024.

Pixel-weighted Multi-pose Fusion for Metal Artifact Reduction in X-ray Computed Tomography

Authors: Diyu Yang, Craig A. J. Kemp, Soumendu Majee, Gregery T. Buzzard, Charles A. Bouman

Abstract: X-ray computed tomography (CT) reconstructs the internal morphology of a three dimensional object from a collection of projection images, most commonly using a single rotation axis. However, for objects containing dense materials like metal, the use of a single rotation axis may leave some regions of the object obscured by the metal, even though projections from other rotation axes (or poses) migh… ▽ More X-ray computed tomography (CT) reconstructs the internal morphology of a three dimensional object from a collection of projection images, most commonly using a single rotation axis. However, for objects containing dense materials like metal, the use of a single rotation axis may leave some regions of the object obscured by the metal, even though projections from other rotation axes (or poses) might contain complementary information that would better resolve these obscured regions. In this paper, we propose pixel-weighted Multi-pose Fusion to reduce metal artifacts by fusing the information from complementary measurement poses into a single reconstruction. Our method uses Multi-Agent Consensus Equilibrium (MACE), an extension of Plug-and-Play, as a framework for integrating projection data from different poses. A primary novelty of the proposed method is that the output of different MACE agents are fused in a pixel-weighted manner to minimize the effects of metal throughout the reconstruction. Using real CT data on an object with and without metal inserts, we demonstrate that the proposed pixel-weighted Multi-pose Fusion method significantly reduces metal artifacts relative to single-pose reconstructions. △ Less

Submitted 25 June, 2024; originally announced June 2024.

Comments: Submitted to IEEE MMSP 2024. arXiv admin note: substantial text overlap with arXiv:2209.07561

CLAMP: Majorized Plug-and-Play for Coherent 3D LIDAR Imaging

Authors: Tony G. Allen, David J. Rabb, Gregery T. Buzzard, Charles A. Bouman

Abstract: Coherent LIDAR uses a chirped laser pulse for 3D imaging of distant targets. However, existing coherent LIDAR image reconstruction methods do not account for the system's aperture, resulting in sub-optimal resolution. Moreover, these methods use majorization-minimization for computational efficiency, but do so without a theoretical treatment of convergence. In this paper, we present Coherent LID… ▽ More Coherent LIDAR uses a chirped laser pulse for 3D imaging of distant targets. However, existing coherent LIDAR image reconstruction methods do not account for the system's aperture, resulting in sub-optimal resolution. Moreover, these methods use majorization-minimization for computational efficiency, but do so without a theoretical treatment of convergence. In this paper, we present Coherent LIDAR Aperture Modeled Plug-and-Play (CLAMP) for multi-look coherent LIDAR image reconstruction. CLAMP uses multi-agent consensus equilibrium (a form of PnP) to combine a neural network denoiser with an accurate physics-based forward model. CLAMP introduces an FFT-based method to account for the effects of the aperture and uses majorization of the forward model for computational efficiency. We also formalize the use of majorization-minimization in consensus optimization problems and prove convergence to the exact consensus equilibrium solution. Finally, we apply CLAMP to synthetic and measured data to demonstrate its effectiveness in producing high-resolution, speckle-free, 3D imagery. △ Less

Submitted 19 June, 2024; originally announced June 2024.

MACE CT Reconstruction for Modular Material Decomposition from Energy Resolving Photon-Counting Data

Authors: Natalie M. Jadue, Madhuri Nagare, Jonathan S. Maltz, Gregery T. Buzzard, Charles A. Bouman

Abstract: X-ray computed tomography (CT) based on photon counting detectors (PCD) extends standard CT by counting detected photons in multiple energy bins. PCD data can be used to increase the contrast-to-noise ratio (CNR), increase spatial resolution, reduce radiation dose, reduce injected contrast dose, and compute a material decomposition using a specified set of basis materials. Current commercial and p… ▽ More X-ray computed tomography (CT) based on photon counting detectors (PCD) extends standard CT by counting detected photons in multiple energy bins. PCD data can be used to increase the contrast-to-noise ratio (CNR), increase spatial resolution, reduce radiation dose, reduce injected contrast dose, and compute a material decomposition using a specified set of basis materials. Current commercial and prototype clinical photon counting CT systems utilize PCD-CT reconstruction methods that either reconstruct from each spectral bin separately, or first create an estimate of a material sinogram using a specified set of basis materials and then reconstruct from these material sinograms. However, existing methods are not able to utilize simultaneously and in a modular fashion both the measured spectral information and advanced prior models in order to produce a material decomposition. We describe an efficient, modular framework for PCD-based CT reconstruction and material decomposition using on Multi-Agent Consensus Equilibrium (MACE). Our method employs a detector proximal map or agent that uses PCD measurements to update an estimate of the pathlength sinogram. We also create a prior agent in the form of a sinogram denoiser that enforces both physical and empirical knowledge about the material-decomposed sinogram. The sinogram reconstruction is computed using the MACE algorithm, which finds an equilibrium solution between the two agents, and the final image is reconstructed from the estimated sinogram. Importantly, the modularity of our method allows the two agents to be designed, implemented, and optimized independently. Our results on simulated data show a substantial (450%) CNR boost vs conventional maximum likelihood reconstruction when applied to a phantom used to evaluate low contrast detectability. △ Less

Submitted 1 February, 2024; originally announced February 2024.

Comments: 11 pages, 10 figures, submitted to SPIE Medical Imaging 2024

Texture Matching GAN for CT Image Enhancement

Authors: Madhuri Nagare, Gregery T. Buzzard, Charles A. Bouman

Abstract: Deep neural networks (DNN) are commonly used to denoise and sharpen X-ray computed tomography (CT) images with the goal of reducing patient X-ray dosage while maintaining reconstruction quality. However, naive application of DNN-based methods can result in image texture that is undesirable in clinical applications. Alternatively, generative adversarial network (GAN) based methods can produce appro… ▽ More Deep neural networks (DNN) are commonly used to denoise and sharpen X-ray computed tomography (CT) images with the goal of reducing patient X-ray dosage while maintaining reconstruction quality. However, naive application of DNN-based methods can result in image texture that is undesirable in clinical applications. Alternatively, generative adversarial network (GAN) based methods can produce appropriate texture, but naive application of GANs can introduce inaccurate or even unreal image detail. In this paper, we propose a texture matching generative adversarial network (TMGAN) that enhances CT images while generating an image texture that can be matched to a target texture. We use parallel generators to separate anatomical features from the generated texture, which allows the GAN to be trained to match the desired texture without directly affecting the underlying CT image. We demonstrate that TMGAN generates enhanced image quality while also producing image texture that is desirable for clinical application. △ Less

Submitted 20 December, 2023; originally announced December 2023.

Comments: Submitted to IEEE Transactions on Medical Imaging

Design of Novel Loss Functions for Deep Learning in X-ray CT

Authors: Obaidullah Rahman, Ken D. Sauer, Madhuri Nagare, Charles A. Bouman, Roman Melnyk, Jie Tang, Brian Nett

Abstract: Deep learning (DL) shows promise of advantages over conventional signal processing techniques in a variety of imaging applications. The networks' being trained from examples of data rather than explicitly designed allows them to learn signal and noise characteristics to most effectively construct a map** from corrupted data to higher quality representations. In inverse problems, one has options… ▽ More Deep learning (DL) shows promise of advantages over conventional signal processing techniques in a variety of imaging applications. The networks' being trained from examples of data rather than explicitly designed allows them to learn signal and noise characteristics to most effectively construct a map** from corrupted data to higher quality representations. In inverse problems, one has options of applying DL in the domain of the originally captured data, in the transformed domain of the desired final representation, or both. X-ray computed tomography (CT), one of the most valuable tools in medical diagnostics, is already being improved by DL methods. Whether for removal of common quantum noise resulting from the Poisson-distributed photon counts, or for reduction of the ill effects of metal implants on image quality, researchers have begun employing DL widely in CT. The selection of training data is driven quite directly by the corruption on which the focus lies. However, the way in which differences between the target signal and measured data is penalized in training generally follows conventional, pointwise loss functions. This work introduces a creative technique for favoring reconstruction characteristics that are not well described by norms such as mean-squared or mean-absolute error. Particularly in a field such as X-ray CT, where radiologists' subjective preferences in image characteristics are key to acceptance, it may be desirable to penalize differences in DL more creatively. This penalty may be applied in the data domain, here the CT sinogram, or in the reconstructed image. We design loss functions for both sha** and selectively preserving frequency content of the signal. △ Less

Submitted 23 September, 2023; originally announced September 2023.

Statistically Adaptive Filtering for Low Signal Correction in X-ray Computed Tomography

Authors: Obaidullah Rahman, Ken D. Sauer, Charles A. Bouman, Roman Melnyk, Brian Nett

Abstract: Low x-ray dose is desirable in x-ray computed tomographic (CT) imaging due to health concerns. But low dose comes with a cost of low signal artifacts such as streaks and low frequency bias in the reconstruction. As a result, low signal correction is needed to help reduce artifacts while retaining relevant anatomical structures. Low signal can be encountered in cases where sufficient number of ph… ▽ More Low x-ray dose is desirable in x-ray computed tomographic (CT) imaging due to health concerns. But low dose comes with a cost of low signal artifacts such as streaks and low frequency bias in the reconstruction. As a result, low signal correction is needed to help reduce artifacts while retaining relevant anatomical structures. Low signal can be encountered in cases where sufficient number of photons do not reach the detector to have confidence in the recorded data. % NOTE: SNR is ratio of powers, not std. dev. X-ray photons, assumed to have Poisson distribution, have signal to noise ratio proportional to the dose, with poorer SNR in low signal areas. Electronic noise added by the data acquisition system further reduces the signal quality. In this paper we will demonstrate a technique to combat low signal artifacts through adaptive filtration. It entails statistics-based filtering on the uncorrected data, correcting the lower signal areas more aggressively than the high signal ones. We look at local averages to decide how aggressive the filtering should be, and local standard deviation to decide how much detail preservation to apply. Implementation consists of a pre-correction step i.e. local linear minimum mean-squared error correction, followed by a variance stabilizing transform, and finally adaptive bilateral filtering. The coefficients of the bilateral filter are computed using local statistics. Results show that improvements were made in terms of low frequency bias, streaks, local average and standard deviation, modulation transfer function and noise power spectrum. △ Less

Submitted 23 September, 2023; originally announced September 2023.

MBIR Training for a 2.5D DL network in X-ray CT

Authors: Obaidullah Rahman, Madhuri Nagare, Ken D. Sauer, Charles A. Bouman, Roman Melnyk, Brian Nett, Jie Tang

Abstract: In computed tomographic imaging, model based iterative reconstruction methods have generally shown better image quality than the more traditional, faster filtered backprojection technique. The cost we have to pay is that MBIR is computationally expensive. In this work we train a 2.5D deep learning (DL) network to mimic MBIR quality image. The network is realized by a modified Unet, and trained usi… ▽ More In computed tomographic imaging, model based iterative reconstruction methods have generally shown better image quality than the more traditional, faster filtered backprojection technique. The cost we have to pay is that MBIR is computationally expensive. In this work we train a 2.5D deep learning (DL) network to mimic MBIR quality image. The network is realized by a modified Unet, and trained using clinical FBP and MBIR image pairs. We achieve the quality of MBIR images faster and with a much smaller computation cost. Visually and in terms of noise power spectrum (NPS), DL-MBIR images have texture similar to that of MBIR, with reduced noise power. Image profile plots, NPS plots, standard deviation, etc. suggest that the DL-MBIR images result from a successful emulation of an MBIR operator. △ Less

Submitted 23 September, 2023; originally announced September 2023.

Generative Plug and Play: Posterior Sampling for Inverse Problems

Authors: Charles A. Bouman, Gregery T. Buzzard

Abstract: Over the past decade, Plug-and-Play (PnP) has become a popular method for reconstructing images using a modular framework consisting of a forward and prior model. The great strength of PnP is that an image denoiser can be used as a prior model while the forward model can be implemented using more traditional physics-based approaches. However, a limitation of PnP is that it reconstructs only a sing… ▽ More Over the past decade, Plug-and-Play (PnP) has become a popular method for reconstructing images using a modular framework consisting of a forward and prior model. The great strength of PnP is that an image denoiser can be used as a prior model while the forward model can be implemented using more traditional physics-based approaches. However, a limitation of PnP is that it reconstructs only a single deterministic image. In this paper, we introduce Generative Plug-and-Play (GPnP), a generalization of PnP to sample from the posterior distribution. As with PnP, GPnP has a modular framework using a physics-based forward model and an image denoising prior model. However, in GPnP these models are extended to become proximal generators, which sample from associated distributions. GPnP applies these proximal generators in alternation to produce samples from the posterior. We present experimental simulations using the well-known BM3D denoiser. Our results demonstrate that the GPnP method is robust, easy to implement, and produces intuitively reasonable samples from the posterior for sparse interpolation and tomographic reconstruction. Code to accompany this paper is available at https://github.com/gbuzzard/generative-pnp-allerton . △ Less

Submitted 12 June, 2023; originally announced June 2023.

Comments: 8 pages, submitted to 2023 IEEE Allerton Conference

MSC Class: 94A08; 68U10; 60J22

Dynamic DH-MBIR for Phase-Error Estimation from Streaming Digital-Holography Data

Authors: Ali G. Sheikh, Casey J. Pellizzari, Sherman J. Kisner, Gregery T. Buzzard, Charles A. Bouman

Abstract: Directed energy applications require the estimation of digital-holographic (DH) phase errors due to atmospheric turbulence in order to accurately focus the outgoing beam. These phase error estimates must be computed with very low latency to keep pace with changing atmospheric parameters, which requires that phase errors be estimated in a single shot of DH data. The digital holography model-based i… ▽ More Directed energy applications require the estimation of digital-holographic (DH) phase errors due to atmospheric turbulence in order to accurately focus the outgoing beam. These phase error estimates must be computed with very low latency to keep pace with changing atmospheric parameters, which requires that phase errors be estimated in a single shot of DH data. The digital holography model-based iterative reconstruction (DH-MBIR) algorithm is capable of accurately estimating phase errors in a single shot using the expectation maximization (EM) algorithm. However, existing implementations of DH-MBIR require hundreds of iterations, which is not practical for real-time applications. In this paper, we present the Dynamic DH-MBIR (DDH-MBIR) algorithm for estimating isoplanatic phase errors from streaming single-shot data with extremely low latency. The Dynamic DH-MBIR algorithm reduces the computation and latency by orders of magnitude relative to conventional DH-MBIR, making real-time throughput and latency feasible in applications. Using simulated data that models frozen flow of atmospheric turbulence, we show that our algorithm can achieve a consistently high Strehl ratio with realistic simulation parameters using only 1 iteration per timestep. △ Less

Submitted 5 May, 2023; originally announced May 2023.

Comments: Submitted to 2023 IEEE Asilomar Conference on Signals, Systems, and Computers

Projected Multi-Agent Consensus Equilibrium (PMACE) with Application to Ptychography

Authors: Qiuchen Zhai, Gregery T. Buzzard, Kevin Mertes, Brendt Wohlberg, Charles A. Bouman

Abstract: Multi-Agent Consensus Equilibrium (MACE) formulates an inverse imaging problem as a balance among multiple update agents such as data-fitting terms and denoisers. However, each such agent operates on a separate copy of the full image, leading to redundant memory use and slow convergence when each agent affects only a small subset of the full image. In this paper, we extend MACE to Projected Multi-… ▽ More Multi-Agent Consensus Equilibrium (MACE) formulates an inverse imaging problem as a balance among multiple update agents such as data-fitting terms and denoisers. However, each such agent operates on a separate copy of the full image, leading to redundant memory use and slow convergence when each agent affects only a small subset of the full image. In this paper, we extend MACE to Projected Multi-Agent Consensus Equilibrium (PMACE), in which each agent updates only a projected component of the full image, thus greatly reducing memory use for some applications.We describe PMACE in terms of an equilibrium problem and an equivalent fixed point problem and show that in most cases the PMACE equilibrium is not the solution of an optimization problem. To demonstrate the value of PMACE, we apply it to the problem of ptychography, in which a sample is reconstructed from the diffraction patterns resulting from coherent X-ray illumination at multiple overlap** spots. In our PMACE formulation, each spot corresponds to a separate data-fitting agent, with the final solution found as an equilibrium among all the agents. Our results demonstrate that the PMACE reconstruction algorithm generates more accurate reconstructions at a lower computational cost than existing ptychography algorithms when the spots are sparsely sampled. △ Less

Submitted 5 October, 2023; v1 submitted 27 March, 2023; originally announced March 2023.

Autonomous Polycrystalline Material Decomposition for Hyperspectral Neutron Tomography

Authors: Mohammad Samin Nur Chowdhury, Diyu Yang, Shimin Tang, Singanallur V. Venkatakrishnan, Hassina Z. Bilheux, Gregery T. Buzzard, Charles A. Bouman

Abstract: Hyperspectral neutron tomography is an effective method for analyzing crystalline material samples with complex compositions in a non-destructive manner. Since the counts in the hyperspectral neutron radiographs directly depend on the neutron cross-sections, materials may exhibit contrasting neutron responses across wavelengths. Therefore, it is possible to extract the unique signatures associated… ▽ More Hyperspectral neutron tomography is an effective method for analyzing crystalline material samples with complex compositions in a non-destructive manner. Since the counts in the hyperspectral neutron radiographs directly depend on the neutron cross-sections, materials may exhibit contrasting neutron responses across wavelengths. Therefore, it is possible to extract the unique signatures associated with each material and use them to separate the crystalline phases simultaneously. We introduce an autonomous material decomposition (AMD) algorithm to automatically characterize and localize polycrystalline structures using Bragg edges with contrasting neutron responses from hyperspectral data. The algorithm estimates the linear attenuation coefficient spectra from the measured radiographs and then uses these spectra to perform polycrystalline material decomposition and reconstructs 3D material volumes to localize materials in the spatial domain. Our results demonstrate that the method can accurately estimate both the linear attenuation coefficient spectra and associated reconstructions on both simulated and experimental neutron data. △ Less

Submitted 21 August, 2023; v1 submitted 27 February, 2023; originally announced February 2023.

X-ray Spectral Estimation using Dictionary Learning

Authors: Wenrui Li, Venkatesh Sridhar, K. Aditya Mohan, Saransh Singh, Jean-Baptiste Forien, Xin Liu, Gregery T. Buzzard, Charles A. Bouman

Abstract: As computational tools for X-ray computed tomography (CT) become more quantitatively accurate, knowledge of the source-detector spectral response is critical for quantitative system-independent reconstruction and material characterization capabilities. Directly measuring the spectral response of a CT system is hard, which motivates spectral estimation using transmission data obtained from a collec… ▽ More As computational tools for X-ray computed tomography (CT) become more quantitatively accurate, knowledge of the source-detector spectral response is critical for quantitative system-independent reconstruction and material characterization capabilities. Directly measuring the spectral response of a CT system is hard, which motivates spectral estimation using transmission data obtained from a collection of known homogeneous objects. However, the associated inverse problem is ill-conditioned, making accurate estimation of the spectrum challenging, particularly in the absence of a close initial guess. In this paper, we describe a dictionary-based spectral estimation method that yields accurate results without the need for any initial estimate of the spectral response. Our method utilizes a MAP estimation framework that combines a physics-based forward model along with an $L_0$ sparsity constraint and a simplex constraint on the dictionary coefficients. Our method uses a greedy support selection method and a new pair-wise iterated coordinate descent method to compute the above estimate. We demonstrate that our dictionary-based method outperforms a state-of-the-art method as shown in a cross-validation experiment on four real datasets collected at beamline 8.3.2 of the Advanced Light Source (ALS). △ Less

Submitted 26 February, 2023; originally announced February 2023.

Comments: Document Release Number: LLNL-CONF-845171 Submitted to 2023 ICIP conference

TRINIDI: Time-of-Flight Resonance Imaging with Neutrons for Isotopic Density Inference

Authors: Thilo Balke, Alexander M. Long, Sven C. Vogel, Brendt Wohlberg, Charles A. Bouman

Abstract: Accurate reconstruction of 2D and 3D isotope densities is a desired capability with great potential impact in applications such as evaluation and development of next-generation nuclear fuels. Neutron time-of-flight (TOF) resonance imaging offers a potential approach by exploiting the characteristic neutron absorption spectra of each isotope. However, it is a major challenge to compute quantitative… ▽ More Accurate reconstruction of 2D and 3D isotope densities is a desired capability with great potential impact in applications such as evaluation and development of next-generation nuclear fuels. Neutron time-of-flight (TOF) resonance imaging offers a potential approach by exploiting the characteristic neutron absorption spectra of each isotope. However, it is a major challenge to compute quantitatively accurate images due to a variety of confounding effects such as severe Poisson noise, background scatter, beam non-uniformity, absorption non-linearity, and extended source pulse duration. We present the TRINIDI algorithm which is based on a two-step process in which we first estimate the neutron flux and background counts, and then reconstruct the areal densities of each isotope and pixel. Both components are based on the inversion of a forward model that accounts for the highly non-linear absorption, energy-dependent emission profile, and Poisson noise, while also modeling the substantial spatio-temporal variation of the background and flux. To do this, we formulate the non-linear inverse problem as two optimization problems that are solved in sequence. We demonstrate on both synthetic and measured data that TRINIDI can reconstruct quantitatively accurate 2D views of isotopic areal density that can then be reconstructed into quantitatively accurate 3D volumes of isotopic volumetric density. △ Less

Submitted 11 September, 2023; v1 submitted 24 February, 2023; originally announced February 2023.

Comments: 15 pages, 20 figures

Report number: LA-UR-23-21022

Ringing Artifact Reduction Method for Ultrasound Reconstruction Using Multi-Agent Consensus Equilibrium

Authors: Abdulrahman M. Alanazi, Singanallur Venkatakrishnan, Gregery T. Buzzard, Charles A. Bouman

Abstract: Non-destructive characterization of multi-layered structures that can be accessed from only a single side is important for applications such as well-bore integrity inspection. Existing methods related to Synthetic Aperture Focusing Technique (SAFT) rapidly produce acceptable results but with significant artifacts. Recently, ultrasound model-based iterative reconstruction (UMBIR) approaches have sh… ▽ More Non-destructive characterization of multi-layered structures that can be accessed from only a single side is important for applications such as well-bore integrity inspection. Existing methods related to Synthetic Aperture Focusing Technique (SAFT) rapidly produce acceptable results but with significant artifacts. Recently, ultrasound model-based iterative reconstruction (UMBIR) approaches have shown significant improvements over SAFT. However, even these methods produce ringing artifacts due to the high fractional-bandwidth of the excitation signal. In this paper, we propose a ringing artifact reduction method for ultrasound image reconstruction that uses a multi-agent consensus equilibrium (RARE-MACE) framework. Our approach integrates a physics-based forward model that accounts for the propagation of a collimated ultrasonic beam in multi-layered media, a spatially varying image prior, and a denoiser designed to suppress the ringing artifacts that are characteristic of reconstructions from high-fractional bandwidth ultrasound sensor data. We test our method on simulated and experimental measurements and show substantial improvements in image quality compared to SAFT and UMBIR. △ Less

Submitted 9 February, 2023; originally announced February 2023.

Comments: arXiv admin note: text overlap with arXiv:2211.15859, arXiv:2202.09703

Adaptive Coverage Path Planning for Efficient Exploration of Unknown Environments

Authors: Amanda Bouman, Joshua Ott, Sung-Kyun Kim, Kenny Chen, Mykel J. Kochenderfer, Brett Lopez, Ali-akbar Agha-mohammadi, Joel Burdick

Abstract: We present a method for solving the coverage problem with the objective of autonomously exploring an unknown environment under mission time constraints. Here, the robot is tasked with planning a path over a horizon such that the accumulated area swept out by its sensor footprint is maximized. Because this problem exhibits a diminishing returns property known as submodularity, we choose to formulat… ▽ More We present a method for solving the coverage problem with the objective of autonomously exploring an unknown environment under mission time constraints. Here, the robot is tasked with planning a path over a horizon such that the accumulated area swept out by its sensor footprint is maximized. Because this problem exhibits a diminishing returns property known as submodularity, we choose to formulate it as a tree-based sequential decision making process. This formulation allows us to evaluate the effects of the robot's actions on future world coverage states, while simultaneously accounting for traversability risk and the dynamic constraints of the robot. To quickly find near-optimal solutions, we propose an effective approximation to the coverage sensor model which adapts to the local environment. Our method was extensively tested across various complex environments and served as the local exploration algorithm for a competing entry in the DARPA Subterranean Challenge. △ Less

Submitted 6 February, 2023; originally announced February 2023.

An Edge Alignment-based Orientation Selection Method for Neutron Tomography

Authors: Diyu Yang, Shimin Tang, Singanallur V. Venkatakrishnan, Mohammad S. N. Chowdhury, Yuxuan Zhang, Hassina Z. Bilheux, Gregery T. Buzzard, Charles A. Bouman

Abstract: Neutron computed tomography (nCT) is a 3D characterization technique used to image the internal morphology or chemical composition of samples in biology and materials sciences. A typical workflow involves placing the sample in the path of a neutron beam, acquiring projection data at a predefined set of orientations, and processing the resulting data using an analytic reconstruction algorithm. Typi… ▽ More Neutron computed tomography (nCT) is a 3D characterization technique used to image the internal morphology or chemical composition of samples in biology and materials sciences. A typical workflow involves placing the sample in the path of a neutron beam, acquiring projection data at a predefined set of orientations, and processing the resulting data using an analytic reconstruction algorithm. Typical nCT scans require hours to days to complete and are then processed using conventional filtered back-projection (FBP), which performs poorly with sparse views or noisy data. Hence, the main methods in order to reduce overall acquisition time are the use of an improved sampling strategy combined with the use of advanced reconstruction methods such as model-based iterative reconstruction (MBIR). In this paper, we propose an adaptive orientation selection method in which an MBIR reconstruction on previously-acquired measurements is used to define an objective function on orientations that balances a data-fitting term promoting edge alignment and a regularization term promoting orientation diversity. Using simulated and experimental data, we demonstrate that our method produces high-quality reconstructions using significantly fewer total measurements than the conventional approach. △ Less

Submitted 8 March, 2023; v1 submitted 1 December, 2022; originally announced December 2022.

Multi-Pose Fusion for Sparse-View CT Reconstruction Using Consensus Equilibrium

Authors: Diyu Yang, Craig A. J. Kemp, Gregery T. Buzzard, Charles A. Bouman

Abstract: CT imaging works by reconstructing an object of interest from a collection of projections. Traditional methods such as filtered-back projection (FBP) work on projection images acquired around a fixed rotation axis. However, for some CT problems, it is desirable to perform a joint reconstruction from projection data acquired from multiple rotation axes. In this paper, we present Multi-Pose Fusion… ▽ More CT imaging works by reconstructing an object of interest from a collection of projections. Traditional methods such as filtered-back projection (FBP) work on projection images acquired around a fixed rotation axis. However, for some CT problems, it is desirable to perform a joint reconstruction from projection data acquired from multiple rotation axes. In this paper, we present Multi-Pose Fusion, a novel algorithm that performs a joint tomographic reconstruction from CT scans acquired from multiple poses of a single object, where each pose has a distinct rotation axis. Our approach uses multi-agent consensus equilibrium (MACE), an extension of plug-and-play, as a framework for integrating projection data from different poses. We apply our method on simulated data and demonstrate that Multi-Pose Fusion can achieve a better reconstruction result than single pose reconstruction. △ Less

Submitted 15 September, 2022; originally announced September 2022.

Comments: To appear in 58th Annual Allerton Conference on Communication, Control, and Computing

Risk-aware Meta-level Decision Making for Exploration Under Uncertainty

Authors: Joshua Ott, Sung-Kyun Kim, Amanda Bouman, Oriana Peltzer, Mamoru Sobue, Harrison Delecki, Mykel J. Kochenderfer, Joel Burdick, Ali-akbar Agha-mohammadi

Abstract: Robotic exploration of unknown environments is fundamentally a problem of decision making under uncertainty where the robot must account for uncertainty in sensor measurements, localization, action execution, as well as many other factors. For large-scale exploration applications, autonomous systems must overcome the challenges of sequentially deciding which areas of the environment are valuable t… ▽ More Robotic exploration of unknown environments is fundamentally a problem of decision making under uncertainty where the robot must account for uncertainty in sensor measurements, localization, action execution, as well as many other factors. For large-scale exploration applications, autonomous systems must overcome the challenges of sequentially deciding which areas of the environment are valuable to explore while safely evaluating the risks associated with obstacles and hazardous terrain. In this work, we propose a risk-aware meta-level decision making framework to balance the tradeoffs associated with local and global exploration. Meta-level decision making builds upon classical hierarchical coverage planners by switching between local and global policies with the overall objective of selecting the policy that is most likely to maximize reward in a stochastic environment. We use information about the environment history, traversability risk, and kinodynamic constraints to reason about the probability of successful policy execution to switch between local and global policies. We have validated our solution in both simulation and on a variety of large-scale real world hardware tests. Our results show that by balancing local and global exploration we are able to significantly explore large-scale environments more efficiently. △ Less

Submitted 30 April, 2024; v1 submitted 12 September, 2022; originally announced September 2022.

Comments: IEEE International Conference on Control, Decision and Information Technologies

ACHORD: Communication-Aware Multi-Robot Coordination with Intermittent Connectivity

Authors: Maira Saboia, Lillian Clark, Vivek Thangavelu, Jeffrey A. Edlund, Kyohei Otsu, Gustavo J. Correa, Vivek Shankar Varadharajan, Angel Santamaria-Navarro, Thomas Touma, Amanda Bouman, Hovhannes Melikyan, Torkom Pailevanian, Sung-Kyun Kim, Avak Archanian, Tiago Stegun Vaquero, Giovanni Beltrame, Nils Napp, Gustavo Pessin, Ali-akbar Agha-mohammadi

Abstract: Communication is an important capability for multi-robot exploration because (1) inter-robot communication (comms) improves coverage efficiency and (2) robot-to-base comms improves situational awareness. Exploring comms-restricted (e.g., subterranean) environments requires a multi-robot system to tolerate and anticipate intermittent connectivity, and to carefully consider comms requirements, other… ▽ More Communication is an important capability for multi-robot exploration because (1) inter-robot communication (comms) improves coverage efficiency and (2) robot-to-base comms improves situational awareness. Exploring comms-restricted (e.g., subterranean) environments requires a multi-robot system to tolerate and anticipate intermittent connectivity, and to carefully consider comms requirements, otherwise mission-critical data may be lost. In this paper, we describe and analyze ACHORD (Autonomous & Collaborative High-Bandwidth Operations with Radio Droppables), a multi-layer networking solution which tightly co-designs the network architecture and high-level decision-making for improved comms. ACHORD provides bandwidth prioritization and timely and reliable data transfer despite intermittent connectivity. Furthermore, it exposes low-layer networking metrics to the application layer to enable robots to autonomously monitor, map, and extend the network via droppable radios, as well as restore connectivity to improve collaborative exploration. We evaluate our solution with respect to the comms performance in several challenging underground environments including the DARPA SubT Finals competition environment. Our findings support the use of data stratification and flow control to improve bandwidth-usage. △ Less

Submitted 5 June, 2022; originally announced June 2022.

Comments: 8 pages, 8 figures

Plug-and-Play Methods for Integrating Physical and Learned Models in Computational Imaging

Authors: Ulugbek S. Kamilov, Charles A. Bouman, Gregery T. Buzzard, Brendt Wohlberg

Abstract: Plug-and-Play Priors (PnP) is one of the most widely-used frameworks for solving computational imaging problems through the integration of physical models and learned models. PnP leverages high-fidelity physical sensor models and powerful machine learning methods for prior modeling of data to provide state-of-the-art reconstruction algorithms. PnP algorithms alternate between minimizing a data-fid… ▽ More Plug-and-Play Priors (PnP) is one of the most widely-used frameworks for solving computational imaging problems through the integration of physical models and learned models. PnP leverages high-fidelity physical sensor models and powerful machine learning methods for prior modeling of data to provide state-of-the-art reconstruction algorithms. PnP algorithms alternate between minimizing a data-fidelity term to promote data consistency and imposing a learned regularizer in the form of an image denoiser. Recent highly-successful applications of PnP algorithms include bio-microscopy, computerized tomography, magnetic resonance imaging, and joint ptycho-tomography. This article presents a unified and principled review of PnP by tracing its roots, describing its major variations, summarizing main results, and discussing applications in computational imaging. We also point the way towards further developments by discussing recent results on equilibrium equations that formulate the problem associated with PnP algorithms. △ Less

Submitted 12 August, 2022; v1 submitted 31 March, 2022; originally announced March 2022.

FIG-OP: Exploring Large-Scale Unknown Environments on a Fixed Time Budget

Authors: Oriana Peltzer, Amanda Bouman, Sung-Kyun Kim, Ransalu Senanayake, Joshua Ott, Harrison Delecki, Mamoru Sobue, Mykel Kochenderfer, Mac Schwager, Joel Burdick, Ali-akbar Agha-mohammadi

Abstract: We present a method for autonomous exploration of large-scale unknown environments under mission time constraints. We start by proposing the Frontloaded Information Gain Orienteering Problem (FIG-OP) -- a generalization of the traditional orienteering problem where the assumption of a reliable environmental model no longer holds. The FIG-OP addresses model uncertainty by frontloading expected info… ▽ More We present a method for autonomous exploration of large-scale unknown environments under mission time constraints. We start by proposing the Frontloaded Information Gain Orienteering Problem (FIG-OP) -- a generalization of the traditional orienteering problem where the assumption of a reliable environmental model no longer holds. The FIG-OP addresses model uncertainty by frontloading expected information gain through the addition of a greedy incentive, effectively expediting the moment in which new area is uncovered. In order to reason across multi-kilometre environments, we solve FIG-OP over an information-efficient world representation, constructed through the aggregation of information from a topological and metric map. Our method was extensively tested and field-hardened across various complex environments, ranging from subway systems to mines. In comparative simulations, we observe that the FIG-OP solution exhibits improved coverage efficiency over solutions generated by greedy and traditional orienteering-based approaches (i.e. severe and minimal model uncertainty assumptions, respectively). △ Less

Submitted 11 March, 2022; originally announced March 2022.

Comments: 9 pages, 8 figures, 1 table

Sparse-View CT Reconstruction using Recurrent Stacked Back Projection

Authors: Wenrui Li, Gregery T. Buzzard, Charles A. Bouman

Abstract: Sparse-view CT reconstruction is important in a wide range of applications due to limitations on cost, acquisition time, or dosage. However, traditional direct reconstruction methods such as filtered back-projection (FBP) lead to low-quality reconstructions in the sub-Nyquist regime. In contrast, deep neural networks (DNNs) can produce high-quality reconstructions from sparse and noisy data, e.g.… ▽ More Sparse-view CT reconstruction is important in a wide range of applications due to limitations on cost, acquisition time, or dosage. However, traditional direct reconstruction methods such as filtered back-projection (FBP) lead to low-quality reconstructions in the sub-Nyquist regime. In contrast, deep neural networks (DNNs) can produce high-quality reconstructions from sparse and noisy data, e.g. through post-processing of FBP reconstructions, as can model-based iterative reconstruction (MBIR), albeit at a higher computational cost. In this paper, we introduce a direct-reconstruction DNN method called Recurrent Stacked Back Projection (RSBP) that uses sequentially-acquired backprojections of individual views as input to a recurrent convolutional LSTM network. The SBP structure maintains all information in the sinogram, while the recurrent processing exploits the correlations between adjacent views and produces an updated reconstruction after each new view. We train our network on simulated data and test on both simulated and real data and demonstrate that RSBP outperforms both DNN post-processing of FBP images and basic MBIR, with a lower computational cost than MBIR. △ Less

Submitted 9 December, 2021; originally announced December 2021.

Comments: 5 pages, 5 pages, 2021 Asilomar Conference on Signals, Systems, and Computers

High-Precision Inversion of Dynamic Radiography Using Hydrodynamic Features

Authors: Maliha Hossain, Balasubramanya T. Nadiga, Oleg Korobkin, Marc L. Klasky, Jennifer L. Schei, Joshua W. Burby, Michael T. McCann, Trevor Wilcox, Soumi De, Charles A. Bouman

Abstract: Radiography is often used to probe complex, evolving density fields in dynamic systems and in so doing gain insight into the underlying physics. This technique has been used in numerous fields including materials science, shock physics, inertial confinement fusion, and other national security applications. In many of these applications, however, complications resulting from noise, scatter, complex… ▽ More Radiography is often used to probe complex, evolving density fields in dynamic systems and in so doing gain insight into the underlying physics. This technique has been used in numerous fields including materials science, shock physics, inertial confinement fusion, and other national security applications. In many of these applications, however, complications resulting from noise, scatter, complex beam dynamics, etc. prevent the reconstruction of density from being accurate enough to identify the underlying physics with sufficient confidence. As such, density reconstruction from static/dynamic radiography has typically been limited to identifying discontinuous features such as cracks and voids in a number of these applications. In this work, we propose a fundamentally new approach to reconstructing density from a temporal sequence of radiographic images. Using only the robust features identifiable in radiographs, we combine them with the underlying hydrodynamic equations of motion using a machine learning approach, namely, conditional generative adversarial networks (cGAN), to determine the density fields from a dynamic sequence of radiographs. Next, we seek to further enhance the hydrodynamic consistency of the ML-based density reconstruction through a process of parameter estimation and projection onto a hydrodynamic manifold. In this context, we note that the distance from the hydrodynamic manifold given by the training data to the test data in the parameter space considered both serves as a diagnostic of the robustness of the predictions and serves to augment the training database, with the expectation that the latter will further reduce future density reconstruction errors. Finally, we demonstrate the ability of this method to outperform a traditional radiographic reconstruction in capturing allowable hydrodynamic paths even when relatively small amounts of scatter are present. △ Less

Submitted 2 December, 2021; originally announced December 2021.

Comments: Submitted to Optics Express

Journal ref: Opt. Express, vol. 30, no. 9, pp. 14432-14452, Apr. 2022

Projected Multi-Agent Consensus Equilibrium for Ptychographic Image Reconstruction

Authors: Qiuchen Zhai, Brendt Wohlberg, Gregery T. Buzzard, Charles A. Bouman

Abstract: Ptychography is a computational imaging technique using multiple, overlap**, coherently illuminated snapshots to achieve nanometer resolution by solving a nonlinear phase-field recovery problem. Ptychography is vital for imaging of manufactured nanomaterials, but existing algorithms have computational shortcomings that limit large-scale application. In this paper, we present the Projected Multi-… ▽ More Ptychography is a computational imaging technique using multiple, overlap**, coherently illuminated snapshots to achieve nanometer resolution by solving a nonlinear phase-field recovery problem. Ptychography is vital for imaging of manufactured nanomaterials, but existing algorithms have computational shortcomings that limit large-scale application. In this paper, we present the Projected Multi-Agent Consensus Equilibrium (PMACE) approach for solving the ptychography inversion problem. This approach extends earlier work on MACE, which formulates an inversion problem as an equilibrium among multiple agents, each acting independently to update a full reconstruction. In PMACE, each agent acts on a portion (projection) corresponding to one of the snapshots, and these updates to projections are then combined to give an update to the full reconstruction. The resulting algorithm is easily parallelized, with convergence properties inherited from convergence results associated with MACE. We apply our method on simulated data and demonstrate that it outperforms competing algorithms in both reconstruction quality and convergence speed. △ Less

Submitted 8 December, 2021; v1 submitted 28 November, 2021; originally announced November 2021.

Comments: To be published in Asilomar Conference on Signals, Systems, and Computers 2021

CodEx: A Modular Framework for Joint Temporal De-blurring and Tomographic Reconstruction

Authors: Soumendu Majee, Selin Aslan, Doga Gursoy, Charles A. Bouman

Abstract: In many computed tomography (CT) imaging applications, it is important to rapidly collect data from an object that is moving or changing with time. Tomographic acquisition is generally assumed to be step-and-shoot, where the object is rotated to each desired angle, and a view is taken. However, step-and-shoot acquisition is slow and can waste photons, so in practice fly-scanning is done where the… ▽ More In many computed tomography (CT) imaging applications, it is important to rapidly collect data from an object that is moving or changing with time. Tomographic acquisition is generally assumed to be step-and-shoot, where the object is rotated to each desired angle, and a view is taken. However, step-and-shoot acquisition is slow and can waste photons, so in practice fly-scanning is done where the object is continuously rotated while collecting data. However, this can result in motion-blurred views and consequently reconstructions with severe motion artifacts. In this paper, we introduce CodEx, a modular framework for joint de-blurring and tomographic reconstruction that can effectively invert the motion blur introduced in sparse view fly-scanning. The method is a synergistic combination of a novel acquisition method with a novel non-convex Bayesian reconstruction algorithm. CodEx works by encoding the acquisition with a known binary code that the reconstruction algorithm then inverts. Using a well chosen binary code to encode the measurements can improve the accuracy of the inversion process. The CodEx reconstruction method uses the alternating direction method of multipliers (ADMM) to split the inverse problem into iterative deblurring and reconstruction sub-problems, making reconstruction practical to implement. We present reconstruction results on both simulated and binned experimental data to demonstrate the effectiveness of our method. △ Less

Submitted 30 July, 2022; v1 submitted 11 November, 2021; originally announced November 2021.

Hyperspectral Neutron CT with Material Decomposition

Authors: Thilo Balke, Alexander M. Long, Sven C. Vogel, Brendt Wohlberg, Charles A. Bouman

Abstract: Energy resolved neutron imaging (ERNI) is an advanced neutron radiography technique capable of non-destructively extracting spatial isotopic information within a given material. Energy-dependent radiography image sequences can be created by utilizing neutron time-of-flight techniques. In combination with uniquely characteristic isotopic neutron cross-section spectra, isotopic areal densities can b… ▽ More Energy resolved neutron imaging (ERNI) is an advanced neutron radiography technique capable of non-destructively extracting spatial isotopic information within a given material. Energy-dependent radiography image sequences can be created by utilizing neutron time-of-flight techniques. In combination with uniquely characteristic isotopic neutron cross-section spectra, isotopic areal densities can be determined on a per-pixel basis, thus resulting in a set of areal density images for each isotope present in the sample. By preforming ERNI measurements over several rotational views, an isotope decomposed 3D computed tomography is possible. We demonstrate a method involving a robust and automated background estimation based on a linear programming formulation. The extremely high noise due to low count measurements is overcome using a sparse coding approach. It allows for a significant computation time improvement, from weeks to a few hours compared to existing neutron evaluation tools, enabling at the present stage a semi-quantitative, user-friendly routine application. △ Less

Submitted 5 October, 2021; originally announced October 2021.

Comments: 5 pages, 4 figures

Report number: LA-UR-21-21281

Journal ref: 2021 IEEE International Conference on Image Processing (ICIP), 2021, pp. 3482-3486

Multi-Resolution Data Fusion for Super Resolution Imaging

Authors: Emma J Reid, Lawrence F Drummy, Charles A Bouman, Gregery T Buzzard

Abstract: Applications in materials and biological imaging are limited by the ability to collect high-resolution data over large areas in practical amounts of time. One solution to this problem is to collect low-resolution data and interpolate to produce a high-resolution image. However, most existing super-resolution algorithms are designed for natural images, often require aligned pairing of high and low-… ▽ More Applications in materials and biological imaging are limited by the ability to collect high-resolution data over large areas in practical amounts of time. One solution to this problem is to collect low-resolution data and interpolate to produce a high-resolution image. However, most existing super-resolution algorithms are designed for natural images, often require aligned pairing of high and low-resolution training data, and may not directly incorporate a model of the imaging sensor. In this paper, we present a Multi-resolution Data Fusion (MDF) algorithm for accurate interpolation of low-resolution data at multiple resolutions up to 8x. Our approach uses small quantities of unpaired high-resolution data to train a neural network prior model denoiser and then uses the Multi-Agent Consensus Equilibrium (MACE) problem formulation to balance this denoiser with a forward model agent that promotes fidelity to measured data. A key theoretical novelty is the analysis of mismatched back-projectors, which modify typical forward model updates for computational efficiency or improved image quality. We use MACE to prove that using a mismatched back-projector is equivalent to using a standard back-projector and an appropriately modified prior model. We present electron microscopy results at 4x and 8x interpolation factors that exhibit reduced artifacts relative to existing methods while maintaining fidelity to acquired data and accurately resolving sub-pixel-scale features. △ Less

Submitted 1 January, 2022; v1 submitted 13 May, 2021; originally announced May 2021.

Algorithm-driven Advances for Scientific CT Instruments: From Model-based to Deep Learning-based Approaches

Authors: S. V. Venkatakrishnan, K. Aditya Mohan, Amir Koushyar Ziabari, Charles A. Bouman

Abstract: Multi-scale 3D characterization is widely used by materials scientists to further their understanding of the relationships between microscopic structure and macroscopic function. Scientific computed tomography (CT) instruments are one of the most popular choices for 3D non-destructive characterization of materials at length scales ranging from the angstrom-scale to the micron-scale. These instrume… ▽ More Multi-scale 3D characterization is widely used by materials scientists to further their understanding of the relationships between microscopic structure and macroscopic function. Scientific computed tomography (CT) instruments are one of the most popular choices for 3D non-destructive characterization of materials at length scales ranging from the angstrom-scale to the micron-scale. These instruments typically have a source of radiation that interacts with the sample to be studied and a detector assembly to capture the result of this interaction. A collection of such high-resolution measurements are made by re-orienting the sample which is mounted on a specially designed stage/holder after which reconstruction algorithms are used to produce the final 3D volume of interest. The end goal of scientific CT scans include determining the morphology,chemical composition or dynamic behavior of materials when subjected to external stimuli. In this article, we will present an overview of recent advances in reconstruction algorithms that have enabled significant improvements in the performance of scientific CT instruments - enabling faster, more accurate and novel imaging capabilities. In the first part, we will focus on model-based image reconstruction algorithms that formulate the inversion as solving a high-dimensional optimization problem involving a data-fidelity term and a regularization term. In the last part of the article, we will present an overview of recent approaches using deep-learning based algorithms for improving scientific CT instruments. △ Less

Submitted 15 September, 2021; v1 submitted 16 April, 2021; originally announced April 2021.

Ultra-Sparse View Reconstruction for Flash X-Ray Imaging using Consensus Equilibrium

Authors: Maliha Hossain, Shane C. Paulson, Hangjie Liao, Weinong W. Chen, Charles A. Bouman

Abstract: A growing number of applications require the reconstructionof 3D objects from a very small number of views. In this research, we consider the problem of reconstructing a 3D object from only 4 Flash X-ray CT views taken during the impact of a Kolsky bar. For such ultra-sparse view datasets, even model-based iterative reconstruction (MBIR) methods produce poor quality results. In this paper, we pr… ▽ More A growing number of applications require the reconstructionof 3D objects from a very small number of views. In this research, we consider the problem of reconstructing a 3D object from only 4 Flash X-ray CT views taken during the impact of a Kolsky bar. For such ultra-sparse view datasets, even model-based iterative reconstruction (MBIR) methods produce poor quality results. In this paper, we present a framework based on a generalization of Plug-and-Play, known as Multi-Agent Consensus Equilibrium (MACE), for incorporating complex and nonlinear prior information into ultra-sparse CT reconstruction. The MACE method allows any number of agents to simultaneously enforce their own prior constraints on the solution. We apply our method on simulated and real data and demonstrate that MACE reduces artifacts, improves reconstructed image quality, and uncovers image features which were otherwise indiscernible. △ Less

Submitted 12 April, 2021; v1 submitted 29 March, 2021; originally announced March 2021.

Comments: To be published in Asilomar Conference on Signals, Systems, and Computers 2020

Comment on Buss et al., Science 2021: An alternative, empirically-supported adjustment for sero-reversion yields a 10 percentage point lower estimate of the cumulative incidence of SARS-CoV-2 in Manaus by October 2020

Authors: Sarah Kadelka, Judith A Bouman, Peter Ashcroft, Roland R Regoes

Abstract: The estimate of the cumulative incidence of SARS-CoV-2 of 76% in Manaus by October 2020 by Buss et al. relies on the assumption of an exponentially-declining probability of sero-reversion over time. We present an alternative, empirically-supported approach that is based on the observed dynamics of antibody titers in sero-positive cases. Through this approach we revise the cumulative incidence esti… ▽ More The estimate of the cumulative incidence of SARS-CoV-2 of 76% in Manaus by October 2020 by Buss et al. relies on the assumption of an exponentially-declining probability of sero-reversion over time. We present an alternative, empirically-supported approach that is based on the observed dynamics of antibody titers in sero-positive cases. Through this approach we revise the cumulative incidence estimate to 66% (63.3% - 68.5%) by October 2020. This estimate has implications for the proximity to herd immunity and future estimates of fitness advantages of virus variants, such as the P.1 variant of concern. This methodology is also relevant for any other sero-survey analysis that has to adjust for sero-reversion. △ Less

Submitted 25 March, 2021; originally announced March 2021.

NeBula: Quest for Robotic Autonomy in Challenging Environments; TEAM CoSTAR at the DARPA Subterranean Challenge

Authors: Ali Agha, Kyohei Otsu, Benjamin Morrell, David D. Fan, Rohan Thakker, Angel Santamaria-Navarro, Sung-Kyun Kim, Amanda Bouman, Xianmei Lei, Jeffrey Edlund, Muhammad Fadhil Ginting, Kamak Ebadi, Matthew Anderson, Torkom Pailevanian, Edward Terry, Michael Wolf, Andrea Tagliabue, Tiago Stegun Vaquero, Matteo Palieri, Scott Tepsuporn, Yun Chang, Arash Kalantari, Fernando Chavez, Brett Lopez, Nobuhiro Funabiki , et al. (47 additional authors not shown)

Abstract: This paper presents and discusses algorithms, hardware, and software architecture developed by the TEAM CoSTAR (Collaborative SubTerranean Autonomous Robots), competing in the DARPA Subterranean Challenge. Specifically, it presents the techniques utilized within the Tunnel (2019) and Urban (2020) competitions, where CoSTAR achieved 2nd and 1st place, respectively. We also discuss CoSTAR's demonstr… ▽ More This paper presents and discusses algorithms, hardware, and software architecture developed by the TEAM CoSTAR (Collaborative SubTerranean Autonomous Robots), competing in the DARPA Subterranean Challenge. Specifically, it presents the techniques utilized within the Tunnel (2019) and Urban (2020) competitions, where CoSTAR achieved 2nd and 1st place, respectively. We also discuss CoSTAR's demonstrations in Martian-analog surface and subsurface (lava tubes) exploration. The paper introduces our autonomy solution, referred to as NeBula (Networked Belief-aware Perceptual Autonomy). NeBula is an uncertainty-aware framework that aims at enabling resilient and modular autonomy solutions by performing reasoning and decision making in the belief space (space of probability distributions over the robot and world states). We discuss various components of the NeBula framework, including: (i) geometric and semantic environment map**; (ii) a multi-modal positioning system; (iii) traversability analysis and local planning; (iv) global motion planning and exploration behavior; (i) risk-aware mission planning; (vi) networking and decentralized reasoning; and (vii) learning-enabled adaptation. We discuss the performance of NeBula on several robot types (e.g. wheeled, legged, flying), in various environments. We discuss the specific results and lessons learned from fielding this solution in the challenging courses of the DARPA Subterranean Challenge competition. △ Less

Submitted 18 October, 2021; v1 submitted 21 March, 2021; originally announced March 2021.

Comments: For team website, see https://costar.jpl.nasa.gov/. Accepted for publication in the Journal of Field Robotics, 2021

PLGRIM: Hierarchical Value Learning for Large-scale Exploration in Unknown Environments

Authors: Sung-Kyun Kim, Amanda Bouman, Gautam Salhotra, David D. Fan, Kyohei Otsu, Joel Burdick, Ali-akbar Agha-mohammadi

Abstract: In order for an autonomous robot to efficiently explore an unknown environment, it must account for uncertainty in sensor measurements, hazard assessment, localization, and motion execution. Making decisions for maximal reward in a stochastic setting requires value learning and policy construction over a belief space, i.e., probability distribution over all possible robot-world states. However, be… ▽ More In order for an autonomous robot to efficiently explore an unknown environment, it must account for uncertainty in sensor measurements, hazard assessment, localization, and motion execution. Making decisions for maximal reward in a stochastic setting requires value learning and policy construction over a belief space, i.e., probability distribution over all possible robot-world states. However, belief space planning in a large spatial environment over long temporal horizons suffers from severe computational challenges. Moreover, constructed policies must safely adapt to unexpected changes in the belief at runtime. This work proposes a scalable value learning framework, PLGRIM (Probabilistic Local and Global Reasoning on Information roadMaps), that bridges the gap between (i) local, risk-aware resiliency and (ii) global, reward-seeking mission objectives. Leveraging hierarchical belief space planners with information-rich graph structures, PLGRIM addresses large-scale exploration problems while providing locally near-optimal coverage plans. We validate our proposed framework with high-fidelity dynamic simulations in diverse environments and on physical robots in Martian-analog lava tubes. △ Less

Submitted 20 July, 2021; v1 submitted 10 February, 2021; originally announced February 2021.

Autonomous Spot: Long-Range Autonomous Exploration of Extreme Environments with Legged Locomotion

Authors: Amanda Bouman, Muhammad Fadhil Ginting, Nikhilesh Alatur, Matteo Palieri, David D. Fan, Thomas Touma, Torkom Pailevanian, Sung-Kyun Kim, Kyohei Otsu, Joel Burdick, Ali-akbar Agha-mohammadi

Abstract: This paper serves as one of the first efforts to enable large-scale and long-duration autonomy using the Boston Dynamics Spot robot. Motivated by exploring extreme environments, particularly those involved in the DARPA Subterranean Challenge, this paper pushes the boundaries of the state-of-practice in enabling legged robotic systems to accomplish real-world complex missions in relevant scenarios.… ▽ More This paper serves as one of the first efforts to enable large-scale and long-duration autonomy using the Boston Dynamics Spot robot. Motivated by exploring extreme environments, particularly those involved in the DARPA Subterranean Challenge, this paper pushes the boundaries of the state-of-practice in enabling legged robotic systems to accomplish real-world complex missions in relevant scenarios. In particular, we discuss the behaviors and capabilities which emerge from the integration of the autonomy architecture NeBula (Networked Belief-aware Perceptual Autonomy) with next-generation mobility systems. We will discuss the hardware and software challenges, and solutions in mobility, perception, autonomy, and very briefly, wireless networking, as well as lessons learned and future directions. We demonstrate the performance of the proposed solutions on physical systems in real-world scenarios. △ Less

Submitted 30 November, 2020; v1 submitted 19 October, 2020; originally announced October 2020.

Comments: Updated copyright statement and acknowledgements

Multi-Slice Fusion for Sparse-View and Limited-Angle 4D CT Reconstruction

Authors: Soumendu Majee, Thilo Balke, Craig A. J. Kemp, Gregery T. Buzzard, Charles A. Bouman

Abstract: Inverse problems spanning four or more dimensions such as space, time and other independent parameters have become increasingly important. State-of-the-art 4D reconstruction methods use model based iterative reconstruction (MBIR), but depend critically on the quality of the prior modeling. Recently, plug-and-play (PnP) methods have been shown to be an effective way to incorporate advanced prior mo… ▽ More Inverse problems spanning four or more dimensions such as space, time and other independent parameters have become increasingly important. State-of-the-art 4D reconstruction methods use model based iterative reconstruction (MBIR), but depend critically on the quality of the prior modeling. Recently, plug-and-play (PnP) methods have been shown to be an effective way to incorporate advanced prior models using state-of-the-art denoising algorithms. However, state-of-the-art denoisers such as BM4D and deep convolutional neural networks (CNNs) are primarily available for 2D or 3D images and extending them to higher dimensions is difficult due to algorithmic complexity and the increased difficulty of effective training. In this paper, we present multi-slice fusion, a novel algorithm for 4D reconstruction, based on the fusion of multiple low-dimensional denoisers. Our approach uses multi-agent consensus equilibrium (MACE), an extension of plug-and-play, as a framework for integrating the multiple lower-dimensional models. We apply our method to 4D cone-beam X-ray CT reconstruction for non destructive evaluation (NDE) of samples that are dynamically moving during acquisition. We implement multi-slice fusion on distributed, heterogeneous clusters in order to reconstruct large 4D volumes in reasonable time and demonstrate the inherent parallelizable nature of the algorithm. We present simulated and real experimental results on sparse-view and limited-angle CT data to demonstrate that multi-slice fusion can substantially improve the quality of reconstructions relative to traditional methods, while also being practical to implement and train. △ Less

Submitted 19 February, 2021; v1 submitted 31 July, 2020; originally announced August 2020.

Comments: arXiv admin note: substantial text overlap with arXiv:1906.06601

COVID-19 infectivity profile correction

Authors: Peter Ashcroft, Jana S. Huisman, Sonja Lehtinen, Judith A. Bouman, Christian L. Althaus, Roland R. Regoes, Sebastian Bonhoeffer

Abstract: The infectivity profile of an individual with COVID-19 is attributed to the paper Temporal dynamics in viral shedding and transmissibility of COVID-19 by He et al., published in Nature Medicine in April 2020. However, the analysis within this paper contains a mistake such that the published infectivity profile is incorrect and the conclusion that infectiousness begins 2.3 days before symptom onset… ▽ More The infectivity profile of an individual with COVID-19 is attributed to the paper Temporal dynamics in viral shedding and transmissibility of COVID-19 by He et al., published in Nature Medicine in April 2020. However, the analysis within this paper contains a mistake such that the published infectivity profile is incorrect and the conclusion that infectiousness begins 2.3 days before symptom onset is no longer supported. In this document we discuss the error and compute the correct infectivity profile. We also establish confidence intervals on this profile, quantify the difference between the published and the corrected profiles, and discuss an issue of normalisation when fitting serial interval data. This infectivity profile plays a central role in policy and decision making, thus it is crucial that this issue is corrected with the utmost urgency to prevent the propagation of this error into further studies and policies. We hope that this preprint will reach all researchers and policy makers who are using the incorrect infectivity profile to inform their work. △ Less

Submitted 13 July, 2020; originally announced July 2020.

Comments: 5 pages, 2 figures

Physics-Based Iterative Reconstruction for Dual Source and Flying Focal Spot Computed Tomography

Authors: Xiao Wang, Robert D. MacDougall, Peng Chen, Charles A. Bouman, Simon K. Warfield

Abstract: For single source helical Computed Tomography (CT), both Filtered-Back Projection (FBP) and statistical iterative reconstruction have been investigated. However for dual source CT with flying focal spot (DS-FFS CT), statistical iterative reconstruction that accurately models the scanner geometry and physics remains unknown to researchers. Therefore, this paper presents a novel physics-based iterat… ▽ More For single source helical Computed Tomography (CT), both Filtered-Back Projection (FBP) and statistical iterative reconstruction have been investigated. However for dual source CT with flying focal spot (DS-FFS CT), statistical iterative reconstruction that accurately models the scanner geometry and physics remains unknown to researchers. Therefore, this paper presents a novel physics-based iterative reconstruction method for DS-FFS CT and assess its image quality. Our algorithm uses precise physics models to reconstruct from the native cone-beam geometry and interleaved dual source helical trajectory of a DS-FFS CT. To do so, we construct a noise physics model to represent data acquisition noise and a prior image model to represent image noise and texture. In addition, we design forward system models to compute the locations of deflected focal spots, the dimension and sensitivity of voxels and detector units, as well as the length of intersection between X-rays and voxels. The forward system models further represent the coordinated movement between the dual sources by computing their X-ray coverage gaps and overlaps at an arbitrary helical pitch. With the above models, we reconstruct images by using an advanced Consensus Equilibrium (CE) numerical method to compute the maximum a posteriori estimate to a joint optimization problem that simultaneously fits all models. We compared our reconstruction with Siemens ADMIRE, which is the clinical standard hybrid iterative reconstruction (IR) method for DS-FFS CT, in terms of spatial resolution, noise profile and image artifacts through both phantoms and clinical datasets. Experiments show that our reconstruction has a consistently higher spatial resolution than the clinical standard hybrid IR. In addition, our reconstruction shows a reduced magnitude of image undersampling artifacts than the clinical standard. △ Less

Submitted 25 April, 2021; v1 submitted 26 January, 2020; originally announced January 2020.

Journal ref: Medical Physics 2021

Design and Autonomous Stabilization of a Ballistically Launched Multirotor

Authors: Amanda Bouman, Paul Nadan, Matthew Anderson, Daniel Pastor, Jacob Izraelevitz, Joel Burdick, Brett Kennedy

Abstract: Aircraft that can launch ballistically and convert to autonomous, free flying drones have applications in many areas such as emergency response, defense, and space exploration, where they can gather critical situational data using onboard sensors. This paper presents a ballistically launched, autonomously stabilizing multirotor prototype (SQUID, Streamlined Quick Unfolding Investigation Drone) wit… ▽ More Aircraft that can launch ballistically and convert to autonomous, free flying drones have applications in many areas such as emergency response, defense, and space exploration, where they can gather critical situational data using onboard sensors. This paper presents a ballistically launched, autonomously stabilizing multirotor prototype (SQUID, Streamlined Quick Unfolding Investigation Drone) with an onboard sensor suite, autonomy pipeline, and passive aerodynamic stability. We demonstrate autonomous transition from passive to vision based, active stabilization, confirming the ability of the multirotor to autonomously stabilize after a ballistic launch in a GPS denied environment. △ Less

Submitted 17 May, 2020; v1 submitted 22 November, 2019; originally announced November 2019.

Comments: Accepted to 2020 International Conference on Robotics and Automation

Distributed Iterative CT Reconstruction using Multi-Agent Consensus Equilibrium

Authors: Venkatesh Sridhar, Xiao Wang, Gregery T. Buzzard, Charles A. Bouman

Abstract: Model-Based Image Reconstruction (MBIR) methods significantly enhance the quality of computed tomographic (CT) reconstructions relative to analytical techniques, but are limited by high computational cost. In this paper, we propose a multi-agent consensus equilibrium (MACE) algorithm for distributing both the computation and memory of MBIR reconstruction across a large number of parallel nodes. In… ▽ More Model-Based Image Reconstruction (MBIR) methods significantly enhance the quality of computed tomographic (CT) reconstructions relative to analytical techniques, but are limited by high computational cost. In this paper, we propose a multi-agent consensus equilibrium (MACE) algorithm for distributing both the computation and memory of MBIR reconstruction across a large number of parallel nodes. In MACE, each node stores only a sparse subset of views and a small portion of the system matrix, and each parallel node performs a local sparse-view reconstruction, which based on repeated feedback from other nodes, converges to the global optimum. Our distributed approach can also incorporate advanced denoisers as priors to enhance reconstruction quality. In this case, we obtain a parallel solution to the serial framework of Plug-n-play (PnP) priors, which we call MACE-PnP. In order to make MACE practical, we introduce a partial update method that eliminates nested iterations and prove that it converges to the same global solution. Finally, we validate our approach on a distributed memory system with real CT data. We also demonstrate an implementation of our approach on a massive supercomputer that can perform large-scale reconstruction in real-time. △ Less

Submitted 20 November, 2019; originally announced November 2019.

Design of a Ballistically-Launched Foldable Multirotor

Authors: Daniel Pastor, Jacob Izraelevitz, Paul Nadan, Amanda Bouman, Joel Burdick, Brett Kennedy

Abstract: The operation of multirotors in crowded environments requires a highly reliable takeoff method, as failures during takeoff can damage more valuable assets nearby. The addition of a ballistic launch system imposes a deterministic path for the multirotor to prevent collisions with its environment, as well as increases the multirotor's range of operation and allows deployment from an unsteady platfor… ▽ More The operation of multirotors in crowded environments requires a highly reliable takeoff method, as failures during takeoff can damage more valuable assets nearby. The addition of a ballistic launch system imposes a deterministic path for the multirotor to prevent collisions with its environment, as well as increases the multirotor's range of operation and allows deployment from an unsteady platform. In addition, outfitting planetary rovers or entry vehicles with such deployable multirotors has the potential to greatly extend the data collection capabilities of a mission. A proof-of-concept multirotor aircraft has been developed, capable of transitioning from a ballistic launch configuration to a fully controllable flight configuration in midair after launch. The transition is accomplished via passive unfolding of the multirotor arms, triggered by a nichrome burn wire release mechanism. The design is 3D printable, launches from a three-inch diameter barrel, and has sufficient thrust to carry a significant payload. The system has been fabricated and field tested from a moving vehicle up to 50mph to successfully demonstrate the feasibility of the concept and experimentally validate the design's aerodynamic stability and deployment reliability. △ Less

Submitted 13 November, 2019; originally announced November 2019.

Comments: IROS 2019

Journal ref: Conference: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

4D X-Ray CT Reconstruction using Multi-Slice Fusion

Authors: Soumendu Majee, Thilo Balke, Craig A. J. Kemp, Gregery T. Buzzard, Charles A. Bouman

Abstract: There is an increasing need to reconstruct objects in four or more dimensions corresponding to space, time and other independent parameters. The best 4D reconstruction algorithms use regularized iterative reconstruction approaches such as model based iterative reconstruction (MBIR), which depends critically on the quality of the prior modeling. Recently, Plug-and-Play methods have been shown to be… ▽ More There is an increasing need to reconstruct objects in four or more dimensions corresponding to space, time and other independent parameters. The best 4D reconstruction algorithms use regularized iterative reconstruction approaches such as model based iterative reconstruction (MBIR), which depends critically on the quality of the prior modeling. Recently, Plug-and-Play methods have been shown to be an effective way to incorporate advanced prior models using state-of-the-art denoising algorithms designed to remove additive white Gaussian noise (AWGN). However, state-of-the-art denoising algorithms such as BM4D and deep convolutional neural networks (CNNs) are primarily available for 2D and sometimes 3D images. In particular, CNNs are difficult and computationally expensive to implement in four or more dimensions, and training may be impossible if there is no associated high-dimensional training data. In this paper, we present Multi-Slice Fusion, a novel algorithm for 4D and higher-dimensional reconstruction, based on the fusion of multiple low-dimensional denoisers. Our approach uses multi-agent consensus equilibrium (MACE), an extension of Plug-and-Play, as a framework for integrating the multiple lower-dimensional prior models. We apply our method to the problem of 4D cone-beam X-ray CT reconstruction for Non Destructive Evaluation (NDE) of moving parts. This is done by solving the MACE equations using lower-dimensional CNN denoisers implemented in parallel on a heterogeneous cluster. Results on experimental CT data demonstrate that Multi-Slice Fusion can substantially improve the quality of reconstructions relative to traditional 4D priors, while also being practical to implement and train. △ Less

Submitted 15 June, 2019; originally announced June 2019.

Comments: 8 pages, 8 figures, IEEE International Conference on Computational Photography 2019, Tokyo

Plug and play methods for magnetic resonance imaging (long version)

Authors: Rizwan Ahmad, Charles A. Bouman, Gregery T. Buzzard, Stanley Chan, Sizhou Liu, Edward T. Reehorst, Philip Schniter

Abstract: Magnetic Resonance Imaging (MRI) is a non-invasive diagnostic tool that provides excellent soft-tissue contrast without the use of ionizing radiation. Compared to other clinical imaging modalities (e.g., CT or ultrasound), however, the data acquisition process for MRI is inherently slow, which motivates undersampling and thus drives the need for accurate, efficient reconstruction methods from unde… ▽ More Magnetic Resonance Imaging (MRI) is a non-invasive diagnostic tool that provides excellent soft-tissue contrast without the use of ionizing radiation. Compared to other clinical imaging modalities (e.g., CT or ultrasound), however, the data acquisition process for MRI is inherently slow, which motivates undersampling and thus drives the need for accurate, efficient reconstruction methods from undersampled datasets. In this article, we describe the use of "plug-and-play" (PnP) algorithms for MRI image recovery. We first describe the linearly approximated inverse problem encountered in MRI. Then we review several PnP methods, where the unifying commonality is to iteratively call a denoising subroutine as one step of a larger optimization-inspired algorithm. Next, we describe how the result of the PnP method can be interpreted as a solution to an equilibrium equation, allowing convergence analysis from the equilibrium perspective. Finally, we present illustrative examples of PnP methods applied to MRI image recovery. △ Less

Submitted 10 December, 2019; v1 submitted 20 March, 2019; originally announced March 2019.

2.5D Deep Learning for CT Image Reconstruction using a Multi-GPU implementation

Authors: Amirkoushyar Ziabari, Dong Hye Ye, Somesh Srivastava, Ken D. Sauer, Jean-Baptiste Thibault, Charles A. Bouman

Abstract: While Model Based Iterative Reconstruction (MBIR) of CT scans has been shown to have better image quality than Filtered Back Projection (FBP), its use has been limited by its high computational cost. More recently, deep convolutional neural networks (CNN) have shown great promise in both denoising and reconstruction applications. In this research, we propose a fast reconstruction algorithm, which… ▽ More While Model Based Iterative Reconstruction (MBIR) of CT scans has been shown to have better image quality than Filtered Back Projection (FBP), its use has been limited by its high computational cost. More recently, deep convolutional neural networks (CNN) have shown great promise in both denoising and reconstruction applications. In this research, we propose a fast reconstruction algorithm, which we call Deep Learning MBIR (DL-MBIR), for approximating MBIR using a deep residual neural network. The DL-MBIR method is trained to produce reconstructions that approximate true MBIR images using a 16 layer residual convolutional neural network implemented on multiple GPUs using Google Tensorflow. In addition, we propose 2D, 2.5D and 3D variations on the DL-MBIR method and show that the 2.5D method achieves similar quality to the fully 3D method, but with reduced computational cost. △ Less

Submitted 20 December, 2018; originally announced December 2018.

Comments: IEEE Asilomar conference on signals systems and computers, 2018

Model Based Iterative Reconstruction With Spatially Adaptive Sinogram Weights for Wide-Cone Cardiac CT

Authors: Amirkoushyar Ziabari, Dong Hye Ye, Lin Fu, Somesh Srivastava, Ken D. Sauer, Jean-Baptist Thibault, Charles A. Bouman

Abstract: With the recent introduction of CT scanners with large cone angles, wide coverage detectors now provide a desirable scanning platform for cardiac CT that allows whole heart imaging in a single rotation. On these scanners, while half-scan data is strictly sufficient to produce images with the best temporal resolution, acquiring a full 360 degree rotation worth of data is beneficial for wide-cone im… ▽ More With the recent introduction of CT scanners with large cone angles, wide coverage detectors now provide a desirable scanning platform for cardiac CT that allows whole heart imaging in a single rotation. On these scanners, while half-scan data is strictly sufficient to produce images with the best temporal resolution, acquiring a full 360 degree rotation worth of data is beneficial for wide-cone image reconstruction at negligible additional radiation dose. Applying Model-Based Iterative Reconstruction (MBIR) algorithm to the heart has shown to yield significant enhancement in image quality for cardiac CT. But imaging the heart in large cone angle geometry leads to apparently conflicting data usage considerations. On the one hand, in addition to using the fastest available scanner rotation speed, a minimal complete data set of 180 degrees plus the fan angle is typically used to minimize both cardiac and respiratory motion. On the other hand, a full 360 degree acquisition helps better handle the challenges of missing frequencies and incomplete projections associated with wide-cone half-scan data acquisition. In this paper, we develop a Spatially Adaptive sinogram Weights MBIR algorithm (SAW-MBIR) that is designed to achieve the benefits of both half and full-scan reconstructions in order to maximize temporal resolution over the heart region while providing stable results over the whole volume covered with the wide-area detector. Spatially-adaptive sinogram weights applied to each projection measurement in SAW-MBIR are designed to selectively perform backprojection from the full and half-scan portion of the sinogram based on both projection angle and reconstructed voxel location. We demonstrate with experimental results of SAW-MBIR applied to whole-heart cardiac CT clinical data that overall temporal resolution matches half-scan while full volume image quality is on par with full-scan MBIR. △ Less

Submitted 20 December, 2018; originally announced December 2018.

Comments: The 5th international Conference on image formation in X-ray Computed Tomography (Proceedings of CT Meeting). Compared to original publication, we slightly modified figure 4 for better clarity

Deep Back Projection for Sparse-View CT Reconstruction

Authors: Dong Hye Ye, Gregery T. Buzzard, Max Ruby, Charles A. Bouman

Abstract: Filtered back projection (FBP) is a classical method for image reconstruction from sinogram CT data. FBP is computationally efficient but produces lower quality reconstructions than more sophisticated iterative methods, particularly when the number of views is lower than the number required by the Nyquist rate. In this paper, we use a deep convolutional neural network (CNN) to produce high-quality… ▽ More Filtered back projection (FBP) is a classical method for image reconstruction from sinogram CT data. FBP is computationally efficient but produces lower quality reconstructions than more sophisticated iterative methods, particularly when the number of views is lower than the number required by the Nyquist rate. In this paper, we use a deep convolutional neural network (CNN) to produce high-quality reconstructions directly from sinogram data. A primary novelty of our approach is that we first back project each view separately to form a stack of back projections and then feed this stack as input into the convolutional neural network. These single-view back projections convert the encoding of sinogram data into the appropriate spatial location, which can then be leveraged by the spatial invariance of the CNN to learn the reconstruction effectively. We demonstrate the benefit of our CNN based back projection on simulated sparse-view CT data over classical FBP. △ Less

Submitted 6 July, 2018; originally announced July 2018.

Comments: GlobalSIP 2018

Deep neural networks for non-linear model-based ultrasound reconstruction

Authors: Hani Almansouri, S. V. Venkatakrishnan, Gregery T. Buzzard, Charles A. Bouman, Hector Santos-Villalobos

Abstract: Ultrasound reflection tomography is widely used to image large complex specimens that are only accessible from a single side, such as well systems and nuclear power plant containment walls. Typical methods for inverting the measurement rely on delay-and-sum algorithms that rapidly produce reconstructions but with significant artifacts. Recently, model-based reconstruction approaches using a linear… ▽ More Ultrasound reflection tomography is widely used to image large complex specimens that are only accessible from a single side, such as well systems and nuclear power plant containment walls. Typical methods for inverting the measurement rely on delay-and-sum algorithms that rapidly produce reconstructions but with significant artifacts. Recently, model-based reconstruction approaches using a linear forward model have been shown to significantly improve image quality compared to the conventional approach. However, even these techniques result in artifacts for complex objects because of the inherent non-linearity of the ultrasound forward model. In this paper, we propose a non-iterative model-based reconstruction method for inverting measurements that are based on non-linear forward models for ultrasound imaging. Our approach involves obtaining an approximate estimate of the reconstruction using a simple linear back-projection and training a deep neural network to refine this to the actual reconstruction. We apply our method to simulated ultrasound data and demonstrate dramatic improvements in image quality compared to the delay-and-sum approach and the linear model-based reconstruction approach. △ Less

Submitted 28 September, 2018; v1 submitted 3 July, 2018; originally announced July 2018.

SLADS-Net: Supervised Learning Approach for Dynamic Sampling using Deep Neural Networks

Authors: Yan Zhang, G. M. Dilshan Godaliyadda, Nicola Ferrier, Emine B. Gulsoy, Charles A. Bouman, Charudatta Phatak

Abstract: In scanning microscopy based imaging techniques, there is a need to develop novel data acquisition schemes that can reduce the time for data acquisition and minimize sample exposure to the probing radiation. Sparse sampling schemes are ideally suited for such applications where the images can be reconstructed from a sparse set of measurements. In particular, dynamic sparse sampling based on superv… ▽ More In scanning microscopy based imaging techniques, there is a need to develop novel data acquisition schemes that can reduce the time for data acquisition and minimize sample exposure to the probing radiation. Sparse sampling schemes are ideally suited for such applications where the images can be reconstructed from a sparse set of measurements. In particular, dynamic sparse sampling based on supervised learning has shown promising results for practical applications. However, a particular drawback of such methods is that it requires training image sets with similar information content which may not always be available. In this paper, we introduce a Supervised Learning Approach for Dynamic Sampling (SLADS) algorithm that uses a deep neural network based training approach. We call this algorithm SLADS- Net. We have performed simulated experiments for dynamic sampling using SLADS-Net in which the training images either have similar information content or completely different information content, when compared to the testing images. We compare the performance across various methods for training such as least- squares, support vector regression and deep neural networks. From these results we observe that deep neural network based training results in superior performance when the training and testing images are not similar. We also discuss the development of a pre-trained SLADS-Net that uses generic images for training. Here, the neural network parameters are pre-trained so that users can directly apply SLADS-Net for imaging experiments. △ Less

Submitted 8 March, 2018; originally announced March 2018.

Comments: 6 pages, 8 figures, Electronic Imaging 2018

Phase-error estimation and image reconstruction from digital-holography data using a Bayesian framework

Authors: Casey J. Pellizzari, Mark F. Spencer, Charles A. Bouman

Abstract: The estimation of phase errors from digital-holography data is critical for applications such as imaging or wave-front sensing. Conventional techniques require multiple i.i.d. data and perform poorly in the presence of high noise or large phase errors. In this paper we propose a method to estimate isoplanatic phase errors from a single data realization. We develop a model-based iterative reconstru… ▽ More The estimation of phase errors from digital-holography data is critical for applications such as imaging or wave-front sensing. Conventional techniques require multiple i.i.d. data and perform poorly in the presence of high noise or large phase errors. In this paper we propose a method to estimate isoplanatic phase errors from a single data realization. We develop a model-based iterative reconstruction algorithm which computes the maximum a posteriori estimate of the phase and the speckle-free object reflectance. Using simulated data, we show that the algorithm is robust against high noise and strong phase errors. △ Less

Submitted 9 June, 2017; originally announced August 2017.

Comments: 10 pages, 8 figures

Reduced Electron Exposure for Energy-Dispersive Spectroscopy using Dynamic Sampling

Authors: Yan Zhang, G. M. Dilshan Godaliyadda, Nicola Ferrier, Emine B. Gulsoy, Charles A. Bouman, Charudatta Phatak

Abstract: Analytical electron microscopy and spectroscopy of biological specimens, polymers, and other beam sensitive materials has been a challenging area due to irradiation damage. There is a pressing need to develop novel imaging and spectroscopic imaging methods that will minimize such sample damage as well as reduce the data acquisition time. The latter is useful for high-throughput analysis of materia… ▽ More Analytical electron microscopy and spectroscopy of biological specimens, polymers, and other beam sensitive materials has been a challenging area due to irradiation damage. There is a pressing need to develop novel imaging and spectroscopic imaging methods that will minimize such sample damage as well as reduce the data acquisition time. The latter is useful for high-throughput analysis of materials structure and chemistry. In this work, we present a novel machine learning based method for dynamic sparse sampling of EDS data using a scanning electron microscope. Our method, based on the supervised learning approach for dynamic sampling algorithm and neural networks based classification of EDS data, allows a dramatic reduction in the total sampling of up to 90%, while maintaining the fidelity of the reconstructed elemental maps and spectroscopic data. We believe this approach will enable imaging and elemental map** of materials that would otherwise be inaccessible to these analysis techniques. △ Less

Submitted 27 June, 2017; originally announced July 2017.

Comments: 22 pages, 8 figures

Plug-and-Play Unplugged: Optimization Free Reconstruction using Consensus Equilibrium

Authors: Gregery T. Buzzard, Stanley H. Chan, Suhas Sreehari, Charles A. Bouman

Abstract: Regularized inversion methods for image reconstruction are used widely due to their tractability and ability to combine complex physical sensor models with useful regularity criteria. Such methods motivated the recently developed Plug-and-Play prior method, which provides a framework to use advanced denoising algorithms as regularizers in inversion. However, the need to formulate regularized inver… ▽ More Regularized inversion methods for image reconstruction are used widely due to their tractability and ability to combine complex physical sensor models with useful regularity criteria. Such methods motivated the recently developed Plug-and-Play prior method, which provides a framework to use advanced denoising algorithms as regularizers in inversion. However, the need to formulate regularized inversion as the solution to an optimization problem limits the possible regularity conditions and physical sensor models. In this paper, we introduce Consensus Equilibrium (CE), which generalizes regularized inversion to include a much wider variety of both forward components and prior components without the need for either to be expressed with a cost function. CE is based on the solution of a set of equilibrium equations that balance data fit and regularity. In this framework, the problem of MAP estimation in regularized inversion is replaced by the problem of solving these equilibrium equations, which can be approached in multiple ways. The key contribution of CE is to provide a novel framework for fusing multiple heterogeneous models of physical sensors or models learned from data. We describe the derivation of the CE equations and prove that the solution of the CE equations generalizes the standard MAP estimate under appropriate circumstances. We also discuss algorithms for solving the CE equations, including ADMM with a novel form of preconditioning and Newton's method. We give examples to illustrate consensus equilibrium and the convergence properties of these algorithms and demonstrate this method on some toy problems and on a denoising example in which we use an array of convolutional neural network denoisers, none of which is tuned to match the noise level in a noisy image but which in consensus can achieve a better result than any of them individually. △ Less

Submitted 13 June, 2018; v1 submitted 24 May, 2017; originally announced May 2017.

Comments: 17 pages

MSC Class: 94A08; 68U10