-
Supervised Reconstruction for Silhouette Tomography
Authors:
Evan Bell,
Michael T. McCann,
Marc Klasky
Abstract:
In this paper, we introduce silhouette tomography, a novel formulation of X-ray computed tomography that relies only on the geometry of the imaging system. We formulate silhouette tomography mathematically and provide a simple method for obtaining a particular solution to the problem, assuming that any solution exists. We then propose a supervised reconstruction approach that uses a deep neural ne…
▽ More
In this paper, we introduce silhouette tomography, a novel formulation of X-ray computed tomography that relies only on the geometry of the imaging system. We formulate silhouette tomography mathematically and provide a simple method for obtaining a particular solution to the problem, assuming that any solution exists. We then propose a supervised reconstruction approach that uses a deep neural network to solve the silhouette tomography problem. We present experimental results on a synthetic dataset that demonstrate the effectiveness of the proposed method.
△ Less
Submitted 11 February, 2024;
originally announced February 2024.
-
PtychoDV: Vision Transformer-Based Deep Unrolling Network for Ptychographic Image Reconstruction
Authors:
Weijie Gan,
Qiuchen Zhai,
Michael Thompson McCann,
Cristina Garcia Cardona,
Ulugbek S. Kamilov,
Brendt Wohlberg
Abstract:
Ptychography is an imaging technique that captures multiple overlap** snapshots of a sample, illuminated coherently by a moving localized probe. The image recovery from ptychographic data is generally achieved via an iterative algorithm that solves a nonlinear phase retrieval problem derived from measured diffraction patterns. However, these iterative approaches have high computational cost. In…
▽ More
Ptychography is an imaging technique that captures multiple overlap** snapshots of a sample, illuminated coherently by a moving localized probe. The image recovery from ptychographic data is generally achieved via an iterative algorithm that solves a nonlinear phase retrieval problem derived from measured diffraction patterns. However, these iterative approaches have high computational cost. In this paper, we introduce PtychoDV, a novel deep model-based network designed for efficient, high-quality ptychographic image reconstruction. PtychoDV comprises a vision transformer that generates an initial image from the set of raw measurements, taking into consideration their mutual correlations. This is followed by a deep unrolling network that refines the initial image using learnable convolutional priors and the ptychography measurement model. Experimental results on simulated data demonstrate that PtychoDV is capable of outperforming existing deep learning methods for this problem, and significantly reduces computational cost compared to iterative methodologies, while maintaining competitive performance.
△ Less
Submitted 6 March, 2024; v1 submitted 11 October, 2023;
originally announced October 2023.
-
Score-based Diffusion Models for Bayesian Image Reconstruction
Authors:
Michael T. McCann,
Hyung** Chung,
Jong Chul Ye,
Marc L. Klasky
Abstract:
This paper explores the use of score-based diffusion models for Bayesian image reconstruction. Diffusion models are an efficient tool for generative modeling. Diffusion models can also be used for solving image reconstruction problems. We present a simple and flexible algorithm for training a diffusion model and using it for maximum a posteriori reconstruction, minimum mean square error reconstruc…
▽ More
This paper explores the use of score-based diffusion models for Bayesian image reconstruction. Diffusion models are an efficient tool for generative modeling. Diffusion models can also be used for solving image reconstruction problems. We present a simple and flexible algorithm for training a diffusion model and using it for maximum a posteriori reconstruction, minimum mean square error reconstruction, and posterior sampling. We present experiments on both a linear and a nonlinear reconstruction problem that highlight the strengths and limitations of the approach.
△ Less
Submitted 25 May, 2023;
originally announced May 2023.
-
Material Identification From Radiographs Without Energy Resolution
Authors:
Michael T. McCann,
Elena Guardincerri,
Samuel M. Gonzales,
Lauren A. Misurek,
Jennifer L. Schei,
Marc L. Klasky
Abstract:
We propose a method for performing material identification from radiographs without energy-resolved measurements. Material identification has a wide variety of applications, including in biomedical imaging, nondestructive testing, and security. While existing techniques for radiographic material identification make use of dual energy sources, energy-resolving detectors, or additional (e.g., neutro…
▽ More
We propose a method for performing material identification from radiographs without energy-resolved measurements. Material identification has a wide variety of applications, including in biomedical imaging, nondestructive testing, and security. While existing techniques for radiographic material identification make use of dual energy sources, energy-resolving detectors, or additional (e.g., neutron) measurements, such setups are not always practical-requiring additional hardware and complicating imaging. We tackle material identification without energy resolution, allowing standard X-ray systems to provide material identification information without requiring additional hardware. Assuming a setting where the geometry of each object in the scene is known and the materials come from a known set of possible materials, we pose the problem as a combinatorial optimization with a loss function that accounts for the presence of scatter and an unknown gain and propose a branch and bound algorithm to efficiently solve it. We present experiments on both synthetic data and real, experimental data with relevance to security applications-thick, dense objects imaged with MeV X-rays. We show that material identification can be efficient and accurate, for example, in a scene with three shells (two copper, one aluminum), our algorithm ran in six minutes on a consumer-level laptop and identified the correct materials as being among the top 10 best matches out of 8,000 possibilities.
△ Less
Submitted 10 March, 2023;
originally announced March 2023.
-
Solving 3D Inverse Problems using Pre-trained 2D Diffusion Models
Authors:
Hyung** Chung,
Dohoon Ryu,
Michael T. McCann,
Marc L. Klasky,
Jong Chul Ye
Abstract:
Diffusion models have emerged as the new state-of-the-art generative model with high quality samples, with intriguing properties such as mode coverage and high flexibility. They have also been shown to be effective inverse problem solvers, acting as the prior of the distribution, while the information of the forward model can be granted at the sampling stage. Nonetheless, as the generative process…
▽ More
Diffusion models have emerged as the new state-of-the-art generative model with high quality samples, with intriguing properties such as mode coverage and high flexibility. They have also been shown to be effective inverse problem solvers, acting as the prior of the distribution, while the information of the forward model can be granted at the sampling stage. Nonetheless, as the generative process remains in the same high dimensional (i.e. identical to data dimension) space, the models have not been extended to 3D inverse problems due to the extremely high memory and computational cost. In this paper, we combine the ideas from the conventional model-based iterative reconstruction with the modern diffusion models, which leads to a highly effective method for solving 3D medical image reconstruction tasks such as sparse-view tomography, limited angle tomography, compressed sensing MRI from pre-trained 2D diffusion models. In essence, we propose to augment the 2D diffusion prior with a model-based prior in the remaining direction at test time, such that one can achieve coherent reconstructions across all dimensions. Our method can be run in a single commodity GPU, and establishes the new state-of-the-art, showing that the proposed method can perform reconstructions of high fidelity and accuracy even in the most extreme cases (e.g. 2-view 3D tomography). We further reveal that the generalization capacity of the proposed method is surprisingly high, and can be used to reconstruct volumes that are entirely different from the training dataset.
△ Less
Submitted 19 November, 2022;
originally announced November 2022.
-
Diffusion Posterior Sampling for General Noisy Inverse Problems
Authors:
Hyung** Chung,
Jeongsol Kim,
Michael T. Mccann,
Marc L. Klasky,
Jong Chul Ye
Abstract:
Diffusion models have been recently studied as powerful generative inverse problem solvers, owing to their high quality reconstructions and the ease of combining existing iterative solvers. However, most works focus on solving simple linear inverse problems in noiseless settings, which significantly under-represents the complexity of real-world problems. In this work, we extend diffusion solvers t…
▽ More
Diffusion models have been recently studied as powerful generative inverse problem solvers, owing to their high quality reconstructions and the ease of combining existing iterative solvers. However, most works focus on solving simple linear inverse problems in noiseless settings, which significantly under-represents the complexity of real-world problems. In this work, we extend diffusion solvers to efficiently handle general noisy (non)linear inverse problems via approximation of the posterior sampling. Interestingly, the resulting posterior sampling scheme is a blended version of diffusion sampling with the manifold constrained gradient without a strict measurement consistency projection step, yielding a more desirable generative path in noisy settings compared to the previous studies. Our method demonstrates that diffusion models can incorporate various measurement noise statistics such as Gaussian and Poisson, and also efficiently handle noisy nonlinear inverse problems such as Fourier phase retrieval and non-uniform deblurring. Code available at https://github.com/DPS2022/diffusion-posterior-sampling
△ Less
Submitted 20 May, 2024; v1 submitted 29 September, 2022;
originally announced September 2022.
-
Learning Sparsity-Promoting Regularizers using Bilevel Optimization
Authors:
Avrajit Ghosh,
Michael T. McCann,
Madeline Mitchell,
Saiprasad Ravishankar
Abstract:
We present a method for supervised learning of sparsity-promoting regularizers for denoising signals and images. Sparsity-promoting regularization is a key ingredient in solving modern signal reconstruction problems; however, the operators underlying these regularizers are usually either designed by hand or learned from data in an unsupervised way. The recent success of supervised learning (mainly…
▽ More
We present a method for supervised learning of sparsity-promoting regularizers for denoising signals and images. Sparsity-promoting regularization is a key ingredient in solving modern signal reconstruction problems; however, the operators underlying these regularizers are usually either designed by hand or learned from data in an unsupervised way. The recent success of supervised learning (mainly convolutional neural networks) in solving image reconstruction problems suggests that it could be a fruitful approach to designing regularizers. Towards this end, we propose to denoise signals using a variational formulation with a parametric, sparsity-promoting regularizer, where the parameters of the regularizer are learned to minimize the mean squared error of reconstructions on a training set of ground truth image and measurement pairs. Training involves solving a challenging bilievel optimization problem; we derive an expression for the gradient of the training loss using the closed-form solution of the denoising problem and provide an accompanying gradient descent algorithm to minimize it. Our experiments with structured 1D signals and natural images show that the proposed method can learn an operator that outperforms well-known regularizers (total variation, DCT-sparsity, and unsupervised dictionary learning) and collaborative filtering for denoising. While the approach we present is specific to denoising, we believe that it could be adapted to the larger class of inverse problems with linear measurement models, giving it applicability in a wide range of signal reconstruction settings.
△ Less
Submitted 5 September, 2023; v1 submitted 18 July, 2022;
originally announced July 2022.
-
High-Precision Inversion of Dynamic Radiography Using Hydrodynamic Features
Authors:
Maliha Hossain,
Balasubramanya T. Nadiga,
Oleg Korobkin,
Marc L. Klasky,
Jennifer L. Schei,
Joshua W. Burby,
Michael T. McCann,
Trevor Wilcox,
Soumi De,
Charles A. Bouman
Abstract:
Radiography is often used to probe complex, evolving density fields in dynamic systems and in so doing gain insight into the underlying physics. This technique has been used in numerous fields including materials science, shock physics, inertial confinement fusion, and other national security applications. In many of these applications, however, complications resulting from noise, scatter, complex…
▽ More
Radiography is often used to probe complex, evolving density fields in dynamic systems and in so doing gain insight into the underlying physics. This technique has been used in numerous fields including materials science, shock physics, inertial confinement fusion, and other national security applications. In many of these applications, however, complications resulting from noise, scatter, complex beam dynamics, etc. prevent the reconstruction of density from being accurate enough to identify the underlying physics with sufficient confidence. As such, density reconstruction from static/dynamic radiography has typically been limited to identifying discontinuous features such as cracks and voids in a number of these applications.
In this work, we propose a fundamentally new approach to reconstructing density from a temporal sequence of radiographic images. Using only the robust features identifiable in radiographs, we combine them with the underlying hydrodynamic equations of motion using a machine learning approach, namely, conditional generative adversarial networks (cGAN), to determine the density fields from a dynamic sequence of radiographs. Next, we seek to further enhance the hydrodynamic consistency of the ML-based density reconstruction through a process of parameter estimation and projection onto a hydrodynamic manifold. In this context, we note that the distance from the hydrodynamic manifold given by the training data to the test data in the parameter space considered both serves as a diagnostic of the robustness of the predictions and serves to augment the training database, with the expectation that the latter will further reduce future density reconstruction errors. Finally, we demonstrate the ability of this method to outperform a traditional radiographic reconstruction in capturing allowable hydrodynamic paths even when relatively small amounts of scatter are present.
△ Less
Submitted 2 December, 2021;
originally announced December 2021.
-
Bilevel learning of l1-regularizers with closed-form gradients(BLORC)
Authors:
Avrajit Ghosh,
Michael T. Mccann,
Saiprasad Ravishankar
Abstract:
We present a method for supervised learning of sparsity-promoting regularizers, a key ingredient in many modern signal reconstruction problems. The parameters of the regularizer are learned to minimize the mean squared error of reconstruction on a training set of ground truth signal and measurement pairs. Training involves solving a challenging bilevel optimization problem with a nonsmooth lower-l…
▽ More
We present a method for supervised learning of sparsity-promoting regularizers, a key ingredient in many modern signal reconstruction problems. The parameters of the regularizer are learned to minimize the mean squared error of reconstruction on a training set of ground truth signal and measurement pairs. Training involves solving a challenging bilevel optimization problem with a nonsmooth lower-level objective. We derive an expression for the gradient of the training loss using the implicit closed-form solution of the lower-level variational problem given by its dual problem, and provide an accompanying gradient descent algorithm (dubbed BLORC) to minimize the loss. Our experiments on simple natural images and for denoising 1D signals show that the proposed method can learn meaningful operators and the analytical gradients calculated are faster than standard automatic differentiation methods. While the approach we present is applied to denoising, we believe that it can be adapted to a wide-variety of inverse problems with linear measurement models, thus giving it applicability in a wide range of scenarios.
△ Less
Submitted 21 November, 2021;
originally announced November 2021.
-
Comparing One-step and Two-step Scatter Correction and Density Reconstruction in X-ray CT
Authors:
Alexander N. Sietsema,
Michael T. McCann,
Marc L. Klasky,
Saiprasad Ravishankar
Abstract:
In this work, we compare one-step and two-step approaches for X-ray computed tomography (CT) scatter correction and density reconstruction. X-ray CT is an important imaging technique in medical and industrial applications. In many cases, the presence of scattered X-rays leads to loss of contrast and undesirable artifacts in reconstructed images. Many approaches to computationally removing scatter…
▽ More
In this work, we compare one-step and two-step approaches for X-ray computed tomography (CT) scatter correction and density reconstruction. X-ray CT is an important imaging technique in medical and industrial applications. In many cases, the presence of scattered X-rays leads to loss of contrast and undesirable artifacts in reconstructed images. Many approaches to computationally removing scatter treat scatter correction as a preprocessing step that is followed by a reconstruction step. Treating scatter correction and reconstruction jointly as a single, more complicated optimization problem is less studied. It is not clear from the existing literature how these two approaches compare in terms of reconstruction accuracy. In this paper, we compare idealized versions of these two approaches with synthetic experiments. Our results show that the one-step approach can offer improved reconstructions over the two-step approach, although the gap between them is highly object-dependent.
△ Less
Submitted 13 May, 2022; v1 submitted 15 October, 2021;
originally announced October 2021.
-
Model-based Reconstruction with Learning: From Unsupervised to Supervised and Beyond
Authors:
Zhishen Huang,
Siqi Ye,
Michael T. McCann,
Saiprasad Ravishankar
Abstract:
Many techniques have been proposed for image reconstruction in medical imaging that aim to recover high-quality images especially from limited or corrupted measurements. Model-based reconstruction methods have been particularly popular (e.g., in magnetic resonance imaging and tomographic modalities) and exploit models of the imaging system's physics together with statistical models of measurements…
▽ More
Many techniques have been proposed for image reconstruction in medical imaging that aim to recover high-quality images especially from limited or corrupted measurements. Model-based reconstruction methods have been particularly popular (e.g., in magnetic resonance imaging and tomographic modalities) and exploit models of the imaging system's physics together with statistical models of measurements, noise and often relatively simple object priors or regularizers. For example, sparsity or low-rankness based regularizers have been widely used for image reconstruction from limited data such as in compressed sensing. Learning-based approaches for image reconstruction have garnered much attention in recent years and have shown promise across biomedical imaging applications. These methods include synthesis dictionary learning, sparsifying transform learning, and different forms of deep learning involving complex neural networks. We briefly discuss classical model-based reconstruction methods and then review reconstruction methods at the intersection of model-based and learning-based paradigms in detail. This review includes many recent methods based on unsupervised learning, and supervised learning, as well as a framework to combine multiple types of learned models together.
△ Less
Submitted 26 March, 2021;
originally announced March 2021.
-
Local Models for Scatter Estimation and Descattering in Polyenergetic X-Ray Tomography
Authors:
Michael T. McCann,
Marc L. Klasky,
Jennifer L. Schei,
Saiprasad Ravishankar
Abstract:
We propose a new modeling approach for scatter estimation and descattering in polyenergetic X-ray computed tomography (CT) based on fitting models to local neighborhoods of a training set. X-ray CT is widely used in medical and industrial applications. X-ray scatter, if not accounted for during reconstruction, creates a loss of contrast in CT reconstructions and introduces severe artifacts includi…
▽ More
We propose a new modeling approach for scatter estimation and descattering in polyenergetic X-ray computed tomography (CT) based on fitting models to local neighborhoods of a training set. X-ray CT is widely used in medical and industrial applications. X-ray scatter, if not accounted for during reconstruction, creates a loss of contrast in CT reconstructions and introduces severe artifacts including cup**, shading, and streaks. Even when these qualitative artifacts are not apparent, scatter can pose a major obstacle in obtaining quantitatively accurate reconstructions. Our approach to estimating scatter is, first, to generate a training set of 2D radiographs with and without scatter using particle transport simulation software. To estimate scatter for a new radiograph, we adaptively fit a scatter model to a small subset of the training data containing the radiographs most similar to it. We compared local and global (fit on full data sets) versions of several X-ray scatter models, including two from the recent literature, as well as a recent deep learning-based scatter model, in the context of descattering and quantitative density reconstruction of simulated, spherically symmetrical, single-material objects comprising shells of various densities. Our results show that, when applied locally, even simple models provide state-of-the-art descattering, reducing the error in density reconstruction due to scatter by more than half.
△ Less
Submitted 28 September, 2021; v1 submitted 11 December, 2020;
originally announced December 2020.
-
Unified Supervised-Unsupervised (SUPER) Learning for X-ray CT Image Reconstruction
Authors:
Siqi Ye,
Zhipeng Li,
Michael T. McCann,
Yong Long,
Saiprasad Ravishankar
Abstract:
Traditional model-based image reconstruction (MBIR) methods combine forward and noise models with simple object priors. Recent machine learning methods for image reconstruction typically involve supervised learning or unsupervised learning, both of which have their advantages and disadvantages. In this work, we propose a unified supervised-unsupervised (SUPER) learning framework for X-ray computed…
▽ More
Traditional model-based image reconstruction (MBIR) methods combine forward and noise models with simple object priors. Recent machine learning methods for image reconstruction typically involve supervised learning or unsupervised learning, both of which have their advantages and disadvantages. In this work, we propose a unified supervised-unsupervised (SUPER) learning framework for X-ray computed tomography (CT) image reconstruction. The proposed learning formulation combines both unsupervised learning-based priors (or even simple analytical priors) together with (supervised) deep network-based priors in a unified MBIR framework based on a fixed point iteration analysis. The proposed training algorithm is also an approximate scheme for a bilevel supervised training optimization problem, wherein the network-based regularizer in the lower-level MBIR problem is optimized using an upper-level reconstruction loss. The training problem is optimized by alternating between updating the network weights and iteratively updating the reconstructions based on those weights. We demonstrate the learned SUPER models' efficacy for low-dose CT image reconstruction, for which we use the NIH AAPM Mayo Clinic Low Dose CT Grand Challenge dataset for training and testing. In our experiments, we studied different combinations of supervised deep network priors and unsupervised learning-based or analytical priors. Both numerical and visual results show the superiority of the proposed unified SUPER methods over standalone supervised learning-based methods, iterative MBIR methods, and variations of SUPER obtained via ablation studies. We also show that the proposed algorithm converges rapidly in practice.
△ Less
Submitted 8 April, 2021; v1 submitted 6 October, 2020;
originally announced October 2020.
-
Supervised Learning of Sparsity-Promoting Regularizers for Denoising
Authors:
Michael T. McCann,
Saiprasad Ravishankar
Abstract:
We present a method for supervised learning of sparsity-promoting regularizers for image denoising. Sparsity-promoting regularization is a key ingredient in solving modern image reconstruction problems; however, the operators underlying these regularizers are usually either designed by hand or learned from data in an unsupervised way. The recent success of supervised learning (mainly convolutional…
▽ More
We present a method for supervised learning of sparsity-promoting regularizers for image denoising. Sparsity-promoting regularization is a key ingredient in solving modern image reconstruction problems; however, the operators underlying these regularizers are usually either designed by hand or learned from data in an unsupervised way. The recent success of supervised learning (mainly convolutional neural networks) in solving image reconstruction problems suggests that it could be a fruitful approach to designing regularizers. As a first experiment in this direction, we propose to denoise images using a variational formulation with a parametric, sparsity-promoting regularizer, where the parameters of the regularizer are learned to minimize the mean squared error of reconstructions on a training set of (ground truth image, measurement) pairs. Training involves solving a challenging bilievel optimization problem; we derive an expression for the gradient of the training loss using Karush-Kuhn-Tucker conditions and provide an accompanying gradient descent algorithm to minimize it. Our experiments on a simple synthetic, denoising problem show that the proposed method can learn an operator that outperforms well-known regularizers (total variation, DCT-sparsity, and unsupervised dictionary learning) and collaborative filtering. While the approach we present is specific to denoising, we believe that it can be adapted to the whole class of inverse problems with linear measurement models, giving it applicability to a wide range of image reconstruction problems.
△ Less
Submitted 9 June, 2020;
originally announced June 2020.
-
Biomedical Image Reconstruction: From the Foundations to Deep Neural Networks
Authors:
Michael T. McCann,
Michael Unser
Abstract:
This tutorial covers biomedical image reconstruction, from the foundational concepts of system modeling and direct reconstruction to modern sparsity and learning-based approaches.
Imaging is a critical tool in biological research and medicine, and most imaging systems necessarily use an image-reconstruction algorithm to create an image; the design of these algorithms has been a topic of research…
▽ More
This tutorial covers biomedical image reconstruction, from the foundational concepts of system modeling and direct reconstruction to modern sparsity and learning-based approaches.
Imaging is a critical tool in biological research and medicine, and most imaging systems necessarily use an image-reconstruction algorithm to create an image; the design of these algorithms has been a topic of research since at least the 1960's. In the last few years, machine learning-based approaches have shown impressive performance on image reconstruction problems, triggering a wave of enthusiasm and creativity around the paradigm of learning. Our goal is to unify this body of research, identifying common principles and reusable building blocks across decades and among diverse imaging modalities.
We first describe system modeling, emphasizing how a few building blocks can be used to describe a broad range of imaging modalities. We then discuss reconstruction algorithms, grou** them into three broad generations. The first are the classical direct methods, including Tikhonov regularization; the second are the variational methods based on sparsity and the theory of compressive sensing; and the third are the learning-based (also called data-driven) methods, especially those using deep convolutional neural networks. There are strong links between these generations: classical (first-generation) methods appear as modules inside the latter two, and the former two are used to inspire new designs for learning-based (third-generation) methods. As a result, a solid understanding of all of three generations is necessary for the design of state-of-the-art algorithms.
△ Less
Submitted 11 March, 2021; v1 submitted 11 January, 2019;
originally announced January 2019.
-
Pocket Guide to Solve Inverse Problems with GlobalBioIm
Authors:
Emmanuel Soubies,
Ferréol Soulez,
Michael T. McCann,
Thanh-an Pham,
Laurène Donati,
Thomas Debarre,
Daniel Sage,
Michael Unser
Abstract:
GlobalBioIm is an open-source MATLAB library for solving inverse problems. The library capitalizes on the strong commonalities between forward models to standardize the resolution of a wide range of imaging inverse problems. Endowed with an operator-algebra mechanism, GlobalBioIm allows one to easily solve inverse problems by combining elementary modules in a lego-like fashion. This user-friendly…
▽ More
GlobalBioIm is an open-source MATLAB library for solving inverse problems. The library capitalizes on the strong commonalities between forward models to standardize the resolution of a wide range of imaging inverse problems. Endowed with an operator-algebra mechanism, GlobalBioIm allows one to easily solve inverse problems by combining elementary modules in a lego-like fashion. This user-friendly toolbox gives access to cutting-edge reconstruction algorithms, while its high modularity makes it easily extensible to new modalities and novel reconstruction methods. We expect GlobalBioIm to respond to the needs of imaging scientists looking for reliable and easy-to-use computational tools for solving their inverse problems. In this paper, we present in detail the structure and main features of the library. We also illustrate its flexibility with examples from multichannel deconvolution microscopy.
△ Less
Submitted 7 January, 2019; v1 submitted 19 December, 2018;
originally announced December 2018.
-
Fast Rotational Sparse Coding
Authors:
Michael T. McCann,
Vincent Andrearczyk,
Michael Unser,
Adrien Depeursinge
Abstract:
We propose an algorithm for rotational sparse coding along with an efficient implementation using steerability. Sparse coding (also called dictionary learning) is an important technique in image processing, useful in inverse problems, compression, and analysis; however, the usual formulation fails to capture an important aspect of the structure of images: images are formed from building blocks, e.…
▽ More
We propose an algorithm for rotational sparse coding along with an efficient implementation using steerability. Sparse coding (also called dictionary learning) is an important technique in image processing, useful in inverse problems, compression, and analysis; however, the usual formulation fails to capture an important aspect of the structure of images: images are formed from building blocks, e.g., edges, lines, or points, that appear at different locations, orientations, and scales. The sparse coding problem can be reformulated to explicitly account for these transforms, at the cost of increased computation. In this work, we propose an algorithm for a rotational version of sparse coding that is based on K-SVD with additional rotation operations. We then propose a method to accelerate these rotations by learning the dictionary in a steerable basis. Our experiments on patch coding and texture classification demonstrate that the proposed algorithm is fast enough for practical use and compares favorably to standard sparse coding.
△ Less
Submitted 29 January, 2020; v1 submitted 12 June, 2018;
originally announced June 2018.
-
A Review of Convolutional Neural Networks for Inverse Problems in Imaging
Authors:
Michael T. McCann,
Kyong Hwan **,
Michael Unser
Abstract:
In this survey paper, we review recent uses of convolution neural networks (CNNs) to solve inverse problems in imaging. It has recently become feasible to train deep CNNs on large databases of images, and they have shown outstanding performance on object classification and segmentation tasks. Motivated by these successes, researchers have begun to apply CNNs to the resolution of inverse problems s…
▽ More
In this survey paper, we review recent uses of convolution neural networks (CNNs) to solve inverse problems in imaging. It has recently become feasible to train deep CNNs on large databases of images, and they have shown outstanding performance on object classification and segmentation tasks. Motivated by these successes, researchers have begun to apply CNNs to the resolution of inverse problems such as denoising, deconvolution, super-resolution, and medical image reconstruction, and they have started to report improvements over state-of-the-art methods, including sparsity-based techniques such as compressed sensing. Here, we review the recent experimental work in these areas, with a focus on the critical design decisions: Where does the training data come from? What is the architecture of the CNN? and How is the learning problem formulated and solved? We also bring together a few key theoretical papers that offer perspective on why CNNs are appropriate for inverse problems and point to some next steps in the field.
△ Less
Submitted 11 October, 2017;
originally announced October 2017.
-
CNN-Based Projected Gradient Descent for Consistent Image Reconstruction
Authors:
Harshit Gupta,
Kyong Hwan **,
Ha Q. Nguyen,
Michael T. McCann,
Michael Unser
Abstract:
We present a new method for image reconstruction which replaces the projector in a projected gradient descent (PGD) with a convolutional neural network (CNN). CNNs trained as high-dimensional (image-to-image) regressors have recently been used to efficiently solve inverse problems in imaging. However, these approaches lack a feedback mechanism to enforce that the reconstructed image is consistent…
▽ More
We present a new method for image reconstruction which replaces the projector in a projected gradient descent (PGD) with a convolutional neural network (CNN). CNNs trained as high-dimensional (image-to-image) regressors have recently been used to efficiently solve inverse problems in imaging. However, these approaches lack a feedback mechanism to enforce that the reconstructed image is consistent with the measurements. This is crucial for inverse problems, and more so in biomedical imaging, where the reconstructions are used for diagnosis. In our scheme, the gradient descent enforces measurement consistency, while the CNN recursively projects the solution closer to the space of desired reconstruction images. We provide a formal framework to ensure that the classical PGD converges to a local minimizer of a non-convex constrained least-squares problem. When the projector is replaced with a CNN, we propose a relaxed PGD, which always converges. Finally, we propose a simple scheme to train a CNN to act like a projector. Our experiments on sparse view Computed Tomography (CT) reconstruction for both noiseless and noisy measurements show an improvement over the total-variation (TV) method and a recent CNN-based technique.
△ Less
Submitted 6 September, 2017;
originally announced September 2017.
-
Deep Convolutional Neural Network for Inverse Problems in Imaging
Authors:
Kyong Hwan **,
Michael T. McCann,
Emmanuel Froustey,
Michael Unser
Abstract:
In this paper, we propose a novel deep convolutional neural network (CNN)-based algorithm for solving ill-posed inverse problems. Regularized iterative algorithms have emerged as the standard approach to ill-posed inverse problems in the past few decades. These methods produce excellent results, but can be challenging to deploy in practice due to factors including the high computational cost of th…
▽ More
In this paper, we propose a novel deep convolutional neural network (CNN)-based algorithm for solving ill-posed inverse problems. Regularized iterative algorithms have emerged as the standard approach to ill-posed inverse problems in the past few decades. These methods produce excellent results, but can be challenging to deploy in practice due to factors including the high computational cost of the forward and adjoint operators and the difficulty of hyper parameter selection. The starting point of our work is the observation that unrolled iterative methods have the form of a CNN (filtering followed by point-wise non-linearity) when the normal operator (H*H, the adjoint of H times H) of the forward model is a convolution. Based on this observation, we propose using direct inversion followed by a CNN to solve normal-convolutional inverse problems. The direct inversion encapsulates the physical model of the system, but leads to artifacts when the problem is ill-posed; the CNN combines multiresolution decomposition and residual learning in order to learn to remove these artifacts while preserving image structure. We demonstrate the performance of the proposed network in sparse-view reconstruction (down to 50 views) on parallel beam X-ray computed tomography in synthetic phantoms as well as in real experimental sinograms. The proposed network outperforms total variation-regularized iterative reconstruction for the more realistic phantoms and requires less than a second to reconstruct a 512 x 512 image on GPU.
△ Less
Submitted 11 November, 2016;
originally announced November 2016.
-
Rotation Invariant Angular Descriptor Via A Bandlimited Gaussian-like Kernel
Authors:
Michael T. McCann,
Matthew Fickus,
Jelena Kovacevic
Abstract:
We present a new smooth, Gaussian-like kernel that allows the kernel density estimate for an angular distribution to be exactly represented by a finite number of its Fourier series coefficients. Distributions of angular quantities, such as gradients, are a central part of several state-of-the-art image processing algorithms, but these distributions are usually described via histograms and therefor…
▽ More
We present a new smooth, Gaussian-like kernel that allows the kernel density estimate for an angular distribution to be exactly represented by a finite number of its Fourier series coefficients. Distributions of angular quantities, such as gradients, are a central part of several state-of-the-art image processing algorithms, but these distributions are usually described via histograms and therefore lack rotation invariance due to binning artifacts. Replacing histograming with kernel density estimation removes these binning artifacts and can provide a finite-dimensional descriptor of the distribution, provided that the kernel is selected to be bandlimited. In this paper, we present a new band-limited kernel that has the added advantage of being Gaussian-like in the angular domain. We then show that it compares favorably to gradient histograms for patch matching, person detection, and texture segmentation.
△ Less
Submitted 8 June, 2016;
originally announced June 2016.
