Showing 1–41 of 41 results for author: Wohlberg, B

Total Variation Regularization for Tomographic Reconstruction of Cylindrically Symmetric Objects

Authors: Maliha Hossain, Charles A. Bouman, Brendt Wohlberg

Abstract: Flash X-ray computed tomography (CT) is an important imaging modality for characterization of high-speed dynamic events, such as Kolsky bar impact experiments for the study of mechanical properties of materials subjected to impulsive forces. Due to experimental constraints, the number of X-ray views that can be obtained is typically very sparse in both space and time, requiring strong priors in or… ▽ More Flash X-ray computed tomography (CT) is an important imaging modality for characterization of high-speed dynamic events, such as Kolsky bar impact experiments for the study of mechanical properties of materials subjected to impulsive forces. Due to experimental constraints, the number of X-ray views that can be obtained is typically very sparse in both space and time, requiring strong priors in order to enable a CT reconstruction. In this paper, we propose an effective method for exploiting the cylindrical symmetry inherent in the experiment via a variant of total variation (TV) regularization that operates in cylindrical coordinates, and demonstrate that it outperforms competing approaches. △ Less

Submitted 25 June, 2024; originally announced June 2024.

PtychoDV: Vision Transformer-Based Deep Unrolling Network for Ptychographic Image Reconstruction

Authors: Weijie Gan, Qiuchen Zhai, Michael Thompson McCann, Cristina Garcia Cardona, Ulugbek S. Kamilov, Brendt Wohlberg

Abstract: Ptychography is an imaging technique that captures multiple overlap** snapshots of a sample, illuminated coherently by a moving localized probe. The image recovery from ptychographic data is generally achieved via an iterative algorithm that solves a nonlinear phase retrieval problem derived from measured diffraction patterns. However, these iterative approaches have high computational cost. In… ▽ More Ptychography is an imaging technique that captures multiple overlap** snapshots of a sample, illuminated coherently by a moving localized probe. The image recovery from ptychographic data is generally achieved via an iterative algorithm that solves a nonlinear phase retrieval problem derived from measured diffraction patterns. However, these iterative approaches have high computational cost. In this paper, we introduce PtychoDV, a novel deep model-based network designed for efficient, high-quality ptychographic image reconstruction. PtychoDV comprises a vision transformer that generates an initial image from the set of raw measurements, taking into consideration their mutual correlations. This is followed by a deep unrolling network that refines the initial image using learnable convolutional priors and the ptychography measurement model. Experimental results on simulated data demonstrate that PtychoDV is capable of outperforming existing deep learning methods for this problem, and significantly reduces computational cost compared to iterative methodologies, while maintaining competitive performance. △ Less

Submitted 6 March, 2024; v1 submitted 11 October, 2023; originally announced October 2023.

Learned Full Waveform Inversion Incorporating Task Information for Ultrasound Computed Tomography

Authors: Luke Lozenski, Hanchen Wang, Fu Li, Mark A. Anastasio, Brendt Wohlberg, Youzuo Lin, Umberto Villa

Abstract: Ultrasound computed tomography (USCT) is an emerging imaging modality that holds great promise for breast imaging. Full-waveform inversion (FWI)-based image reconstruction methods incorporate accurate wave physics to produce high spatial resolution quantitative images of speed of sound or other acoustic properties of the breast tissues from USCT measurement data. However, the high computational co… ▽ More Ultrasound computed tomography (USCT) is an emerging imaging modality that holds great promise for breast imaging. Full-waveform inversion (FWI)-based image reconstruction methods incorporate accurate wave physics to produce high spatial resolution quantitative images of speed of sound or other acoustic properties of the breast tissues from USCT measurement data. However, the high computational cost of FWI reconstruction represents a significant burden for its widespread application in a clinical setting. The research reported here investigates the use of a convolutional neural network (CNN) to learn a map** from USCT waveform data to speed of sound estimates. The CNN was trained using a supervised approach with a task-informed loss function aiming at preserving features of the image that are relevant to the detection of lesions. A large set of anatomically and physiologically realistic numerical breast phantoms (NBPs) and corresponding simulated USCT measurements was employed during training. Once trained, the CNN can perform real-time FWI image reconstruction from USCT waveform data. The performance of the proposed method was assessed and compared against FWI using a hold-out sample of 41 NBPs and corresponding USCT data. Accuracy was measured using relative mean square error (RMSE), structural self-similarity index measure (SSIM), and lesion detection performance (DICE score). This numerical experiment demonstrates that a supervised learning model can achieve accuracy comparable to FWI in terms of RMSE and SSIM, and better performance in terms of task performance, while significantly reducing computational time. △ Less

Submitted 30 August, 2023; originally announced August 2023.

Comments: 13 pages, 12 figures

Projected Multi-Agent Consensus Equilibrium (PMACE) with Application to Ptychography

Authors: Qiuchen Zhai, Gregery T. Buzzard, Kevin Mertes, Brendt Wohlberg, Charles A. Bouman

Abstract: Multi-Agent Consensus Equilibrium (MACE) formulates an inverse imaging problem as a balance among multiple update agents such as data-fitting terms and denoisers. However, each such agent operates on a separate copy of the full image, leading to redundant memory use and slow convergence when each agent affects only a small subset of the full image. In this paper, we extend MACE to Projected Multi-… ▽ More Multi-Agent Consensus Equilibrium (MACE) formulates an inverse imaging problem as a balance among multiple update agents such as data-fitting terms and denoisers. However, each such agent operates on a separate copy of the full image, leading to redundant memory use and slow convergence when each agent affects only a small subset of the full image. In this paper, we extend MACE to Projected Multi-Agent Consensus Equilibrium (PMACE), in which each agent updates only a projected component of the full image, thus greatly reducing memory use for some applications.We describe PMACE in terms of an equilibrium problem and an equivalent fixed point problem and show that in most cases the PMACE equilibrium is not the solution of an optimization problem. To demonstrate the value of PMACE, we apply it to the problem of ptychography, in which a sample is reconstructed from the diffraction patterns resulting from coherent X-ray illumination at multiple overlap** spots. In our PMACE formulation, each spot corresponds to a separate data-fitting agent, with the final solution found as an equilibrium among all the agents. Our results demonstrate that the PMACE reconstruction algorithm generates more accurate reconstructions at a lower computational cost than existing ptychography algorithms when the spots are sparsely sampled. △ Less

Submitted 5 October, 2023; v1 submitted 27 March, 2023; originally announced March 2023.

Deep Equilibrium Learning of Explicit Regularizers for Imaging Inverse Problems

Authors: Zihao Zou, Jiaming Liu, Brendt Wohlberg, Ulugbek S. Kamilov

Abstract: There has been significant recent interest in the use of deep learning for regularizing imaging inverse problems. Most work in the area has focused on regularization imposed implicitly by convolutional neural networks (CNNs) pre-trained for image reconstruction. In this work, we follow an alternative line of work based on learning explicit regularization functionals that promote preferred solution… ▽ More There has been significant recent interest in the use of deep learning for regularizing imaging inverse problems. Most work in the area has focused on regularization imposed implicitly by convolutional neural networks (CNNs) pre-trained for image reconstruction. In this work, we follow an alternative line of work based on learning explicit regularization functionals that promote preferred solutions. We develop the Explicit Learned Deep Equilibrium Regularizer (ELDER) method for learning explicit regularizers that minimize a mean-squared error (MSE) metric. ELDER is based on a regularization functional parameterized by a CNN and a deep equilibrium learning (DEQ) method for training the functional to be MSE-optimal at the fixed points of the reconstruction algorithm. The explicit regularizer enables ELDER to directly inherit fundamental convergence results from optimization theory. On the other hand, DEQ training enables ELDER to improve over existing explicit regularizers without prohibitive memory complexity during training. We use ELDER to train several approaches to parameterizing explicit regularizers and test their performance on three distinct imaging inverse problems. Our results show that ELDER can greatly improve the quality of explicit regularizers compared to existing methods, and show that learning explicit regularizers does not compromise performance relative to methods based on implicit regularization. △ Less

Submitted 9 March, 2023; originally announced March 2023.

TRINIDI: Time-of-Flight Resonance Imaging with Neutrons for Isotopic Density Inference

Authors: Thilo Balke, Alexander M. Long, Sven C. Vogel, Brendt Wohlberg, Charles A. Bouman

Abstract: Accurate reconstruction of 2D and 3D isotope densities is a desired capability with great potential impact in applications such as evaluation and development of next-generation nuclear fuels. Neutron time-of-flight (TOF) resonance imaging offers a potential approach by exploiting the characteristic neutron absorption spectra of each isotope. However, it is a major challenge to compute quantitative… ▽ More Accurate reconstruction of 2D and 3D isotope densities is a desired capability with great potential impact in applications such as evaluation and development of next-generation nuclear fuels. Neutron time-of-flight (TOF) resonance imaging offers a potential approach by exploiting the characteristic neutron absorption spectra of each isotope. However, it is a major challenge to compute quantitatively accurate images due to a variety of confounding effects such as severe Poisson noise, background scatter, beam non-uniformity, absorption non-linearity, and extended source pulse duration. We present the TRINIDI algorithm which is based on a two-step process in which we first estimate the neutron flux and background counts, and then reconstruct the areal densities of each isotope and pixel. Both components are based on the inversion of a forward model that accounts for the highly non-linear absorption, energy-dependent emission profile, and Poisson noise, while also modeling the substantial spatio-temporal variation of the background and flux. To do this, we formulate the non-linear inverse problem as two optimization problems that are solved in sequence. We demonstrate on both synthetic and measured data that TRINIDI can reconstruct quantitatively accurate 2D views of isotopic areal density that can then be reconstructed into quantitatively accurate 3D volumes of isotopic volumetric density. △ Less

Submitted 11 September, 2023; v1 submitted 24 February, 2023; originally announced February 2023.

Comments: 15 pages, 20 figures

Report number: LA-UR-23-21022

Coordinate-Based Seismic Interpolation in Irregular Land Survey: A Deep Internal Learning Approach

Authors: Paul Goyes, Edwin Vargas, Claudia Correa, Yu Sun, Ulugbek Kamilov, Brendt Wohlberg, Henry Arguello

Abstract: Physical and budget constraints often result in irregular sampling, which complicates accurate subsurface imaging. Pre-processing approaches, such as missing trace or shot interpolation, are typically employed to enhance seismic data in such cases. Recently, deep learning has been used to address the trace interpolation problem at the expense of large amounts of training data to adequately represe… ▽ More Physical and budget constraints often result in irregular sampling, which complicates accurate subsurface imaging. Pre-processing approaches, such as missing trace or shot interpolation, are typically employed to enhance seismic data in such cases. Recently, deep learning has been used to address the trace interpolation problem at the expense of large amounts of training data to adequately represent typical seismic events. Nonetheless, most research in this area has focused on trace reconstruction, with little attention having been devoted to shot interpolation. Furthermore, existing methods assume regularly spaced receivers/sources failing in approximating seismic data from real (irregular) surveys. This work presents a novel shot gather interpolation approach which uses a continuous coordinate-based representation of the acquired seismic wavefield parameterized by a neural network. The proposed unsupervised approach, which we call coordinate-based seismic interpolation(CoBSI), enables the prediction of specific seismic characteristics in irregular land surveys without using external data during neural network training. Experimental results on real and synthetic 3D data validate the ability of the proposed method to estimate continuous smooth seismic events in the time-space and frequency-wavenumber domains, improving sparsity or low-rank-based interpolation methods. △ Less

Submitted 9 February, 2023; v1 submitted 21 November, 2022; originally announced November 2022.

Plug-and-Play Methods for Integrating Physical and Learned Models in Computational Imaging

Authors: Ulugbek S. Kamilov, Charles A. Bouman, Gregery T. Buzzard, Brendt Wohlberg

Abstract: Plug-and-Play Priors (PnP) is one of the most widely-used frameworks for solving computational imaging problems through the integration of physical models and learned models. PnP leverages high-fidelity physical sensor models and powerful machine learning methods for prior modeling of data to provide state-of-the-art reconstruction algorithms. PnP algorithms alternate between minimizing a data-fid… ▽ More Plug-and-Play Priors (PnP) is one of the most widely-used frameworks for solving computational imaging problems through the integration of physical models and learned models. PnP leverages high-fidelity physical sensor models and powerful machine learning methods for prior modeling of data to provide state-of-the-art reconstruction algorithms. PnP algorithms alternate between minimizing a data-fidelity term to promote data consistency and imposing a learned regularizer in the form of an image denoiser. Recent highly-successful applications of PnP algorithms include bio-microscopy, computerized tomography, magnetic resonance imaging, and joint ptycho-tomography. This article presents a unified and principled review of PnP by tracing its roots, describing its major variations, summarizing main results, and discussing applications in computational imaging. We also point the way towards further developments by discussing recent results on equilibrium equations that formulate the problem associated with PnP algorithms. △ Less

Submitted 12 August, 2022; v1 submitted 31 March, 2022; originally announced March 2022.

Projected Multi-Agent Consensus Equilibrium for Ptychographic Image Reconstruction

Authors: Qiuchen Zhai, Brendt Wohlberg, Gregery T. Buzzard, Charles A. Bouman

Abstract: Ptychography is a computational imaging technique using multiple, overlap**, coherently illuminated snapshots to achieve nanometer resolution by solving a nonlinear phase-field recovery problem. Ptychography is vital for imaging of manufactured nanomaterials, but existing algorithms have computational shortcomings that limit large-scale application. In this paper, we present the Projected Multi-… ▽ More Ptychography is a computational imaging technique using multiple, overlap**, coherently illuminated snapshots to achieve nanometer resolution by solving a nonlinear phase-field recovery problem. Ptychography is vital for imaging of manufactured nanomaterials, but existing algorithms have computational shortcomings that limit large-scale application. In this paper, we present the Projected Multi-Agent Consensus Equilibrium (PMACE) approach for solving the ptychography inversion problem. This approach extends earlier work on MACE, which formulates an inversion problem as an equilibrium among multiple agents, each acting independently to update a full reconstruction. In PMACE, each agent acts on a portion (projection) corresponding to one of the snapshots, and these updates to projections are then combined to give an update to the full reconstruction. The resulting algorithm is easily parallelized, with convergence properties inherited from convergence results associated with MACE. We apply our method on simulated data and demonstrate that it outperforms competing algorithms in both reconstruction quality and convergence speed. △ Less

Submitted 8 December, 2021; v1 submitted 28 November, 2021; originally announced November 2021.

Comments: To be published in Asilomar Conference on Signals, Systems, and Computers 2021

Hyperspectral Neutron CT with Material Decomposition

Authors: Thilo Balke, Alexander M. Long, Sven C. Vogel, Brendt Wohlberg, Charles A. Bouman

Abstract: Energy resolved neutron imaging (ERNI) is an advanced neutron radiography technique capable of non-destructively extracting spatial isotopic information within a given material. Energy-dependent radiography image sequences can be created by utilizing neutron time-of-flight techniques. In combination with uniquely characteristic isotopic neutron cross-section spectra, isotopic areal densities can b… ▽ More Energy resolved neutron imaging (ERNI) is an advanced neutron radiography technique capable of non-destructively extracting spatial isotopic information within a given material. Energy-dependent radiography image sequences can be created by utilizing neutron time-of-flight techniques. In combination with uniquely characteristic isotopic neutron cross-section spectra, isotopic areal densities can be determined on a per-pixel basis, thus resulting in a set of areal density images for each isotope present in the sample. By preforming ERNI measurements over several rotational views, an isotope decomposed 3D computed tomography is possible. We demonstrate a method involving a robust and automated background estimation based on a linear programming formulation. The extremely high noise due to low count measurements is overcome using a sparse coding approach. It allows for a significant computation time improvement, from weeks to a few hours compared to existing neutron evaluation tools, enabling at the present stage a semi-quantitative, user-friendly routine application. △ Less

Submitted 5 October, 2021; originally announced October 2021.

Comments: 5 pages, 4 figures

Report number: LA-UR-21-21281

Journal ref: 2021 IEEE International Conference on Image Processing (ICIP), 2021, pp. 3482-3486

Recovery Analysis for Plug-and-Play Priors using the Restricted Eigenvalue Condition

Authors: Jiaming Liu, M. Salman Asif, Brendt Wohlberg, Ulugbek S. Kamilov

Abstract: The plug-and-play priors (PnP) and regularization by denoising (RED) methods have become widely used for solving inverse problems by leveraging pre-trained deep denoisers as image priors. While the empirical imaging performance and the theoretical convergence properties of these algorithms have been widely investigated, their recovery properties have not previously been theoretically analyzed. We… ▽ More The plug-and-play priors (PnP) and regularization by denoising (RED) methods have become widely used for solving inverse problems by leveraging pre-trained deep denoisers as image priors. While the empirical imaging performance and the theoretical convergence properties of these algorithms have been widely investigated, their recovery properties have not previously been theoretically analyzed. We address this gap by showing how to establish theoretical recovery guarantees for PnP/RED by assuming that the solution of these methods lies near the fixed-points of a deep neural network. We also present numerical results comparing the recovery performance of PnP/RED in compressive sensing against that of recent compressive sensing algorithms based on generative models. Our numerical results suggest that PnP with a pre-trained artifact removal network provides significantly better results compared to the existing state-of-the-art methods. △ Less

Submitted 26 October, 2021; v1 submitted 7 June, 2021; originally announced June 2021.

Comments: 27 pages, 13 figures

Connect the Dots: In Situ 4D Seismic Monitoring of CO2 Storage with Spatio-temporal CNNs

Authors: Shihang Feng, Xitong Zhang, Brendt Wohlberg, Neill Symons, Youzuo Lin

Abstract: 4D seismic imaging has been widely used in CO$_2$ sequestration projects to monitor the fluid flow in the volumetric subsurface region that is not sampled by wells. Ideally, real-time monitoring and near-future forecasting would provide site operators with great insights to understand the dynamics of the subsurface reservoir and assess any potential risks. However, due to obstacles such as high de… ▽ More 4D seismic imaging has been widely used in CO$_2$ sequestration projects to monitor the fluid flow in the volumetric subsurface region that is not sampled by wells. Ideally, real-time monitoring and near-future forecasting would provide site operators with great insights to understand the dynamics of the subsurface reservoir and assess any potential risks. However, due to obstacles such as high deployment cost, availability of acquisition equipment, exclusion zones around surface structures, only very sparse seismic imaging data can be obtained during monitoring. That leads to an unavoidable and growing knowledge gap over time. The operator needs to understand the fluid flow throughout the project lifetime and the seismic data are only available at a limited number of times. This is insufficient for understanding the reservoir behavior. To overcome those challenges, we have developed spatio-temporal neural-network-based models that can produce high-fidelity interpolated or extrapolated images effectively and efficiently. Specifically, our models are built on an autoencoder, and incorporate the long short-term memory (LSTM) structure with a new loss function regularized by optical flow. We validate the performance of our models using real 4D post-stack seismic imaging data acquired at the Sleipner CO$_2$ sequestration field. We employ two different strategies in evaluating our models. Numerically, we compare our models with different baseline approaches using classic pixel-based metrics. We also conduct a blind survey and collect a total of 20 responses from domain experts to evaluate the quality of data generated by our models. Via both numerical and expert evaluation, we conclude that our models can produce high-quality 2D/3D seismic imaging data at a reasonable cost, offering the possibility of real-time monitoring or even near-future forecasting of the CO$_2$ storage reservoir. △ Less

Submitted 25 August, 2021; v1 submitted 24 May, 2021; originally announced May 2021.

Comments: 15 pages, 13 figures

Randomized Algorithms for Scientific Computing (RASC)

Authors: Aydin Buluc, Tamara G. Kolda, Stefan M. Wild, Mihai Anitescu, Anthony DeGennaro, John Jakeman, Chandrika Kamath, Ramakrishnan Kannan, Miles E. Lopes, Per-Gunnar Martinsson, Kary Myers, Jelani Nelson, Juan M. Restrepo, C. Seshadhri, Draguna Vrabie, Brendt Wohlberg, Stephen J. Wright, Chao Yang, Peter Zwart

Abstract: Randomized algorithms have propelled advances in artificial intelligence and represent a foundational research area in advancing AI for Science. Future advancements in DOE Office of Science priority areas such as climate science, astrophysics, fusion, advanced materials, combustion, and quantum computing all require randomized algorithms for surmounting challenges of complexity, robustness, and sc… ▽ More Randomized algorithms have propelled advances in artificial intelligence and represent a foundational research area in advancing AI for Science. Future advancements in DOE Office of Science priority areas such as climate science, astrophysics, fusion, advanced materials, combustion, and quantum computing all require randomized algorithms for surmounting challenges of complexity, robustness, and scalability. This report summarizes the outcomes of that workshop, "Randomized Algorithms for Scientific Computing (RASC)," held virtually across four days in December 2020 and January 2021. △ Less

Submitted 21 March, 2022; v1 submitted 19 April, 2021; originally announced April 2021.

InversionNet3D: Efficient and Scalable Learning for 3D Full Waveform Inversion

Authors: Qili Zeng, Shihang Feng, Brendt Wohlberg, Youzuo Lin

Abstract: Seismic full-waveform inversion (FWI) techniques aim to find a high-resolution subsurface geophysical model provided with waveform data. Some recent effort in data-driven FWI has shown some encouraging results in obtaining 2D velocity maps. However, due to high computational complexity and large memory consumption, the reconstruction of 3D high-resolution velocity maps via deep networks is still a… ▽ More Seismic full-waveform inversion (FWI) techniques aim to find a high-resolution subsurface geophysical model provided with waveform data. Some recent effort in data-driven FWI has shown some encouraging results in obtaining 2D velocity maps. However, due to high computational complexity and large memory consumption, the reconstruction of 3D high-resolution velocity maps via deep networks is still a great challenge. In this paper, we present InversionNet3D, an efficient and scalable encoder-decoder network for 3D FWI. The proposed method employs group convolution in the encoder to establish an effective hierarchy for learning information from multiple sources while cutting down unnecessary parameters and operations at the same time. The introduction of invertible layers further reduces the memory consumption of intermediate features during training and thus enables the development of deeper networks with more layers and higher capacity as required by different application scenarios. Experiments on the 3D Kimberlina dataset demonstrate that InversionNet3D achieves state-of-the-art reconstruction performance with lower computational cost and lower memory footprint compared to the baseline. △ Less

Submitted 27 October, 2021; v1 submitted 25 March, 2021; originally announced March 2021.

Multiscale Data-driven Seismic Full-waveform Inversion with Field Data Study

Authors: Shihang Feng, Youzuo Lin, Brendt Wohlberg

Abstract: Seismic full-waveform inversion (FWI), which uses iterative methods to estimate high-resolution subsurface models from seismograms, is a powerful imaging technique in exploration geophysics. In recent years, the computational cost of FWI has grown exponentially due to the increasing size and resolution of seismic data. Moreover, it is a non-convex problem and can encounter local minima due to the… ▽ More Seismic full-waveform inversion (FWI), which uses iterative methods to estimate high-resolution subsurface models from seismograms, is a powerful imaging technique in exploration geophysics. In recent years, the computational cost of FWI has grown exponentially due to the increasing size and resolution of seismic data. Moreover, it is a non-convex problem and can encounter local minima due to the limited accuracy of the initial velocity models or the absence of low frequencies in the measurements. To overcome these computational issues, we develop a multiscale data-driven FWI method based on fully convolutional networks (FCN). In preparing the training data, we first develop a real-time style transform method to create a large set of synthetic subsurface velocity models from natural images. We then develop two convolutional neural networks with encoder-decoder structure to reconstruct the low- and high-frequency components of the subsurface velocity models, separately. To validate the performance of our data-driven inversion method and the effectiveness of the synthesized training set, we compare it with conventional physics-based waveform inversion approaches using both synthetic and field data. These numerical results demonstrate that, once our model is fully trained, it can significantly reduce the computation time, and yield more accurate subsurface velocity models in comparison with conventional FWI. △ Less

Submitted 25 November, 2023; v1 submitted 5 March, 2021; originally announced March 2021.

Comments: 15 pages, 14 figures

CoIL: Coordinate-based Internal Learning for Imaging Inverse Problems

Authors: Yu Sun, Jiaming Liu, Mingyang Xie, Brendt Wohlberg, Ulugbek S. Kamilov

Abstract: We propose Coordinate-based Internal Learning (CoIL) as a new deep-learning (DL) methodology for the continuous representation of measurements. Unlike traditional DL methods that learn a map** from the measurements to the desired image, CoIL trains a multilayer perceptron (MLP) to encode the complete measurement field by map** the coordinates of the measurements to their responses. CoIL is a s… ▽ More We propose Coordinate-based Internal Learning (CoIL) as a new deep-learning (DL) methodology for the continuous representation of measurements. Unlike traditional DL methods that learn a map** from the measurements to the desired image, CoIL trains a multilayer perceptron (MLP) to encode the complete measurement field by map** the coordinates of the measurements to their responses. CoIL is a self-supervised method that requires no training examples besides the measurements of the test object itself. Once the MLP is trained, CoIL generates new measurements that can be used within a majority of image reconstruction methods. We validate CoIL on sparse-view computed tomography using several widely-used reconstruction methods, including purely model-based methods and those based on DL. Our results demonstrate the ability of CoIL to consistently improve the performance of all the considered methods by providing high-fidelity measurement fields. △ Less

Submitted 9 February, 2021; originally announced February 2021.

SGD-Net: Efficient Model-Based Deep Learning with Theoretical Guarantees

Authors: Jiaming Liu, Yu Sun, Weijie Gan, Xiaojian Xu, Brendt Wohlberg, Ulugbek S. Kamilov

Abstract: Deep unfolding networks have recently gained popularity in the context of solving imaging inverse problems. However, the computational and memory complexity of data-consistency layers within traditional deep unfolding networks scales with the number of measurements, limiting their applicability to large-scale imaging inverse problems. We propose SGD-Net as a new methodology for improving the effic… ▽ More Deep unfolding networks have recently gained popularity in the context of solving imaging inverse problems. However, the computational and memory complexity of data-consistency layers within traditional deep unfolding networks scales with the number of measurements, limiting their applicability to large-scale imaging inverse problems. We propose SGD-Net as a new methodology for improving the efficiency of deep unfolding through stochastic approximations of the data-consistency layers. Our theoretical analysis shows that SGD-Net can be trained to approximate batch deep unfolding networks to an arbitrary precision. Our numerical results on intensity diffraction tomography and sparse-view computed tomography show that SGD-Net can match the performance of the batch network at a fraction of training and testing complexity. △ Less

Submitted 22 January, 2021; originally announced January 2021.

PSF Estimation in Crowded Astronomical Imagery as a Convolutional Dictionary Learning Problem

Authors: Brendt Wohlberg, Przemek Wozniak

Abstract: We present a new algorithm for estimating the Point Spread Function (PSF) in wide-field astronomical images with extreme source crowding. Robust and accurate PSF estimation in crowded astronomical images dramatically improves the fidelity of astrometric and photometric measurements extracted from wide-field sky monitoring imagery. Our radically new approach utilizes convolutional sparse representa… ▽ More We present a new algorithm for estimating the Point Spread Function (PSF) in wide-field astronomical images with extreme source crowding. Robust and accurate PSF estimation in crowded astronomical images dramatically improves the fidelity of astrometric and photometric measurements extracted from wide-field sky monitoring imagery. Our radically new approach utilizes convolutional sparse representations to model the continuous functions involved in the image formation. This approach avoids the need to detect and precisely localize individual point sources that is shared by existing methods. In experiments involving simulated astronomical imagery, it significantly outperforms the recent alternative method with which it is compared. △ Less

Submitted 7 February, 2021; v1 submitted 4 January, 2021; originally announced January 2021.

Report number: LA-UR-20-28195

Joint Reconstruction and Calibration using Regularization by Denoising

Authors: Mingyang Xie, Yu Sun, Jiaming Liu, Brendt Wohlberg, Ulugbek S. Kamilov

Abstract: Regularization by denoising (RED) is a broadly applicable framework for solving inverse problems by using priors specified as denoisers. While RED has been shown to provide state-of-the-art performance in a number of applications, existing RED algorithms require exact knowledge of the measurement operator characterizing the imaging system, limiting their applicability in problems where the measure… ▽ More Regularization by denoising (RED) is a broadly applicable framework for solving inverse problems by using priors specified as denoisers. While RED has been shown to provide state-of-the-art performance in a number of applications, existing RED algorithms require exact knowledge of the measurement operator characterizing the imaging system, limiting their applicability in problems where the measurement operator has parametric uncertainties. We propose a new method, called Calibrated RED (Cal-RED), that enables joint calibration of the measurement operator along with reconstruction of the unknown image. Cal-RED extends the traditional RED methodology to imaging problems that require the calibration of the measurement operator. We validate Cal-RED on the problem of image reconstruction in computerized tomography (CT) under perturbed projection angles. Our results corroborate the effectiveness of Cal-RED for joint calibration and reconstruction using pre-trained deep denoisers as image priors. △ Less

Submitted 26 November, 2020; originally announced November 2020.

Async-RED: A Provably Convergent Asynchronous Block Parallel Stochastic Method using Deep Denoising Priors

Authors: Yu Sun, Jiaming Liu, Yiran Sun, Brendt Wohlberg, Ulugbek S. Kamilov

Abstract: Regularization by denoising (RED) is a recently developed framework for solving inverse problems by integrating advanced denoisers as image priors. Recent work has shown its state-of-the-art performance when combined with pre-trained deep denoisers. However, current RED algorithms are inadequate for parallel processing on multicore systems. We address this issue by proposing a new asynchronous RED… ▽ More Regularization by denoising (RED) is a recently developed framework for solving inverse problems by integrating advanced denoisers as image priors. Recent work has shown its state-of-the-art performance when combined with pre-trained deep denoisers. However, current RED algorithms are inadequate for parallel processing on multicore systems. We address this issue by proposing a new asynchronous RED (ASYNC-RED) algorithm that enables asynchronous parallel processing of data, making it significantly faster than its serial counterparts for large-scale inverse problems. The computational complexity of ASYNC-RED is further reduced by using a random subset of measurements at every iteration. We present complete theoretical analysis of the algorithm by establishing its convergence under explicit assumptions on the data-fidelity and the denoiser. We validate ASYNC-RED on image recovery using pre-trained deep denoisers as priors. △ Less

Submitted 3 October, 2020; originally announced October 2020.

Physics-Consistent Data-driven Waveform Inversion with Adaptive Data Augmentation

Authors: Renán Rojas-Gómez, Jihyun Yang, Youzuo Lin, James Theiler, Brendt Wohlberg

Abstract: Seismic full-waveform inversion (FWI) is a nonlinear computational imaging technique that can provide detailed estimates of subsurface geophysical properties. Solving the FWI problem can be challenging due to its ill-posedness and high computational cost. In this work, we develop a new hybrid computational approach to solve FWI that combines physics-based models with data-driven methodologies. In… ▽ More Seismic full-waveform inversion (FWI) is a nonlinear computational imaging technique that can provide detailed estimates of subsurface geophysical properties. Solving the FWI problem can be challenging due to its ill-posedness and high computational cost. In this work, we develop a new hybrid computational approach to solve FWI that combines physics-based models with data-driven methodologies. In particular, we develop a data augmentation strategy that can not only improve the representativity of the training set but also incorporate important governing physics into the training process and therefore improve the inversion accuracy. To validate the performance, we apply our method to synthetic elastic seismic waveform data generated from a subsurface geologic model built on a carbon sequestration site at Kimberlina, California. We compare our physics-consistent data-driven inversion method to both purely physics-based and purely data-driven approaches and observe that our method yields higher accuracy and greater generalization ability. △ Less

Submitted 3 September, 2020; originally announced September 2020.

Scalable Plug-and-Play ADMM with Convergence Guarantees

Authors: Yu Sun, Zihui Wu, Xiaojian Xu, Brendt Wohlberg, Ulugbek S. Kamilov

Abstract: Plug-and-play priors (PnP) is a broadly applicable methodology for solving inverse problems by exploiting statistical priors specified as denoisers. Recent work has reported the state-of-the-art performance of PnP algorithms using pre-trained deep neural nets as denoisers in a number of imaging applications. However, current PnP algorithms are impractical in large-scale settings due to their heavy… ▽ More Plug-and-play priors (PnP) is a broadly applicable methodology for solving inverse problems by exploiting statistical priors specified as denoisers. Recent work has reported the state-of-the-art performance of PnP algorithms using pre-trained deep neural nets as denoisers in a number of imaging applications. However, current PnP algorithms are impractical in large-scale settings due to their heavy computational and memory requirements. This work addresses this issue by proposing an incremental variant of the widely used PnP-ADMM algorithm, making it scalable to large-scale datasets. We theoretically analyze the convergence of the algorithm under a set of explicit assumptions, extending recent theoretical results in the area. Additionally, we show the effectiveness of our algorithm with nonsmooth data-fidelity terms and deep neural net priors, its fast convergence compared to existing PnP algorithms, and its scalability in terms of speed and memory. △ Less

Submitted 22 January, 2021; v1 submitted 5 June, 2020; originally announced June 2020.

Comments: First three authors contribute equally and are listed in alphabetical order

Provable Convergence of Plug-and-Play Priors with MMSE denoisers

Authors: Xiaojian Xu, Yu Sun, Jiaming Liu, Brendt Wohlberg, Ulugbek S. Kamilov

Abstract: Plug-and-play priors (PnP) is a methodology for regularized image reconstruction that specifies the prior through an image denoiser. While PnP algorithms are well understood for denoisers performing maximum a posteriori probability (MAP) estimation, they have not been analyzed for the minimum mean squared error (MMSE) denoisers. This letter addresses this gap by establishing the first theoretical… ▽ More Plug-and-play priors (PnP) is a methodology for regularized image reconstruction that specifies the prior through an image denoiser. While PnP algorithms are well understood for denoisers performing maximum a posteriori probability (MAP) estimation, they have not been analyzed for the minimum mean squared error (MMSE) denoisers. This letter addresses this gap by establishing the first theoretical convergence result for the iterative shrinkage/thresholding algorithm (ISTA) variant of PnP for MMSE denoisers. We show that the iterates produced by PnP-ISTA with an MMSE denoiser converge to a stationary point of some global cost function. We validate our analysis on sparse signal recovery in compressive sensing by comparing two types of denoisers, namely the exact MMSE denoiser and the approximate MMSE denoiser obtained by training a deep neural net. △ Less

Submitted 15 May, 2020; originally announced May 2020.

Diagram Image Retrieval using Sketch-Based Deep Learning and Transfer Learning

Authors: Manish Bhattarai, Diane Oyen, Juan Castorena, Li** Yang, Brendt Wohlberg

Abstract: Resolution of the complex problem of image retrieval for diagram images has yet to be reached. Deep learning methods continue to excel in the fields of object detection and image classification applied to natural imagery. However, the application of such methodologies applied to binary imagery remains limited due to lack of crucial features such as textures,color and intensity information. This pa… ▽ More Resolution of the complex problem of image retrieval for diagram images has yet to be reached. Deep learning methods continue to excel in the fields of object detection and image classification applied to natural imagery. However, the application of such methodologies applied to binary imagery remains limited due to lack of crucial features such as textures,color and intensity information. This paper presents a deep learning based method for image-based search for binary patent images by taking advantage of existing large natural image repositories for image search and sketch-based methods (Sketches are not identical to diagrams, but they do share some characteristics; for example, both imagery types are gray scale (binary), composed of contours, and are lacking in texture). We begin by using deep learning to generate sketches from natural images for image retrieval and then train a second deep learning model on the sketches. We then use our small set of manually labeled patent diagram images via transfer learning to adapt the image search from sketches of natural images to diagrams. Our experiment results show the effectiveness of deep learning with transfer learning for detecting near-identical copies in patent images and querying similar images based on content. △ Less

Submitted 22 April, 2020; originally announced April 2020.

TGGLines: A Robust Topological Graph Guided Line Segment Detector for Low Quality Binary Images

Authors: Ming Gong, Li** Yang, Catherine Potts, Vijayan K. Asari, Diane Oyen, Brendt Wohlberg

Abstract: Line segment detection is an essential task in computer vision and image analysis, as it is the critical foundation for advanced tasks such as shape modeling and road lane line detection for autonomous driving. We present a robust topological graph guided approach for line segment detection in low quality binary images (hence, we call it TGGLines). Due to the graph-guided approach, TGGLines not on… ▽ More Line segment detection is an essential task in computer vision and image analysis, as it is the critical foundation for advanced tasks such as shape modeling and road lane line detection for autonomous driving. We present a robust topological graph guided approach for line segment detection in low quality binary images (hence, we call it TGGLines). Due to the graph-guided approach, TGGLines not only detects line segments, but also organizes the segments with a line segment connectivity graph, which means the topological relationships (e.g., intersection, an isolated line segment) of the detected line segments are captured and stored; whereas other line detectors only retain a collection of loose line segments. Our empirical results show that the TGGLines detector visually and quantitatively outperforms state-of-the-art line segment detection methods. In addition, our TGGLines approach has the following two competitive advantages: (1) our method only requires one parameter and it is adaptive, whereas almost all other line segment detection methods require multiple (non-adaptive) parameters, and (2) the line segments detected by TGGLines are organized by a line segment connectivity graph. △ Less

Submitted 27 February, 2020; originally announced February 2020.

Boosting the Performance of Plug-and-Play Priors via Denoiser Scaling

Authors: Xiaojian Xu, Jiaming Liu, Yu Sun, Brendt Wohlberg, Ulugbek S. Kamilov

Abstract: Plug-and-play priors (PnP) is an image reconstruction framework that uses an image denoiser as an imaging prior. Unlike traditional regularized inversion, PnP does not require the prior to be expressible in the form of a regularization function. This flexibility enables PnP algorithms to exploit the most effective image denoisers, leading to their state-of-the-art performance in various imaging ta… ▽ More Plug-and-play priors (PnP) is an image reconstruction framework that uses an image denoiser as an imaging prior. Unlike traditional regularized inversion, PnP does not require the prior to be expressible in the form of a regularization function. This flexibility enables PnP algorithms to exploit the most effective image denoisers, leading to their state-of-the-art performance in various imaging tasks. In this paper, we propose a new denoiser scaling technique to explicitly control the amount of PnP regularization. Traditionally, the performance of PnP algorithms is controlled via intrinsic parameters of the denoiser related to the noise variance. However, many powerful denoisers, such as the ones based on convolutional neural networks (CNNs), do not have tunable parameters that would allow controlling their influence within PnP. To address this issue, we introduce a scaling parameter that adjusts the magnitude of the denoiser input and output. We theoretical justify the denoiser scaling from the perspectives of proximal optimization, statistical estimation, and consensus equilibrium. Finally, we provide numerical experiments demonstrating the ability of denoiser scaling to systematically improve the performance of PnP for denoising CNN priors that do not have explicitly tunable parameters. △ Less

Submitted 26 February, 2020; originally announced February 2020.

Two-layer Residual Sparsifying Transform Learning for Image Reconstruction

Authors: Xuehang Zheng, Saiprasad Ravishankar, Yong Long, Marc Louis Klasky, Brendt Wohlberg

Abstract: Signal models based on sparsity, low-rank and other properties have been exploited for image reconstruction from limited and corrupted data in medical imaging and other computational imaging applications. In particular, sparsifying transform models have shown promise in various applications, and offer numerous advantages such as efficiencies in sparse coding and learning. This work investigates pr… ▽ More Signal models based on sparsity, low-rank and other properties have been exploited for image reconstruction from limited and corrupted data in medical imaging and other computational imaging applications. In particular, sparsifying transform models have shown promise in various applications, and offer numerous advantages such as efficiencies in sparse coding and learning. This work investigates pre-learning a two-layer extension of the transform model for image reconstruction, wherein the transform domain or filtering residuals of the image are further sparsified in the second layer. The proposed block coordinate descent optimization algorithms involve highly efficient updates. Preliminary numerical experiments demonstrate the usefulness of a two-layer model over the previous related schemes for CT image reconstruction from low-dose measurements. △ Less

Submitted 7 January, 2020; v1 submitted 1 June, 2019; originally announced June 2019.

Comments: Accepted to IEEE ISBI 2020

Plug-In Stochastic Gradient Method

Authors: Yu Sun, Brendt Wohlberg, Ulugbek S. Kamilov

Abstract: Plug-and-play priors (PnP) is a popular framework for regularized signal reconstruction by using advanced denoisers within an iterative algorithm. In this paper, we discuss our recent online variant of PnP that uses only a subset of measurements at every iteration, which makes it scalable to very large datasets. We additionally present novel convergence results for both batch and online PnP algori… ▽ More Plug-and-play priors (PnP) is a popular framework for regularized signal reconstruction by using advanced denoisers within an iterative algorithm. In this paper, we discuss our recent online variant of PnP that uses only a subset of measurements at every iteration, which makes it scalable to very large datasets. We additionally present novel convergence results for both batch and online PnP algorithms. △ Less

Submitted 8 November, 2018; originally announced November 2018.

Comments: To be presented at International Biomedical and Astronomical Signal Processing (BASP) Frontiers workshop 2019

Regularized Fourier Ptychography using an Online Plug-and-Play Algorithm

Authors: Yu Sun, Shiqi Xu, Yunzhe Li, Lei Tian, Brendt Wohlberg, Ulugbek S. Kamilov

Abstract: The plug-and-play priors (PnP) framework has been recently shown to achieve state-of-the-art results in regularized image reconstruction by leveraging a sophisticated denoiser within an iterative algorithm. In this paper, we propose a new online PnP algorithm for Fourier ptychographic microscopy (FPM) based on the fast iterative shrinkage/threshold algorithm (FISTA). Specifically, the proposed alg… ▽ More The plug-and-play priors (PnP) framework has been recently shown to achieve state-of-the-art results in regularized image reconstruction by leveraging a sophisticated denoiser within an iterative algorithm. In this paper, we propose a new online PnP algorithm for Fourier ptychographic microscopy (FPM) based on the fast iterative shrinkage/threshold algorithm (FISTA). Specifically, the proposed algorithm uses only a subset of measurements, which makes it scalable to a large set of measurements. We validate the algorithm by showing that it can lead to significant performance gains on both simulated and experimental data. △ Less

Submitted 2 November, 2018; v1 submitted 31 October, 2018; originally announced November 2018.

Convolutional Dictionary Regularizers for Tomographic Inversion

Authors: Singanallur Venkatakrishnan, Brendt Wohlberg

Abstract: There has been a growing interest in the use of data-driven regularizers to solve inverse problems associated with computational imaging systems. The convolutional sparse representation model has recently gained attention, driven by the development of fast algorithms for solving the dictionary learning and sparse coding problems for sufficiently large images and data sets. Nevertheless, this model… ▽ More There has been a growing interest in the use of data-driven regularizers to solve inverse problems associated with computational imaging systems. The convolutional sparse representation model has recently gained attention, driven by the development of fast algorithms for solving the dictionary learning and sparse coding problems for sufficiently large images and data sets. Nevertheless, this model has seen very limited application to tomographic reconstruction problems. In this paper, we present a model-based tomographic reconstruction algorithm using a learnt convolutional dictionary as a regularizer. The key contribution is the use of a data-dependent weighting scheme for the l1 regularization to construct an effective denoising method that is integrated into the inversion using the Plug-and-Play reconstruction framework. Using simulated data sets we demonstrate that our approach can improve performance over traditional regularizers based on a Markov random field model and a patch-based sparse representation model for sparse and limited-view tomographic data sets. △ Less

Submitted 18 February, 2019; v1 submitted 30 October, 2018; originally announced October 2018.

Learning Multi-Layer Transform Models

Authors: Saiprasad Ravishankar, Brendt Wohlberg

Abstract: Learned data models based on sparsity are widely used in signal processing and imaging applications. A variety of methods for learning synthesis dictionaries, sparsifying transforms, etc., have been proposed in recent years, often imposing useful structures or properties on the models. In this work, we focus on sparsifying transform learning, which enjoys a number of advantages. We consider multi-… ▽ More Learned data models based on sparsity are widely used in signal processing and imaging applications. A variety of methods for learning synthesis dictionaries, sparsifying transforms, etc., have been proposed in recent years, often imposing useful structures or properties on the models. In this work, we focus on sparsifying transform learning, which enjoys a number of advantages. We consider multi-layer or nested extensions of the transform model, and propose efficient learning algorithms. Numerical experiments with image data illustrate the behavior of the multi-layer transform learning algorithm and its usefulness for image denoising. Multi-layer models provide better denoising quality than single layer schemes. △ Less

Submitted 18 October, 2018; originally announced October 2018.

Comments: In Proceedings of the Annual Allerton Conference on Communication, Control, and Computing, 2018

An Online Plug-and-Play Algorithm for Regularized Image Reconstruction

Authors: Yu Sun, Brendt Wohlberg, Ulugbek S. Kamilov

Abstract: Plug-and-play priors (PnP) is a powerful framework for regularizing imaging inverse problems by using advanced denoisers within an iterative algorithm. Recent experimental evidence suggests that PnP algorithms achieve state-of-the-art performance in a range of imaging applications. In this paper, we introduce a new online PnP algorithm based on the iterative shrinkage/thresholding algorithm (ISTA)… ▽ More Plug-and-play priors (PnP) is a powerful framework for regularizing imaging inverse problems by using advanced denoisers within an iterative algorithm. Recent experimental evidence suggests that PnP algorithms achieve state-of-the-art performance in a range of imaging applications. In this paper, we introduce a new online PnP algorithm based on the iterative shrinkage/thresholding algorithm (ISTA). The proposed algorithm uses only a subset of measurements at every iteration, which makes it scalable to very large datasets. We present a new theoretical convergence analysis, for both batch and online variants of PnP-ISTA, for denoisers that do not necessarily correspond to proximal operators. We also present simulations illustrating the applicability of the algorithm to image reconstruction in diffraction tomography. The results in this paper have the potential to expand the applicability of the PnP framework to very large and redundant datasets. △ Less

Submitted 12 September, 2018; originally announced September 2018.

Efficient Projection onto the $\ell_{\infty,1}$ Mixed-Norm Ball using a Newton root search method

Authors: Gustavo Chau, Brendt Wohlberg, Paul Rodriguez

Abstract: Mixed norms that promote structured sparsity have numerous applications in signal processing and machine learning problems. In this work, we present a new algorithm, based on a Newton root search technique, for computing the projection onto the $\ell_{\infty,1}$ ball, which has found application in cognitive neuroscience and classification tasks. Numerical simulations show that our proposed method… ▽ More Mixed norms that promote structured sparsity have numerous applications in signal processing and machine learning problems. In this work, we present a new algorithm, based on a Newton root search technique, for computing the projection onto the $\ell_{\infty,1}$ ball, which has found application in cognitive neuroscience and classification tasks. Numerical simulations show that our proposed method is between 8 and 10 times faster on average, and up to 20 times faster for very sparse solutions, than the previous state of the art. Tests on real functional magnetic resonance image data show that, for some data distributions, our algorithm can obtain speed improvements by a factor of between 10 and 100, depending on the implementation. △ Less

Submitted 23 May, 2019; v1 submitted 26 June, 2018; originally announced June 2018.

Comments: 21 pages, 5 figures

Convolutional Dictionary Learning: A Comparative Review and New Algorithms

Authors: Cristina Garcia-Cardona, Brendt Wohlberg

Abstract: Convolutional sparse representations are a form of sparse representation with a dictionary that has a structure that is equivalent to convolution with a set of linear filters. While effective algorithms have recently been developed for the convolutional sparse coding problem, the corresponding dictionary learning problem is substantially more challenging. Furthermore, although a number of differen… ▽ More Convolutional sparse representations are a form of sparse representation with a dictionary that has a structure that is equivalent to convolution with a set of linear filters. While effective algorithms have recently been developed for the convolutional sparse coding problem, the corresponding dictionary learning problem is substantially more challenging. Furthermore, although a number of different approaches have been proposed, the absence of thorough comparisons between them makes it difficult to determine which of them represents the current state of the art. The present work both addresses this deficiency and proposes some new approaches that outperform existing ones in certain contexts. A thorough set of performance comparisons indicates a very wide range of performance differences among the existing and proposed methods, and clearly identifies those that are the most effective. △ Less

Submitted 5 September, 2018; v1 submitted 8 September, 2017; originally announced September 2017.

Comments: Corrected typos in Eq. (18) and (19)

Journal ref: IEEE Transactions on Computational Imaging, vol. 4, no. 3, pp. 366-381, Sep 2018

First and Second Order Methods for Online Convolutional Dictionary Learning

Authors: Jialin Liu, Cristina Garcia-Cardona, Brendt Wohlberg, Wotao Yin

Abstract: Convolutional sparse representations are a form of sparse representation with a structured, translation invariant dictionary. Most convolutional dictionary learning algorithms to date operate in batch mode, requiring simultaneous access to all training images during the learning process, which results in very high memory usage and severely limits the training data that can be used. Very recently,… ▽ More Convolutional sparse representations are a form of sparse representation with a structured, translation invariant dictionary. Most convolutional dictionary learning algorithms to date operate in batch mode, requiring simultaneous access to all training images during the learning process, which results in very high memory usage and severely limits the training data that can be used. Very recently, however, a number of authors have considered the design of online convolutional dictionary learning algorithms that offer far better scaling of memory and computational cost with training set size than batch methods. This paper extends our prior work, improving a number of aspects of our previous algorithm; proposing an entirely new one, with better performance, and that supports the inclusion of a spatial mask for learning from incomplete data; and providing a rigorous theoretical analysis of these methods. △ Less

Submitted 16 June, 2018; v1 submitted 31 August, 2017; originally announced September 2017.

Journal ref: SIAM J. Imaging Sci., 11(2), 1589-1628, 2018

Convolutional Sparse Coding with Overlap** Group Norms

Authors: Brendt Wohlberg

Abstract: The most widely used form of convolutional sparse coding uses an $\ell_1$ regularization term. While this approach has been successful in a variety of applications, a limitation of the $\ell_1$ penalty is that it is homogeneous across the spatial and filter index dimensions of the sparse representation array, so that sparsity cannot be separately controlled across these dimensions. The present pap… ▽ More The most widely used form of convolutional sparse coding uses an $\ell_1$ regularization term. While this approach has been successful in a variety of applications, a limitation of the $\ell_1$ penalty is that it is homogeneous across the spatial and filter index dimensions of the sparse representation array, so that sparsity cannot be separately controlled across these dimensions. The present paper considers the consequences of replacing the $\ell_1$ penalty with a mixed group norm, motivated by recent theoretical results for convolutional sparse representations. Algorithms are developed for solving the resulting problems, which are quite challenging, and the impact on the performance of the denoising problem is evaluated. The mixed group norms are found to perform very poorly in this application. While their performance is greatly improved by introducing a weighting strategy, such a strategy also improves the performance obtained from the much simpler and computationally cheaper $\ell_1$ norm. △ Less

Submitted 29 August, 2017; originally announced August 2017.

Convolutional Sparse Coding: Boundary Handling Revisited

Authors: Brendt Wohlberg, Paul Rodriguez

Abstract: Two different approaches have recently been proposed for boundary handling in convolutional sparse representations, avoiding potential boundary artifacts arising from the circular boundary conditions implied by the use of frequency domain solution methods by introducing a spatial mask into the convolutional sparse coding problem. In the present paper we show that, under certain circumstances, thes… ▽ More Two different approaches have recently been proposed for boundary handling in convolutional sparse representations, avoiding potential boundary artifacts arising from the circular boundary conditions implied by the use of frequency domain solution methods by introducing a spatial mask into the convolutional sparse coding problem. In the present paper we show that, under certain circumstances, these methods fail in their design goal of avoiding boundary artifacts. The reasons for this failure are discussed, a solution is proposed, and the practical implications are illustrated in an image deblurring problem. △ Less

Submitted 20 July, 2017; originally announced July 2017.

Online Convolutional Dictionary Learning

Authors: Jialin Liu, Cristina Garcia-Cardona, Brendt Wohlberg, Wotao Yin

Abstract: While a number of different algorithms have recently been proposed for convolutional dictionary learning, this remains an expensive problem. The single biggest impediment to learning from large training sets is the memory requirements, which grow at least linearly with the size of the training set since all existing methods are batch algorithms. The work reported here addresses this limitation by… ▽ More While a number of different algorithms have recently been proposed for convolutional dictionary learning, this remains an expensive problem. The single biggest impediment to learning from large training sets is the memory requirements, which grow at least linearly with the size of the training set since all existing methods are batch algorithms. The work reported here addresses this limitation by extending online dictionary learning ideas to the convolutional context. △ Less

Submitted 30 August, 2017; v1 submitted 28 June, 2017; originally announced June 2017.

Comments: Accepted to be presented at ICIP 2017

Journal ref: Proceedings of IEEE International Conference on Image Processing (ICIP), 2017, pp. 1707-1711

Convolutional Sparse Representations with Gradient Penalties

Authors: Brendt Wohlberg

Abstract: While convolutional sparse representations enjoy a number of useful properties, they have received limited attention for image reconstruction problems. The present paper compares the performance of block-based and convolutional sparse representations in the removal of Gaussian white noise. While the usual formulation of the convolutional sparse coding problem is slightly inferior to the block-base… ▽ More While convolutional sparse representations enjoy a number of useful properties, they have received limited attention for image reconstruction problems. The present paper compares the performance of block-based and convolutional sparse representations in the removal of Gaussian white noise. While the usual formulation of the convolutional sparse coding problem is slightly inferior to the block-based representations in this problem, the performance of the convolutional form can be boosted beyond that of the block-based form by the inclusion of suitable penalties on the gradients of the coefficient maps. △ Less

Submitted 15 February, 2018; v1 submitted 11 May, 2017; originally announced May 2017.

ADMM Penalty Parameter Selection by Residual Balancing

Authors: Brendt Wohlberg

Abstract: Appropriate selection of the penalty parameter is crucial to obtaining good performance from the Alternating Direction Method of Multipliers (ADMM). While analytic results for optimal selection of this parameter are very limited, there is a heuristic method that appears to be relatively successful in a number of different problems. The contribution of this paper is to demonstrate that their is a p… ▽ More Appropriate selection of the penalty parameter is crucial to obtaining good performance from the Alternating Direction Method of Multipliers (ADMM). While analytic results for optimal selection of this parameter are very limited, there is a heuristic method that appears to be relatively successful in a number of different problems. The contribution of this paper is to demonstrate that their is a potentially serious flaw in this heuristic approach, and to propose a modification that at least partially addresses it. △ Less

Submitted 20 April, 2017; originally announced April 2017.

Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation

Authors: Suhas Sreehari, S. V. Venkatakrishnan, Brendt Wohlberg, Lawrence F. Drummy, Jeffrey P. Simmons, Charles A. Bouman

Abstract: Many material and biological samples in scientific imaging are characterized by non-local repeating structures. These are studied using scanning electron microscopy and electron tomography. Sparse sampling of individual pixels in a 2D image acquisition geometry, or sparse sampling of projection images with large tilt increments in a tomography experiment, can enable high speed data acquisition and… ▽ More Many material and biological samples in scientific imaging are characterized by non-local repeating structures. These are studied using scanning electron microscopy and electron tomography. Sparse sampling of individual pixels in a 2D image acquisition geometry, or sparse sampling of projection images with large tilt increments in a tomography experiment, can enable high speed data acquisition and minimize sample damage caused by the electron beam. In this paper, we present an algorithm for electron tomographic reconstruction and sparse image interpolation that exploits the non-local redundancy in images. We adapt a framework, termed plug-and-play (P&P) priors, to solve these imaging problems in a regularized inversion setting. The power of the P&P approach is that it allows a wide array of modern denoising algorithms to be used as a "prior model" for tomography and image interpolation. We also present sufficient mathematical conditions that ensure convergence of the P&P approach, and we use these insights to design a new non-local means denoising algorithm. Finally, we demonstrate that the algorithm produces higher quality reconstructions on both simulated and real electron microscope data, along with improved convergence properties compared to other methods. △ Less

Submitted 22 December, 2015; originally announced December 2015.

Comments: 13 pages, 11 figures