-
An efficient hierarchical Bayesian method for the Kuopio tomography challenge 2023
Authors:
Monica Pragliola,
Daniela Calvetti,
Erkki Somersalo
Abstract:
The aim of Electrical Impedance Tomography (EIT) is to determine the electrical conductivity distribution inside a domain by applying currents and measuring voltages on its boundary. Mathematically, the EIT reconstruction task can be formulated as a non-linear inverse problem. The Bayesian inverse problems framework has been applied expensively to solutions of the EIT inverse problem, in particula…
▽ More
The aim of Electrical Impedance Tomography (EIT) is to determine the electrical conductivity distribution inside a domain by applying currents and measuring voltages on its boundary. Mathematically, the EIT reconstruction task can be formulated as a non-linear inverse problem. The Bayesian inverse problems framework has been applied expensively to solutions of the EIT inverse problem, in particular in the cases when the unknown conductivity is believed to be blocky. Recently, the Sparsity Promoting Iterative Alternating Sequential (PS-IAS) algorithm, originally proposed for the solution of linear inverse problems, has been adapted for the non linear case of EIT reconstruction in a computationally efficient manner. Here we introduce a hybrid version of the SP-IAS algorithms for the nonlinear EIT inverse problem, providing a detailed description of the implementation details, with a specific focus on parameters selection. The method is applied to the 2023 Kuopio Tomography Challenge dataset, with a comprehensive report of the running times for the different cases and parameter selections.
△ Less
Submitted 6 May, 2024;
originally announced May 2024.
-
Sparsity-promoting hierarchical Bayesian model for EIT with a blocky target
Authors:
Daniela Calvetti,
Monica Pragliola,
Erkki Somersalo
Abstract:
The electrical impedance tomography (EIT) problem of estimating the unknown conductivity distribution inside a domain from boundary current or voltage measurements requires the solution of a nonlinear inverse problem. Sparsity promoting hierarchical Bayesian models have been shown to be very effective in the recovery of almost piecewise constant solutions in linear inverse problems. We demonstrate…
▽ More
The electrical impedance tomography (EIT) problem of estimating the unknown conductivity distribution inside a domain from boundary current or voltage measurements requires the solution of a nonlinear inverse problem. Sparsity promoting hierarchical Bayesian models have been shown to be very effective in the recovery of almost piecewise constant solutions in linear inverse problems. We demonstrate that by exploiting linear algebraic considerations it is possible to organize the calculation for the Bayesian solution of the nonlinear EIT inverse problem via finite element methods with sparsity promoting priors in a computationally efficient manner. The proposed approach uses the Iterative Alternating Sequential (IAS) algorithm for the solution of the linearized problems. Within the IAS algorithm, a substantial reduction in computational complexity is attained by exploiting the low dimensionality of the data space and an adjoint formulation of the Tikhonov regularized solution that constitutes part of the iterative updating scheme. Numerical tests illustrate the computational efficiency of the proposed algorithm. The paper sheds light also on the convexity properties of the objective function of the maximum a posteriori (MAP) estimation problem.
△ Less
Submitted 24 May, 2024; v1 submitted 29 April, 2024;
originally announced April 2024.
-
Whiteness-based bilevel learning of regularization parameters in imaging
Authors:
Carlo Santambrogio,
Monica Pragliola,
Alessandro Lanza,
Marco Donatelli,
Luca Calatroni
Abstract:
We consider an unsupervised bilevel optimization strategy for learning regularization parameters in the context of imaging inverse problems in the presence of additive white Gaussian noise. Compared to supervised and semi-supervised metrics relying either on the prior knowledge of reference data and/or on some (partial) knowledge on the noise statistics, the proposed approach optimizes the whitene…
▽ More
We consider an unsupervised bilevel optimization strategy for learning regularization parameters in the context of imaging inverse problems in the presence of additive white Gaussian noise. Compared to supervised and semi-supervised metrics relying either on the prior knowledge of reference data and/or on some (partial) knowledge on the noise statistics, the proposed approach optimizes the whiteness of the residual between the observed data and the observation model with no need of ground-truth data.We validate the approach on standard Total Variation-regularized image deconvolution problems which show that the proposed quality metric provides estimates close to the mean-square error oracle and to discrepancy-based principles.
△ Less
Submitted 10 March, 2024;
originally announced March 2024.
-
Bilevel learning of regularization models and their discretization for image deblurring and super-resolution
Authors:
Tatiana A. Bubba,
Luca Calatroni,
Ambra Catozzi,
Serena Crisci,
Thomas Pock,
Monica Pragliola,
Siiri Rautio,
Danilo Riccio,
Andrea Sebastiani
Abstract:
Bilevel learning is a powerful optimization technique that has extensively been employed in recent years to bridge the world of model-driven variational approaches with data-driven methods. Upon suitable parametrization of the desired quantities of interest (e.g., regularization terms or discretization filters), such approach computes optimal parameter values by solving a nested optimization probl…
▽ More
Bilevel learning is a powerful optimization technique that has extensively been employed in recent years to bridge the world of model-driven variational approaches with data-driven methods. Upon suitable parametrization of the desired quantities of interest (e.g., regularization terms or discretization filters), such approach computes optimal parameter values by solving a nested optimization problem where the variational model acts as a constraint. In this work, we consider two different use cases of bilevel learning for the problem of image restoration. First, we focus on learning scalar weights and convolutional filters defining a Field of Experts regularizer to restore natural images degraded by blur and noise. For improving the practical performance, the lower-level problem is solved by means of a gradient descent scheme combined with a line-search strategy based on the Barzilai-Borwein rule. As a second application, the bilevel setup is employed for learning a discretization of the popular total variation regularizer for solving image restoration problems (in particular, deblurring and super-resolution). Numerical results show the effectiveness of the approach and their generalization to multiple tasks.
△ Less
Submitted 27 October, 2023; v1 submitted 20 February, 2023;
originally announced February 2023.
-
Whiteness-based parameter selection for Poisson data in variational image processing
Authors:
Francesca Bevilacqua,
Alessandro Lanza,
Monica Pragliola,
Fiorella Sgallari
Abstract:
We propose a novel automatic parameter selection strategy for variational imaging problems under Poisson noise corruption. The selection of a suitable regularization parameter, whose value is crucial in order to achieve high quality reconstructions, is known to be a particularly hard task in low photon-count regimes. In this work, we extend the so-called residual whiteness principle originally des…
▽ More
We propose a novel automatic parameter selection strategy for variational imaging problems under Poisson noise corruption. The selection of a suitable regularization parameter, whose value is crucial in order to achieve high quality reconstructions, is known to be a particularly hard task in low photon-count regimes. In this work, we extend the so-called residual whiteness principle originally designed for additive white noise to Poisson data. The proposed strategy relies on the study of the whiteness property of a standardized Poisson noise process. After deriving the theoretical properties that motivate our proposal, we solve the target minimization problem with a linearized version of the alternating direction method of multipliers, which is particularly suitable in presence of a general linear forward operator. Our strategy is extensively tested on image restoration and computed tomography reconstruction problems, and compared to the well-known discrepancy principle for Poisson noise proposed by Zanella at al. and with a nearly exact version of it previously proposed by the authors.
△ Less
Submitted 21 July, 2022;
originally announced July 2022.
-
Automatic parameter selection for the TGV regularizer in image restoration under Poisson noise
Authors:
Daniela di Serafino,
Monica Pragliola
Abstract:
We address the image restoration problem under Poisson noise corruption. The Kullback-Leibler divergence, which is typically adopted in the variational framework as data fidelity term in this case, is coupled with the second-order Total Generalized Variation (TGV$^2$). The TGV$^2$ regularizer is known to be capable of preserving both smooth and piece-wise constant features in the image, however it…
▽ More
We address the image restoration problem under Poisson noise corruption. The Kullback-Leibler divergence, which is typically adopted in the variational framework as data fidelity term in this case, is coupled with the second-order Total Generalized Variation (TGV$^2$). The TGV$^2$ regularizer is known to be capable of preserving both smooth and piece-wise constant features in the image, however its behavior is subject to a suitable setting of the parameters arising in its expression. We propose a hierarchical Bayesian formulation of the original problem coupled with a Maximum A Posteriori estimation approach, according to which the unknown image and parameters can be jointly and automatically estimated by minimizing a given cost functional. The minimization problem is tackled via a scheme based on the Alternating Direction Method of Multipliers, which also incorporates a procedure for the automatic selection of the regularization parameter by means of a popular discrepancy principle. Computational results show the effectiveness of our proposal.
△ Less
Submitted 26 May, 2022;
originally announced May 2022.
-
ADMM-based residual whiteness principle for automatic parameter selection in super-resolution problems
Authors:
Monica Pragliola,
Luca Calatroni,
Alessandro Lanza,
Fiorella Sgallari
Abstract:
We propose an automatic parameter selection strategy for the problem of image super-resolution for images corrupted by blur and additive white Gaussian noise with unknown standard deviation. The proposed approach exploits the structure of both the down-sampling and the blur operators in the frequency domain and computes the optimal regularisation parameter as the one optimising a suitable residual…
▽ More
We propose an automatic parameter selection strategy for the problem of image super-resolution for images corrupted by blur and additive white Gaussian noise with unknown standard deviation. The proposed approach exploits the structure of both the down-sampling and the blur operators in the frequency domain and computes the optimal regularisation parameter as the one optimising a suitable residual whiteness measure. Computationally, the proposed strategy relies on the fast solution of generalised Tikhonov $\ell_2$-$\ell_2$ problems as proposed in a work from Zhao et al. These problems naturally appear as substeps of the Alternating Direction Method of Multipliers (ADMM) optimisation approach used to solve super-resolution problems with non-quadratic and often non-smooth, sparsity-promoting regularisers both in convex and in non-convex regimes. After detailing the theoretical properties defined in the frequency domain which allow to express the whiteness functional in a compact way, we report an exhaustive list of numerical experiments proving the effectiveness of the proposed approach for different type of problems, in comparison with well-known parameter selection strategy such as, e.g., the discrepancy principle.
△ Less
Submitted 30 August, 2021;
originally announced August 2021.
-
On and beyond Total Variation regularisation in imaging: the role of space variance
Authors:
Monica Pragliola,
Luca Calatroni,
Alessandro Lanza,
Fiorella Sgallari
Abstract:
Over the last 30 years a plethora of variational regularisation models for image reconstruction has been proposed and thoroughly inspected by the applied mathematics community. Among them, the pioneering prototype often taught and learned in basic courses in mathematical image processing is the celebrated Rudin-Osher-Fatemi (ROF) model \cite{ROF} which relies on the minimisation of the edge-preser…
▽ More
Over the last 30 years a plethora of variational regularisation models for image reconstruction has been proposed and thoroughly inspected by the applied mathematics community. Among them, the pioneering prototype often taught and learned in basic courses in mathematical image processing is the celebrated Rudin-Osher-Fatemi (ROF) model \cite{ROF} which relies on the minimisation of the edge-preserving Total Variation (TV) semi-norm as regularisation term. Despite its (often limiting) simplicity, this model is still very much employed in many applications and used as a benchmark for assessing the performance of modern learning-based image reconstruction approaches, thanks to its thorough analytical and numerical understanding. Among the many extensions to TV proposed over the years, a large class is based on the concept of \emph{space variance}. Space-variant models can indeed overcome the intrinsic inability of TV to describe \emph{local} features (strength, sharpness, directionality) by means of an adaptive mathematical modelling which accommodates local regularisation weighting, variable smoothness and anisotropy. Those ideas can further be cast in the flexible Bayesian framework of generalised Gaussian distributions and combined with maximum likelihood and hierarchical optimisation approaches for efficient hyper-parameter estimation. In this work, we review and connect the major contributions in the field of space-variant TV-type image reconstruction models, focusing, in particular, on their Bayesian interpretation which paves the way to new exciting and unexplored research directions.
△ Less
Submitted 8 April, 2021;
originally announced April 2021.
-
Residual whiteness principle for automatic parameter selection in $\ell_2$-$\ell_2$ image super-resolution problems
Authors:
Monica Pragliola,
Luca Calatroni,
Alessandro Lanza,
Fiorella Sgallari
Abstract:
We propose an automatic parameter selection strategy for variational image super-resolution of blurred and down-sampled images corrupted by additive white Gaussian noise (AWGN) with unknown standard deviation. By exploiting particular properties of the operators describing the problem in the frequency domain, our strategy selects the optimal parameter as the one optimising a suitable residual whit…
▽ More
We propose an automatic parameter selection strategy for variational image super-resolution of blurred and down-sampled images corrupted by additive white Gaussian noise (AWGN) with unknown standard deviation. By exploiting particular properties of the operators describing the problem in the frequency domain, our strategy selects the optimal parameter as the one optimising a suitable residual whiteness measure. Numerical tests show the effectiveness of the proposed strategy for generalised $\ell_2$-$\ell_2$ Tikhonov problems.
△ Less
Submitted 2 April, 2021;
originally announced April 2021.
-
Overcomplete representation in a hierarchical Bayesian framework
Authors:
Monica Pragliola,
Daniela Calvetti,
Erkki Somersalo
Abstract:
A common task in inverse problems and imaging is finding a solution that is sparse, in the sense that most of its components vanish. In the framework of compressed sensing, general results guaranteeing exact recovery have been proven. In practice, sparse solutions are often computed combining $\ell_1$-penalized least squares optimization with an appropriate numerical scheme to accomplish the task.…
▽ More
A common task in inverse problems and imaging is finding a solution that is sparse, in the sense that most of its components vanish. In the framework of compressed sensing, general results guaranteeing exact recovery have been proven. In practice, sparse solutions are often computed combining $\ell_1$-penalized least squares optimization with an appropriate numerical scheme to accomplish the task. A computationally efficient alternative for finding sparse solutions to linear inverse problems is provided by Bayesian hierarchical models, in which the sparsity is encoded by defining a conditionally Gaussian prior model with the prior parameter obeying a generalized gamma distribution. An iterative alternating sequential (IAS) algorithm has been demonstrated to lead to a computationally efficient scheme, and combined with Krylov subspace iterations with an early termination condition, the approach is particularly well suited for large scale problems. Here the Bayesian approach to sparsity is extended to problems whose solution allows a sparse coding in an overcomplete system such as composite frames. It is shown that among the multiple possible representations of the unknown, the IAS algorithm, and in particular, a hybrid version of it, is effectively identifying the most sparse solution. Computed examples show that the method is particularly well suited not only for traditional imaging applications but also for dictionary learning problems in the framework of machine learning.
△ Less
Submitted 28 April, 2021; v1 submitted 24 June, 2020;
originally announced June 2020.
-
Sparsity promoting hybrid solvers for hierarchical Bayesian inverse problems
Authors:
Daniela Calvetti,
Monica Pragliola,
Erkki Somersalo
Abstract:
The recovery of sparse generative models from few noisy measurements is an important and challenging problem. Many deterministic algorithms rely on some form of $\ell_1$-$\ell_2$ minimization to combine the computational convenience of the $\ell_2$ penalty and the sparsity promotion of the $\ell_1$. It was recently shown within the Bayesian framework that sparsity promotion and computational effic…
▽ More
The recovery of sparse generative models from few noisy measurements is an important and challenging problem. Many deterministic algorithms rely on some form of $\ell_1$-$\ell_2$ minimization to combine the computational convenience of the $\ell_2$ penalty and the sparsity promotion of the $\ell_1$. It was recently shown within the Bayesian framework that sparsity promotion and computational efficiency can be attained with hierarchical models with conditionally Gaussian priors and gamma hyperpriors. The related Gibbs energy function is a convex functional and its minimizer, which is the MAP estimate of the posterior, can be computed efficiently with the globally convergent Iterated Alternating Sequential (IAS) algorithm \cite{CSS}. Generalization of the hyperpriors for these sparsity promoting hierarchical models to generalized gamma family yield either globally convex Gibbs energy functionals, or can exhibit local convexity for some choices for the hyperparameters. \cite{CPrSS}. The main problem in computing the MAP solution for greedy hyperpriors that strongly promote sparsity is the presence of local minima. To overcome the premature stop** at a spurious local minimizer, we propose two hybrid algorithms that first exploit the global convergence associated with gamma hyperpriors to arrive in a neighborhood of the unique minimizer, then adopt a generalized gamma hyperprior that promote sparsity more strongly. The performance of the two algorithms is illustrated with computed examples.
△ Less
Submitted 19 March, 2020; v1 submitted 13 March, 2020;
originally announced March 2020.
-
Space-adaptive anisotropic bivariate Laplacian regularization for image restoration
Authors:
Luca Calatroni,
Alessandro Lanza,
Monica Pragliola,
Fiorella Sgallari
Abstract:
In this paper we present a new regularization term for variational image restoration which can be regarded as a space-variant anisotropic extension of the classical isotropic Total Variation (TV) regularizer. The proposed regularizer comes from the statistical assumption that the gradients of the target image distribute locally according to space-variant bivariate Laplacian distributions. The high…
▽ More
In this paper we present a new regularization term for variational image restoration which can be regarded as a space-variant anisotropic extension of the classical isotropic Total Variation (TV) regularizer. The proposed regularizer comes from the statistical assumption that the gradients of the target image distribute locally according to space-variant bivariate Laplacian distributions. The highly flexible variational structure of the corresponding regularizer encodes several free parameters which hold the potential for faithfully modelling the local geometry in the image and describing local orientation preferences. For an automatic estimation of such parameters, we design a robust maximum likelihood approach and report results on its reliability on synthetic data and natural images. A minimization algorithm based on the Alternating Direction Method of Multipliers (ADMM) is presented for the efficient numerical solution of the proposed variational model. Some experimental results are reported which demonstrate the high-quality of restorations achievable by the proposed model, in particular with respect to classical Total Variation regularization.
△ Less
Submitted 2 August, 2019;
originally announced August 2019.
-
Space-variant TV regularization for image restoration
Authors:
Alessandro Lanza,
Serena Morigi,
Monica Pragliola,
Fiorella Sgallari
Abstract:
We propose two new variational models aimed to outperform the popular total variation (TV) model for image restoration with L$_2$ and L$_1$ fidelity terms. In particular, we introduce a space-variant generalization of the TV regularizer, referred to as TV$_p^{SV}$, where the so-called shape parameter $p\,$ is automatically and locally estimated by applying a statistical inference technique based o…
▽ More
We propose two new variational models aimed to outperform the popular total variation (TV) model for image restoration with L$_2$ and L$_1$ fidelity terms. In particular, we introduce a space-variant generalization of the TV regularizer, referred to as TV$_p^{SV}$, where the so-called shape parameter $p\,$ is automatically and locally estimated by applying a statistical inference technique based on the generalized Gaussian distribution. The restored image is efficiently computed by using an alternating direction method of multipliers procedure. We validated our models on images corrupted by Gaussian blur and two important types of noise, namely the additive white Gaussian noise and the impulsive salt and pepper noise. Numerical examples show that the proposed approach is particularly effective and well suited for images characterized by a wide range of gradient distributions.
△ Less
Submitted 21 June, 2019;
originally announced June 2019.
-
Space-variant Generalized Gaussian Regularization for Image Restoration
Authors:
Alessandro Lanza,
Serena Morigi,
Monica Pragliola,
Fiorella Sgallari
Abstract:
We propose a new space-variant regularization term for variational image restoration based on the assumption that the gradient magnitudes of the target image distribute locally according to a half-Generalized Gaussian distribution. This leads to a highly flexible regularizer characterized by two per-pixel free parameters, which are automatically estimated from the observed image. The proposed regu…
▽ More
We propose a new space-variant regularization term for variational image restoration based on the assumption that the gradient magnitudes of the target image distribute locally according to a half-Generalized Gaussian distribution. This leads to a highly flexible regularizer characterized by two per-pixel free parameters, which are automatically estimated from the observed image. The proposed regularizer is coupled with either the $L_2$ or the $L_1$ fidelity terms, in order to effectively deal with additive white Gaussian noise or impulsive noises such as, e.g, additive white Laplace and salt and pepper noise. The restored image is efficiently computed by means of an iterative numerical algorithm based on the alternating direction method of multipliers. Numerical examples indicate that the proposed regularizer holds the potential for achieving high quality restorations for a wide range of target images characterized by different gradient distributions and for the different types of noise considered.
△ Less
Submitted 26 June, 2019; v1 submitted 21 June, 2019;
originally announced June 2019.
-
Adaptive parameter selection for weighted-TV image reconstruction problems
Authors:
Luca Calatroni,
Alessandro Lanza,
Monica Pragliola,
Fiorella Sgallari
Abstract:
We propose an efficient estimation technique for the automatic selection of locally-adaptive Total Variation regularisation parameters based on an hybrid strategy which combines a local maximum-likelihood approach estimating space-variant image scales with a global discrepancy principle related to noise statistics. We verify the effectiveness of the proposed approach solving some exemplar image re…
▽ More
We propose an efficient estimation technique for the automatic selection of locally-adaptive Total Variation regularisation parameters based on an hybrid strategy which combines a local maximum-likelihood approach estimating space-variant image scales with a global discrepancy principle related to noise statistics. We verify the effectiveness of the proposed approach solving some exemplar image reconstruction problems and show its outperformance in comparison to state-of-the-art parameter estimation strategies, the former weighting locally the fit with the data (Dong et al. '11), the latter relying on a bilevel learning paradigm (Hintermüller et al., '17)
△ Less
Submitted 27 May, 2019;
originally announced May 2019.
-
A flexible space-variant anisotropic regularisation for image restoration with automated parameter selection
Authors:
Luca Calatroni,
Alessandro Lanza,
Monica Pragliola,
Fiorella Sgallari
Abstract:
We propose a new space-variant anisotropic regularisation term for variational image restoration, based on the statistical assumption that the gradients of the target image distribute locally according to a bivariate generalised Gaussian distribution. The highly flexible variational structure of the corresponding regulariser encodes several free parameters which hold the potential for faithfully m…
▽ More
We propose a new space-variant anisotropic regularisation term for variational image restoration, based on the statistical assumption that the gradients of the target image distribute locally according to a bivariate generalised Gaussian distribution. The highly flexible variational structure of the corresponding regulariser encodes several free parameters which hold the potential for faithfully modelling the local geometry in the image and describing local orientation preferences. For an automatic estimation of such parameters, we design a robust maximum likelihood approach and report results on its reliability on synthetic data and natural images. For the numerical solution of the corresponding image restoration model, we use an iterative algorithm based on the Alternating Direction Method of Multipliers (ADMM). A suitable preliminary variable splitting together with a novel result in multivariate non-convex proximal calculus yield a very efficient minimisation algorithm. Several numerical results showing significant quality-improvement of the proposed model with respect to some related state-of-the-art competitors are reported, in particular in terms of texture and detail preservation.
△ Less
Submitted 3 April, 2019;
originally announced April 2019.
-
Computational issues and numerical experiments for Linear Multistep Method Particle Filtering
Authors:
Daniela Calvetti,
Salvatore Cuomo,
Monica Pragliola,
Erkki Somersalo,
Gerardo Toraldo
Abstract:
The Linear Multistep Method Particle Filter (LMM PF) is a method for predicting the evolution in time of a evolutionary system governed by a system of differential equations. If some of the parameters of the governing equations are unknowns, it is possible to organize the calculations so as to estimate them while following the evolution of the system in time. The underlying assumption in the appro…
▽ More
The Linear Multistep Method Particle Filter (LMM PF) is a method for predicting the evolution in time of a evolutionary system governed by a system of differential equations. If some of the parameters of the governing equations are unknowns, it is possible to organize the calculations so as to estimate them while following the evolution of the system in time. The underlying assumption in the approach that we present is that all unknowns are modelled as random variables, where the randomness is an indication of the uncertainty of their values rather than an intrinsic property of the quantities. Consequently, the states of the system and the parameters are described in probabilistic terms by their density, often in the form of representative samples. This approach is particularly attractive in the context of parameter estimation inverse problems, because the statistical formulation naturally provides a means of assessing the uncertainty in the solution via the spread of the distribution. The computational efficiency of the underlying sampling technique is crucial for the success of the method, because the accuracy of the solution depends on the ability to produce representative samples from the distribution of the unknown parameters. In this paper LMM PF is tested on a skeletal muscle metabolism problem, which was previously treated within the Ensemble Kalman filtering framework. Here numerical evidences are used to highlight the correlation between the main sources of errors and the influence of the linera multistep method adopted. Finally, we analyzed the effect of replacing LMM with Runge-Kutta class integration methods for supporting the PF technique.
△ Less
Submitted 17 May, 2016;
originally announced May 2016.
