-
Whiteness-based bilevel learning of regularization parameters in imaging
Authors:
Carlo Santambrogio,
Monica Pragliola,
Alessandro Lanza,
Marco Donatelli,
Luca Calatroni
Abstract:
We consider an unsupervised bilevel optimization strategy for learning regularization parameters in the context of imaging inverse problems in the presence of additive white Gaussian noise. Compared to supervised and semi-supervised metrics relying either on the prior knowledge of reference data and/or on some (partial) knowledge on the noise statistics, the proposed approach optimizes the whitene…
▽ More
We consider an unsupervised bilevel optimization strategy for learning regularization parameters in the context of imaging inverse problems in the presence of additive white Gaussian noise. Compared to supervised and semi-supervised metrics relying either on the prior knowledge of reference data and/or on some (partial) knowledge on the noise statistics, the proposed approach optimizes the whiteness of the residual between the observed data and the observation model with no need of ground-truth data.We validate the approach on standard Total Variation-regularized image deconvolution problems which show that the proposed quality metric provides estimates close to the mean-square error oracle and to discrepancy-based principles.
△ Less
Submitted 10 March, 2024;
originally announced March 2024.
-
Deep image prior inpainting of ancient frescoes in the Mediterranean Alpine arc
Authors:
Fabio Merizzi,
Perrine Saillard,
Oceane Acquier,
Elena Morotti,
Elena Loli Piccolomini,
Luca Calatroni,
Rosa Maria Dessì
Abstract:
The unprecedented success of image reconstruction approaches based on deep neural networks has revolutionised both the processing and the analysis paradigms in several applied disciplines. In the field of digital humanities, the task of digital reconstruction of ancient frescoes is particularly challenging due to the scarce amount of available training data caused by ageing, wear, tear and retouch…
▽ More
The unprecedented success of image reconstruction approaches based on deep neural networks has revolutionised both the processing and the analysis paradigms in several applied disciplines. In the field of digital humanities, the task of digital reconstruction of ancient frescoes is particularly challenging due to the scarce amount of available training data caused by ageing, wear, tear and retouching over time. To overcome these difficulties, we consider the Deep Image Prior (DIP) inpainting approach which computes appropriate reconstructions by relying on the progressive updating of an untrained convolutional neural network so as to match the reliable piece of information in the image at hand while promoting regularisation elsewhere. In comparison with state-of-the-art approaches (based on variational/PDEs and patch-based methods), DIP-based inpainting reduces artefacts and better adapts to contextual/non-local information, thus providing a valuable and effective tool for art historians. As a case study, we apply such approach to reconstruct missing image contents in a dataset of highly damaged digital images of medieval paintings located into several chapels in the Mediterranean Alpine Arc and provide a detailed description on how visible and invisible (e.g., infrared) information can be integrated for identifying and reconstructing damaged image regions.
△ Less
Submitted 11 December, 2023; v1 submitted 25 June, 2023;
originally announced June 2023.
-
Fluctuation-based deconvolution in fluorescence microscopy using plug-and-play denoisers
Authors:
Vasiliki Stergiopoulou,
Subhadip Mukherjee,
Luca Calatroni,
Laure Blanc-Féraud
Abstract:
The spatial resolution of images of living samples obtained by fluorescence microscopes is physically limited due to the diffraction of visible light, which makes the study of entities of size less than the diffraction barrier (around 200 nm in the x-y plane) very challenging. To overcome this limitation, several deconvolution and super-resolution techniques have been proposed. Within the framewor…
▽ More
The spatial resolution of images of living samples obtained by fluorescence microscopes is physically limited due to the diffraction of visible light, which makes the study of entities of size less than the diffraction barrier (around 200 nm in the x-y plane) very challenging. To overcome this limitation, several deconvolution and super-resolution techniques have been proposed. Within the framework of inverse problems, modern approaches in fluorescence microscopy reconstruct a super-resolved image from a temporal stack of frames by carefully designing suitable hand-crafted sparsity-promoting regularisers. Numerically, such approaches are solved by proximal gradient-based iterative schemes. Aiming at obtaining a reconstruction more adapted to sample geometries (e.g. thin filaments), we adopt a plug-and-play denoising approach with convergence guarantees and replace the proximity operator associated with the explicit image regulariser with an image denoiser (i.e. a pre-trained network) which, upon appropriate training, mimics the action of an implicit prior. To account for the independence of the fluctuations between molecules, the model relies on second-order statistics. The denoiser is then trained on covariance images coming from data representing sequences of fluctuating fluorescent molecules with filament structure. The method is evaluated on both simulated and real fluorescence microscopy images, showing its ability to correctly reconstruct filament structures with high values of peak signal-to-noise ratio (PSNR).
△ Less
Submitted 20 March, 2023;
originally announced March 2023.
-
A cortical-inspired sub-Riemannian model for Poggendorff-type visual illusions
Authors:
Emre Baspinar,
Luca Calatroni,
Valentina Franceschi,
Dario Prandi
Abstract:
We consider Wilson-Cowan-type models for the mathematical description of orientation-dependent Poggendorff-like illusions. Our modelling improves two previously proposed cortical-inspired approaches embedding the sub-Riemannian heat kernel into the neuronal interaction term, in agreement with the intrinsically anisotropic functional architecture of V1 based on both local and lateral connections. F…
▽ More
We consider Wilson-Cowan-type models for the mathematical description of orientation-dependent Poggendorff-like illusions. Our modelling improves two previously proposed cortical-inspired approaches embedding the sub-Riemannian heat kernel into the neuronal interaction term, in agreement with the intrinsically anisotropic functional architecture of V1 based on both local and lateral connections. For the numerical realisation of both models, we consider standard gradient descent algorithms combined with Fourier-based approaches for the efficient computation of the sub-Laplacian evolution. Our numerical results show that the use of the sub-Riemannian kernel allows to reproduce numerically visual misperceptions and inpainting-type biases in a stronger way in comparison with the previous approaches.
△ Less
Submitted 29 January, 2021; v1 submitted 28 December, 2020;
originally announced December 2020.
-
Non-convex Super-resolution of OCT images via sparse representation
Authors:
Gabriele Scrivanti,
Luca Calatroni,
Serena Morigi,
Lindsay Nicholson,
Alin Achim
Abstract:
We propose a non-convex variational model for the super-resolution of Optical Coherence Tomography (OCT) images of the murine eye, by enforcing sparsity with respect to suitable dictionaries learnt from high-resolution OCT data. The statistical characteristics of OCT images motivate the use of α-stable distributions for learning dictionaries, by considering the non-Gaussian case, α=1. The sparsity…
▽ More
We propose a non-convex variational model for the super-resolution of Optical Coherence Tomography (OCT) images of the murine eye, by enforcing sparsity with respect to suitable dictionaries learnt from high-resolution OCT data. The statistical characteristics of OCT images motivate the use of α-stable distributions for learning dictionaries, by considering the non-Gaussian case, α=1. The sparsity-promoting cost function relies on a non-convex penalty - Cauchy-based or Minimax Concave Penalty (MCP) - which makes the problem particularly challenging. We propose an efficient algorithm for minimizing the function based on the forward-backward splitting strategy which guarantees at each iteration the existence and uniqueness of the proximal point. Comparisons with standard convex L1-based reconstructions show the better performance of non-convex models, especially in view of further OCT image analysis
△ Less
Submitted 22 October, 2020;
originally announced October 2020.
-
Cortical-inspired Wilson-Cowan-type equations for orientation-dependent contrast perception modelling
Authors:
Marcelo Bertalmío,
Luca Calatroni,
Valentina Franceschi,
Benedetta Franceschiello,
Dario Prandi
Abstract:
We consider the evolution model proposed in [9, 6] to describe illusory contrast perception phenomena induced by surrounding orientations. Firstly, we highlight its analogies and differences with the widely used Wilson-Cowan equations [48], mainly in terms of efficient representation properties. Then, in order to explicitly encode local directional information, we exploit the model of the primary…
▽ More
We consider the evolution model proposed in [9, 6] to describe illusory contrast perception phenomena induced by surrounding orientations. Firstly, we highlight its analogies and differences with the widely used Wilson-Cowan equations [48], mainly in terms of efficient representation properties. Then, in order to explicitly encode local directional information, we exploit the model of the primary visual cortex (V1) proposed in [20] and largely used over the last years for several image processing problems [24,38,28]. The resulting model is thus defined in the space of positions and orientation and it is capable to describe assimilation and contrast visual bias at the same time. We report several numerical tests showing the ability of the model to reproduce, in particular, orientation-dependent phenomena such as grating induction and a modified version of the Poggendorff illusion. For this latter example, we empirically show the existence of a set of threshold parameters differentiating from inpainting to perception-type reconstructions and describing long-range connectivity between different hypercolumns in V1.
△ Less
Submitted 17 July, 2020; v1 submitted 15 October, 2019;
originally announced October 2019.
-
Variational Osmosis for Non-linear Image Fusion
Authors:
Simone Parisotto,
Luca Calatroni,
Aurélie Bugeau,
Nicolas Papadakis,
Carola-Bibiane Schönlieb
Abstract:
We propose a new variational model for non-linear image fusion. Our approach is based on the use of an osmosis energy term related to the one studied in Vogel et al. (2013) and Weickert et al. (2013) The minimization of the proposed non-convex energy realizes visually plausible image data fusion, invariant to multiplicative brightness changes. On the practical side, it requires minimal supervision…
▽ More
We propose a new variational model for non-linear image fusion. Our approach is based on the use of an osmosis energy term related to the one studied in Vogel et al. (2013) and Weickert et al. (2013) The minimization of the proposed non-convex energy realizes visually plausible image data fusion, invariant to multiplicative brightness changes. On the practical side, it requires minimal supervision and parameter tuning and can encode prior information on the structure of the images to be fused. For the numerical solution of the proposed model, we develop a primal-dual algorithm and we apply the resulting minimization scheme to solve multi-modal face fusion, color transfer and cultural heritage conservation problems. Visual and quantitative comparisons to state-of-the-art approaches prove the out-performance and the flexibility of our method.
△ Less
Submitted 24 March, 2020; v1 submitted 4 October, 2019;
originally announced October 2019.
-
Space-adaptive anisotropic bivariate Laplacian regularization for image restoration
Authors:
Luca Calatroni,
Alessandro Lanza,
Monica Pragliola,
Fiorella Sgallari
Abstract:
In this paper we present a new regularization term for variational image restoration which can be regarded as a space-variant anisotropic extension of the classical isotropic Total Variation (TV) regularizer. The proposed regularizer comes from the statistical assumption that the gradients of the target image distribute locally according to space-variant bivariate Laplacian distributions. The high…
▽ More
In this paper we present a new regularization term for variational image restoration which can be regarded as a space-variant anisotropic extension of the classical isotropic Total Variation (TV) regularizer. The proposed regularizer comes from the statistical assumption that the gradients of the target image distribute locally according to space-variant bivariate Laplacian distributions. The highly flexible variational structure of the corresponding regularizer encodes several free parameters which hold the potential for faithfully modelling the local geometry in the image and describing local orientation preferences. For an automatic estimation of such parameters, we design a robust maximum likelihood approach and report results on its reliability on synthetic data and natural images. A minimization algorithm based on the Alternating Direction Method of Multipliers (ADMM) is presented for the efficient numerical solution of the proposed variational model. Some experimental results are reported which demonstrate the high-quality of restorations achievable by the proposed model, in particular with respect to classical Total Variation regularization.
△ Less
Submitted 2 August, 2019;
originally announced August 2019.
-
A cortical-inspired model for orientation-dependent contrast perception: a link with Wilson-Cowan equations
Authors:
Marcelo Bertalmío,
Luca Calatroni,
Valentina Franceschi,
Benedetta Franceschiello,
Dario Prandi
Abstract:
We consider a differential model describing neuro-physiological contrast perception phenomena induced by surrounding orientations. The mathematical formulation relies on a cortical-inspired modelling [10] largely used over the last years to describe neuron interactions in the primary visual cortex (V1) and applied to several image processing problems [12,19,13]. Our model connects to Wilson-Cowan-…
▽ More
We consider a differential model describing neuro-physiological contrast perception phenomena induced by surrounding orientations. The mathematical formulation relies on a cortical-inspired modelling [10] largely used over the last years to describe neuron interactions in the primary visual cortex (V1) and applied to several image processing problems [12,19,13]. Our model connects to Wilson-Cowan-type equations [23] and it is analogous to the one used in [3,2,14] to describe assimilation and contrast phenomena, the main novelty being its explicit dependence on local image orientation. To confirm the validity of the model, we report some numerical tests showing its ability to explain orientation-dependent phenomena (such as grating induction) and geometric-optical illusions [21,16] classically explained only by filtering-based techniques [6,18].
△ Less
Submitted 18 December, 2018;
originally announced December 2018.
-
Unveiling the invisible - mathematical methods for restoring and interpreting illuminated manuscripts
Authors:
Luca Calatroni,
Marie d'Autume,
Rob Hocking,
Stella Panayotova,
Simone Parisotto,
Paola Ricciardi,
Carola-Bibiane Schönlieb
Abstract:
The last fifty years have seen an impressive development of mathematical methods for the analysis and processing of digital images, mostly in the context of photography, biomedical imaging and various forms of engineering. The arts have been mostly overlooked in this process, apart from a few exceptional works in the last ten years. With the rapid emergence of digitisation in the arts, however, th…
▽ More
The last fifty years have seen an impressive development of mathematical methods for the analysis and processing of digital images, mostly in the context of photography, biomedical imaging and various forms of engineering. The arts have been mostly overlooked in this process, apart from a few exceptional works in the last ten years. With the rapid emergence of digitisation in the arts, however, the arts domain is becoming increasingly receptive to digital image processing methods and the importance of paying attention to this therefore increases. In this paper we discuss a range of mathematical methods for digital image restoration and digital visualisation for illuminated manuscripts. The latter provide an interesting opportunity for digital manipulation because they traditionally remain physically untouched. At the same time they also serve as an example for the possibilities mathematics and digital restoration offer as a generic and objective toolkit for the arts.
△ Less
Submitted 19 March, 2018;
originally announced March 2018.
-
Graph clustering, variational image segmentation methods and Hough transform scale detection for object measurement in images
Authors:
Luca Calatroni,
Yves van Gennip,
Carola-Bibiane Schönlieb,
Hannah Rowland,
Arjuna Flenner
Abstract:
We consider the problem of scale detection in images where a region of interest is present together with a measurement tool (e.g. a ruler). For the segmentation part, we focus on the graph based method by Flenner and Bertozzi which reinterprets classical continuous Ginzburg-Landau minimisation models in a totally discrete framework. To overcome the numerical difficulties due to the large size of t…
▽ More
We consider the problem of scale detection in images where a region of interest is present together with a measurement tool (e.g. a ruler). For the segmentation part, we focus on the graph based method by Flenner and Bertozzi which reinterprets classical continuous Ginzburg-Landau minimisation models in a totally discrete framework. To overcome the numerical difficulties due to the large size of the images considered we use matrix completion and splitting techniques. The scale on the measurement tool is detected via a Hough transform based algorithm. The method is then applied to some measurement tasks arising in real-world applications such as zoology, medicine and archaeology.
△ Less
Submitted 23 September, 2016; v1 submitted 27 February, 2016;
originally announced February 2016.
-
Bilevel approaches for learning of variational imaging models
Authors:
Luca Calatroni,
Cao Chung,
Juan Carlos De Los Reyes,
Carola-Bibiane Schönlieb,
Tuomo Valkonen
Abstract:
We review some recent learning approaches in variational imaging, based on bilevel optimisation, and emphasize the importance of their treatment in function space. The paper covers both analytical and numerical techniques. Analytically, we include results on the existence and structure of minimisers, as well as optimality conditions for their characterisation. Based on this information, Newton typ…
▽ More
We review some recent learning approaches in variational imaging, based on bilevel optimisation, and emphasize the importance of their treatment in function space. The paper covers both analytical and numerical techniques. Analytically, we include results on the existence and structure of minimisers, as well as optimality conditions for their characterisation. Based on this information, Newton type methods are studied for the solution of the problems at hand, combining them with sampling techniques in case of large databases. The computational verification of the developed techniques is extensively documented, covering instances with different type of regularisers, several noise models, spatially dependent weights and large image databases.
△ Less
Submitted 8 May, 2015;
originally announced May 2015.
