Showing 1–9 of 9 results for author: Kervrann, C

On normalization-equivariance properties of supervised and unsupervised denoising methods: a survey

Authors: Sébastien Herbreteau, Charles Kervrann

Abstract: Image denoising is probably the oldest and still one of the most active research topic in image processing. Many methodological concepts have been introduced in the past decades and have improved performances significantly in recent years, especially with the emergence of convolutional neural networks and supervised deep learning. In this paper, we propose a survey of guided tour of supervised and… ▽ More Image denoising is probably the oldest and still one of the most active research topic in image processing. Many methodological concepts have been introduced in the past decades and have improved performances significantly in recent years, especially with the emergence of convolutional neural networks and supervised deep learning. In this paper, we propose a survey of guided tour of supervised and unsupervised learning methods for image denoising, classifying the main principles elaborated during this evolution, with a particular concern given to recent developments in supervised learning. It is conceived as a tutorial organizing in a comprehensive framework current approaches. We give insights on the rationales and limitations of the most performant methods in the literature, and we highlight the common features between many of them. Finally, we focus on on the normalization equivariance properties that is surprisingly not guaranteed with most of supervised methods. It is of paramount importance that intensity shifting or scaling applied to the input image results in a corresponding change in the denoiser output. △ Less

Submitted 23 February, 2024; originally announced February 2024.

A unified framework of non-local parametric methods for image denoising

Authors: Sébastien Herbreteau, Charles Kervrann

Abstract: We propose a unified view of non-local methods for single-image denoising, for which BM3D is the most popular representative, that operate by gathering noisy patches together according to their similarities in order to process them collaboratively. Our general estimation framework is based on the minimization of the quadratic risk, which is approximated in two steps, and adapts to photon and elect… ▽ More We propose a unified view of non-local methods for single-image denoising, for which BM3D is the most popular representative, that operate by gathering noisy patches together according to their similarities in order to process them collaboratively. Our general estimation framework is based on the minimization of the quadratic risk, which is approximated in two steps, and adapts to photon and electronic noises. Relying on unbiased risk estimation (URE) for the first step and on ``internal adaptation'', a concept borrowed from deep learning theory, for the second, we show that our approach enables to reinterpret and reconcile previous state-of-the-art non-local methods. Within this framework, we propose a novel denoiser called NL-Ridge that exploits linear combinations of patches. While conceptually simpler, we show that NL-Ridge can outperform well-established state-of-the-art single-image denoisers. △ Less

Submitted 21 February, 2024; originally announced February 2024.

Comments: arXiv admin note: text overlap with arXiv:2203.00570

Normalization-Equivariant Neural Networks with Application to Image Denoising

Authors: Sébastien Herbreteau, Emmanuel Moebel, Charles Kervrann

Abstract: In many information processing systems, it may be desirable to ensure that any change of the input, whether by shifting or scaling, results in a corresponding change in the system response. While deep neural networks are gradually replacing all traditional automatic processing methods, they surprisingly do not guarantee such normalization-equivariance (scale + shift) property, which can be detrime… ▽ More In many information processing systems, it may be desirable to ensure that any change of the input, whether by shifting or scaling, results in a corresponding change in the system response. While deep neural networks are gradually replacing all traditional automatic processing methods, they surprisingly do not guarantee such normalization-equivariance (scale + shift) property, which can be detrimental in many applications. To address this issue, we propose a methodology for adapting existing neural networks so that normalization-equivariance holds by design. Our main claim is that not only ordinary convolutional layers, but also all activation functions, including the ReLU (rectified linear unit), which are applied element-wise to the pre-activated neurons, should be completely removed from neural networks and replaced by better conditioned alternatives. To this end, we introduce affine-constrained convolutions and channel-wise sort pooling layers as surrogates and show that these two architectural modifications do preserve normalization-equivariance without loss of performance. Experimental results in image denoising show that normalization-equivariant neural networks, in addition to their better conditioning, also provide much better generalization across noise levels. △ Less

Submitted 21 February, 2024; v1 submitted 8 June, 2023; originally announced June 2023.

Unsupervised Linear and Iterative Combinations of Patches for Image Denoising

Authors: Sébastien Herbreteau, Charles Kervrann

Abstract: We introduce a parametric view of non-local two-step denoisers, for which BM3D is a major representative, where quadratic risk minimization is leveraged for unsupervised optimization. Within this paradigm, we propose to extend the underlying mathematical parametric formulation by iteration. This generalization can be expected to further improve the denoising performance, somehow curbed by the impr… ▽ More We introduce a parametric view of non-local two-step denoisers, for which BM3D is a major representative, where quadratic risk minimization is leveraged for unsupervised optimization. Within this paradigm, we propose to extend the underlying mathematical parametric formulation by iteration. This generalization can be expected to further improve the denoising performance, somehow curbed by the impracticality of repeating the second stage for all two-step denoisers. The resulting formulation involves estimating an even larger amount of parameters in a unsupervised manner which is all the more challenging. Focusing on the parameterized form of NL-Ridge, the simplest but also most efficient non-local two-step denoiser, we propose a progressive scheme to approximate the parameters minimizing the risk. In the end, the denoised images are made up of iterative linear combinations of patches. Experiments on artificially noisy images but also on real-world noisy images demonstrate that our method compares favorably with the very best unsupervised denoisers such as WNNM, outperforming the recent deep-learning-based approaches, while being much faster. △ Less

Submitted 1 December, 2022; originally announced December 2022.

Immersive and Interactive Visualization of 3D Spatio-Temporal Data using a Space Time Hypercube

Authors: Gwendal Fouché, Ferran Argelaguet, Emmanuel Faure, Charles Kervrann

Abstract: We propose an extension of the well-known Space-Time Cube (STC) visualization technique in order to visualize time-varying 3D spatial data, taking advantage of the interaction capabilities of Virtual Reality (VR). The analysis of multidimensional time-varying datasets, which size grows as recording and simulating techniques advance, faces challenges on the representation and visualization of dense… ▽ More We propose an extension of the well-known Space-Time Cube (STC) visualization technique in order to visualize time-varying 3D spatial data, taking advantage of the interaction capabilities of Virtual Reality (VR). The analysis of multidimensional time-varying datasets, which size grows as recording and simulating techniques advance, faces challenges on the representation and visualization of dense data, as well as on the study of temporal variations. First, we propose the Space-Time Hypercube (STH) as an abstraction for 3D temporal data, extended from the STC concept. Second, through the example of embryo development imaging dataset, we detail the construction and visualization of a STC based on a user-driven projection of the spatial and temporal information. This projection yields a 3D STC visualization, which can also encode additional numerical and categorical data. Additionally, we propose a set of tools allowing the user to filter and manipulate the 3D STC which benefits from the visualization, exploration and interaction possibilities offered by VR. Finally, we evaluated the proposed visualization method in the context of the visualization of spatio-temporal biological data. Several biology experts accompanied the application design to provide insight on how the STC visualization could be used to explore such data. We report a user study (n=12) using non-expert users performing a set of exploration and query tasks to evaluate the system. △ Less

Submitted 27 June, 2022; originally announced June 2022.

Comments: 21 1-column pages, 7 figures

Timeline Design Space for Immersive Exploration of Time-Varying Spatial 3D Data

Authors: Gwendal Fouché, Ferran Argelaguet, Emmanuel Faure, Charles Kervrann

Abstract: Timelines are common visualizations to represent and manipulate temporal data, from historical events storytelling to animation authoring. However, timeline visualizations rarely consider spatio-temporal 3D data (e.g. mesh or volumetric models) directly, which are typically explored using 3D visualizers only displaying one time-step at a time. In this paper, leveraging the increased workspace and… ▽ More Timelines are common visualizations to represent and manipulate temporal data, from historical events storytelling to animation authoring. However, timeline visualizations rarely consider spatio-temporal 3D data (e.g. mesh or volumetric models) directly, which are typically explored using 3D visualizers only displaying one time-step at a time. In this paper, leveraging the increased workspace and 3D interaction capabilities of virtual reality, we propose to use timelines for the visualization of 3D temporal data to support exploration and analysis. First, we propose a timeline design space for 3D temporal data extending the timeline design space proposed by Brehmer et al. The proposed design space adapts the scale, layout and representation dimensions to account for the depth dimension and how 3D temporal data can be partitioned and structured. In our approach, an additional dimension is introduced, the support, which further characterizes the 3D dimension of the visualization. To complement the design space and the interaction capabilities of VR systems, we discuss the interaction methods required for the efficient visualization of 3D timelines. Then, to evaluate the benefits of 3D timelines, we conducted a formal evaluation with two main objectives: comparing the proposed visualization with a traditional visualization method; exploring how users interact with different 3D timeline designs. Our results showed that time-related tasks can be achieved more comfortably using timelines, and more efficiently for specific tasks requiring the analysis of the surrounding temporal context. Though the comparison between the different timeline designs were inconclusive, participants reported a clear preference towards the timeline design that did not occupy the vertical space. Finally, we illustrate the use of the 3D timelines to a real use-case on the analysis of biological 3D temporal datasets. △ Less

Submitted 20 June, 2022; originally announced June 2022.

Comments: 11 pages, 6 figures

Towards a unified view of unsupervised non-local methods for image denoising: the NL-Ridge approach

Authors: Sébastien Herbreteau, Charles Kervrann

Abstract: We propose a unified view of unsupervised non-local methods for image denoising that linearily combine noisy image patches. The best methods, established in different modeling and estimation frameworks, are two-step algorithms. Leveraging Stein's unbiased risk estimate (SURE) for the first step and the "internal adaptation", a concept borrowed from deep learning theory, for the second one, we show… ▽ More We propose a unified view of unsupervised non-local methods for image denoising that linearily combine noisy image patches. The best methods, established in different modeling and estimation frameworks, are two-step algorithms. Leveraging Stein's unbiased risk estimate (SURE) for the first step and the "internal adaptation", a concept borrowed from deep learning theory, for the second one, we show that our NL-Ridge approach enables to reconcile several patch aggregation methods for image denoising. In the second step, our closed-form aggregation weights are computed through multivariate Ridge regressions. Experiments on artificially noisy images demonstrate that NL-Ridge may outperform well established state-of-the-art unsupervised denoisers such as BM3D and NL-Bayes, as well as recent unsupervised deep learning methods, while being simpler conceptually. △ Less

Submitted 1 March, 2022; originally announced March 2022.

DCT2net: an interpretable shallow CNN for image denoising

Authors: Sébastien Herbreteau, Charles Kervrann

Abstract: This work tackles the issue of noise removal from images, focusing on the well-known DCT image denoising algorithm. The latter, stemming from signal processing, has been well studied over the years. Though very simple, it is still used in crucial parts of state-of-the-art "traditional" denoising algorithms such as BM3D. Since a few years however, deep convolutional neural networks (CNN) have outpe… ▽ More This work tackles the issue of noise removal from images, focusing on the well-known DCT image denoising algorithm. The latter, stemming from signal processing, has been well studied over the years. Though very simple, it is still used in crucial parts of state-of-the-art "traditional" denoising algorithms such as BM3D. Since a few years however, deep convolutional neural networks (CNN) have outperformed their traditional counterparts, making signal processing methods less attractive. In this paper, we demonstrate that a DCT denoiser can be seen as a shallow CNN and thereby its original linear transform can be tuned through gradient descent in a supervised manner, improving considerably its performance. This gives birth to a fully interpretable CNN called DCT2net. To deal with remaining artifacts induced by DCT2net, an original hybrid solution between DCT and DCT2net is proposed combining the best that these two methods can offer; DCT2net is selected to process non-stationary image patches while DCT is optimal for piecewise smooth patches. Experiments on artificially noisy images demonstrate that two-layer DCT2net provides comparable results to BM3D and is as fast as DnCNN algorithm composed of more than a dozen of layers. △ Less

Submitted 31 July, 2021; originally announced July 2021.

Aggregation of local parametric candidates with exemplar-based occlusion handling for optical flow

Authors: Denis Fortun, Patrick Bouthemy, Charles Kervrann

Abstract: Handling all together large displacements, motion details and occlusions remains an open issue for reliable computation of optical flow in a video sequence. We propose a two-step aggregation paradigm to address this problem. The idea is to supply local motion candidates at every pixel in a first step, and then to combine them to determine the global optical flow field in a second step. We exploit… ▽ More Handling all together large displacements, motion details and occlusions remains an open issue for reliable computation of optical flow in a video sequence. We propose a two-step aggregation paradigm to address this problem. The idea is to supply local motion candidates at every pixel in a first step, and then to combine them to determine the global optical flow field in a second step. We exploit local parametric estimations combined with patch correspondences and we experimentally demonstrate that they are sufficient to produce highly accurate motion candidates. The aggregation step is designed as the discrete optimization of a global regularized energy. The occlusion map is estimated jointly with the flow field throughout the two steps. We propose a generic exemplar-based approach for occlusion filling with motion vectors. We achieve state-of-the-art results in computer vision benchmarks, with particularly significant improvements in the case of large displacements and occlusions. △ Less

Submitted 22 July, 2014; originally announced July 2014.

Comments: Submission,IEEE Transactions on Image Processing (2014)