Saliency Enhancement using Gradient Domain Edges Merging
Authors:
Dominique Beaini,
Sofiane Achiche,
Alexandre Duperre,
Maxime Raison
Abstract:
In recent years, there has been a rapid progress in solving the binary problems in computer vision, such as edge detection which finds the boundaries of an image and salient object detection which finds the important object in an image. This progress happened thanks to the rise of deep-learning and convolutional neural networks (CNN) which allow to extract complex and abstract features. However, e…
▽ More
In recent years, there has been a rapid progress in solving the binary problems in computer vision, such as edge detection which finds the boundaries of an image and salient object detection which finds the important object in an image. This progress happened thanks to the rise of deep-learning and convolutional neural networks (CNN) which allow to extract complex and abstract features. However, edge detection and saliency are still two different fields and do not interact together, although it is intuitive for a human to detect salient objects based on its boundaries. Those features are not well merged in a CNN because edges and surfaces do not intersect since one feature represents a region while the other represents boundaries between different regions. In the current work, the main objective is to develop a method to merge the edges with the saliency maps to improve the performance of the saliency. Hence, we developed the gradient-domain merging (GDM) which can be used to quickly combine the image-domain information of salient object detection with the gradient-domain information of the edge detection. This leads to our proposed saliency enhancement using edges (SEE) with an average improvement of the F-measure of at least 3.4 times higher on the DUT-OMRON dataset and 6.6 times higher on the ECSSD dataset, when compared to competing algorithm such as denseCRF and BGOF. The SEE algorithm is split into 2 parts, SEE-Pre for preprocessing and SEE-Post pour postprocessing.
△ Less
Submitted 11 February, 2020;
originally announced February 2020.
Deep Green Function Convolution for Improving Saliency in Convolutional Neural Networks
Authors:
Dominique Beaini,
Sofiane Achiche,
Alexandre Duperré,
Maxime Raison
Abstract:
Current saliency methods require to learn large scale regional features using small convolutional kernels, which is not possible with a simple feed-forward network. Some methods solve this problem by using segmentation into superpixels while others downscale the image through the network and rescale it back to its original size. The objective of this paper is to show that saliency convolutional ne…
▽ More
Current saliency methods require to learn large scale regional features using small convolutional kernels, which is not possible with a simple feed-forward network. Some methods solve this problem by using segmentation into superpixels while others downscale the image through the network and rescale it back to its original size. The objective of this paper is to show that saliency convolutional neural networks (CNN) can be improved by using a Green's function convolution (GFC) to extrapolate edges features into salient regions. The GFC acts as a gradient integrator, allowing to produce saliency features by filling thin edges directly inside the CNN. Hence, we propose the gradient integration and sum (GIS) layer that combines the edges features with the saliency features. Using the HED and DSS architecture, we demonstrated that adding a GIS layer near the network's output allows to reduce the sensitivity to the parameter initialization, to reduce the overfitting and to improve the repeatability of the training. By simply adding a GIS layer to the state-of-the-art DSS model, there is an absolute increase of 1.6% for the F-measure on the DUT-OMRON dataset, with only 10ms of additional computation time. The GIS layer further allows the network to perform significantly better in the case of highly noisy images or low-brightness images. In fact, we observed an F-measure improvement of 5.2% when noise was added to the dataset and 2.8% when the brightness was reduced. Since the GIS layer is model agnostic, it can be implemented into different fully convolutional networks. A major contribution of the current work is the first implementation of Green's function convolution inside a neural network, which allows the network to operate in the feature domain and in the gradient domain at the same time, thus improving the regional representation via edge filling.
△ Less
Submitted 14 November, 2019; v1 submitted 22 August, 2019;
originally announced August 2019.
