-
Overt visual attention on rendered 3D objects
Authors:
Oleksii Sidorov,
Joshua S. Harvey,
Hannah E. Smithson,
Jon Y. Hardeberg
Abstract:
This work covers multiple aspects of overt visual attention on 3D renders: measurement, projection, visualization, and application to studying the influence of material appearance on looking behaviour. In the scope of this work, we ran an eye-tracking experiment in which the observers are presented with animations of rotating 3D objects. The objects were rendered to simulate different metallic app…
▽ More
This work covers multiple aspects of overt visual attention on 3D renders: measurement, projection, visualization, and application to studying the influence of material appearance on looking behaviour. In the scope of this work, we ran an eye-tracking experiment in which the observers are presented with animations of rotating 3D objects. The objects were rendered to simulate different metallic appearance, particularly smooth (glossy), rough (matte), and coated gold. The eye-tracking results illustrate how material appearance itself influences the observer's attention, while all the other parameters remain unchanged. In order to make visualization of the attention maps more natural and also make the analysis more accurate, we develop a novel technique of projection of gaze fixations on the 3D surface of the figure itself, instead of the conventional 2D plane of the screen. The proposed methodology will be useful for further studies of attention and saliency in the computer graphics domain.
△ Less
Submitted 24 May, 2019;
originally announced May 2019.
-
Craquelure as a Graph: Application of Image Processing and Graph Neural Networks to the Description of Fracture Patterns
Authors:
Oleksii Sidorov,
Jon Yngve Hardeberg
Abstract:
Cracks on a painting is not a defect but an inimitable signature of an artwork which can be used for origin examination, aging monitoring, damage identification, and even forgery detection. This work presents the development of a new methodology and corresponding toolbox for the extraction and characterization of information from an image of a craquelure pattern.
The proposed approach processes…
▽ More
Cracks on a painting is not a defect but an inimitable signature of an artwork which can be used for origin examination, aging monitoring, damage identification, and even forgery detection. This work presents the development of a new methodology and corresponding toolbox for the extraction and characterization of information from an image of a craquelure pattern.
The proposed approach processes craquelure network as a graph. The graph representation captures the network structure via mutual organization of junctions and fractures. Furthermore, it is invariant to any geometrical distortions. At the same time, our tool extracts the properties of each node and edge individually, which allows to characterize the pattern statistically.
We illustrate benefits from the graph representation and statistical features individually using novel Graph Neural Network and hand-crafted descriptors correspondingly. However, we also show that the best performance is achieved when both techniques are merged into one framework. We perform experiments on the dataset for paintings' origin classification and demonstrate that our approach outperforms existing techniques by a large margin.
△ Less
Submitted 4 December, 2019; v1 submitted 13 May, 2019;
originally announced May 2019.
-
Hyperspectral Calibration of Art: Acquisition and Calibration Workflows
Authors:
Ruven Pillay,
Jon Y Hardeberg,
Sony George
Abstract:
Hyperspectral imaging has become an increasingly used tool in the analysis of works of art. However, the quality of the acquired data and the processing of that data to produce accurate and reproducible spectral image cubes can be a challenge to many cultural heritage users. The calibration of data that is both spectrally and spatially accurate is an essential step in order to obtain useful and re…
▽ More
Hyperspectral imaging has become an increasingly used tool in the analysis of works of art. However, the quality of the acquired data and the processing of that data to produce accurate and reproducible spectral image cubes can be a challenge to many cultural heritage users. The calibration of data that is both spectrally and spatially accurate is an essential step in order to obtain useful and relevant results from hyperspectral imaging. Data that is too noisy or inaccurate will produce sub-optimal results when used for pigment map**, the detection of hidden features, change detection or for quantitative spectral documentation. To help address this, therefore, we will examine the specific acquisition and calibration workflows necessary for works of art. These workflows includes the key parameters that must be addressed during acquisition and the essential steps and issues at each of the stages required during post-processing in order to fully calibrate hyperspectral data. In addition we will look in detail at the key issues that affect data quality and propose practical solutions that can make significant differences to overall hyperspectral image quality.
△ Less
Submitted 11 March, 2019;
originally announced March 2019.
-
Deep Hyperspectral Prior: Denoising, Inpainting, Super-Resolution
Authors:
Oleksii Sidorov,
Jon Yngve Hardeberg
Abstract:
Deep learning algorithms have demonstrated state-of-the-art performance in various tasks of image restoration. This was made possible through the ability of CNNs to learn from large exemplar sets. However, the latter becomes an issue for hyperspectral image processing where datasets commonly consist of just a few images. In this work, we propose a new approach to denoising, inpainting, and super-r…
▽ More
Deep learning algorithms have demonstrated state-of-the-art performance in various tasks of image restoration. This was made possible through the ability of CNNs to learn from large exemplar sets. However, the latter becomes an issue for hyperspectral image processing where datasets commonly consist of just a few images. In this work, we propose a new approach to denoising, inpainting, and super-resolution of hyperspectral image data using intrinsic properties of a CNN without any training. The performance of the given algorithm is shown to be comparable to the performance of trained networks, while its application is not restricted by the availability of training data. This work is an extension of original "deep prior" algorithm to HSI domain and 3D-convolutional networks.
△ Less
Submitted 4 December, 2019; v1 submitted 1 February, 2019;
originally announced February 2019.
-
Ensemble of Convolutional Neural Networks for Dermoscopic Images Classification
Authors:
Tomáš Majtner,
Buda Bajić,
Sule Yildirim,
Jon Yngve Hardeberg,
Joakim Lindblad,
Nataša Sladoje
Abstract:
In this report, we are presenting our automated prediction system for disease classification within dermoscopic images. The proposed solution is based on deep learning, where we employed transfer learning strategy on VGG16 and GoogLeNet architectures. The key feature of our solution is preprocessing based primarily on image augmentation and colour normalization. The solution was evaluated on Task…
▽ More
In this report, we are presenting our automated prediction system for disease classification within dermoscopic images. The proposed solution is based on deep learning, where we employed transfer learning strategy on VGG16 and GoogLeNet architectures. The key feature of our solution is preprocessing based primarily on image augmentation and colour normalization. The solution was evaluated on Task 3: Lesion Diagnosis of the ISIC 2018: Skin Lesion Analysis Towards Melanoma Detection.
△ Less
Submitted 15 August, 2018;
originally announced August 2018.
-
A bag-to-class divergence approach to multiple-instance learning
Authors:
Kajsa Møllersen,
Jon Yngve Hardeberg,
Fred Godtliebsen
Abstract:
In multi-instance (MI) learning, each object (bag) consists of multiple feature vectors (instances), and is most commonly regarded as a set of points in a multidimensional space. A different viewpoint is that the instances are realisations of random vectors with corresponding probability distribution, and that a bag is the distribution, not the realisations. In MI classification, each bag in the t…
▽ More
In multi-instance (MI) learning, each object (bag) consists of multiple feature vectors (instances), and is most commonly regarded as a set of points in a multidimensional space. A different viewpoint is that the instances are realisations of random vectors with corresponding probability distribution, and that a bag is the distribution, not the realisations. In MI classification, each bag in the training set has a class label, but the instances are unlabelled. By introducing the probability distribution space to bag-level classification problems, dissimilarities between probability distributions (divergences) can be applied. The bag-to-bag Kullback-Leibler information is asymptotically the best classifier, but the typical sparseness of MI training sets is an obstacle. We introduce bag-to-class divergence to MI learning, emphasising the hierarchical nature of the random vectors that makes bags from the same class different. We propose two properties for bag-to-class divergences, and an additional property for sparse training sets.
△ Less
Submitted 12 October, 2018; v1 submitted 7 March, 2018;
originally announced March 2018.
