-
Measuring arousal and stress physiology on Esports, a League of Legends case study
Authors:
David Berga,
Alexandre Pereda,
Eleonora De Filippi,
Arijit Nandi,
Eulalia Febrer,
Marta Reverte,
Lautaro Russo
Abstract:
Esports gaming is an area in which videogame players need to cooperate and compete with each other, influencing their cognitive load, processing, stress, and social skills. Here it is unknown to which extent competitive videogame play using a desktop setting can affect the physiological responses of players' autonomic nervous system. For such, we propose a study where we have measured distinct ele…
▽ More
Esports gaming is an area in which videogame players need to cooperate and compete with each other, influencing their cognitive load, processing, stress, and social skills. Here it is unknown to which extent competitive videogame play using a desktop setting can affect the physiological responses of players' autonomic nervous system. For such, we propose a study where we have measured distinct electrodermal and cardiac activity metrics over competitive players during several League of Legends gameplay sessions in a Esports stadium. We mainly found that game performance (whether winning or losing the game) significantly affects both electrodermal and cardiac activity, where players who lost the game showed higher stress-related physiological responses, as compared to winning players. We also found that important specific in-game events such as "Killing", "Dying" or "Destroying Turret" significantly increased both electrodermal and cardiac activity over players more than other less-relevant events such as "Placing Wards" or "Destroying Turret Plates". Finally, by analyzing activity over player roles we found different trends of activity on these measurements, this could foster the exploration on human physiology with a higher set of participants in future Esports studies.
△ Less
Submitted 22 May, 2023; v1 submitted 27 February, 2023;
originally announced February 2023.
-
IQUAFLOW: A new framework to measure image quality
Authors:
P. Gallés,
K. Takats,
M. Hernández-Cabronero,
D. Berga,
L. Pega,
L. Riordan-Chen,
C. Garcia,
G. Becker,
A. Garriga,
A. Bukva,
J. Serra-Sagristà,
D. Vilaseca,
J. Marín
Abstract:
IQUAFLOW is a new image quality framework that provides a set of tools to assess image quality. The user can add custom metrics that can be easily integrated. Furthermore, iquaflow allows to measure quality by using the performance of AI models trained on the images as a proxy. This also helps to easily make studies of performance degradation of several modifications of the original dataset, for i…
▽ More
IQUAFLOW is a new image quality framework that provides a set of tools to assess image quality. The user can add custom metrics that can be easily integrated. Furthermore, iquaflow allows to measure quality by using the performance of AI models trained on the images as a proxy. This also helps to easily make studies of performance degradation of several modifications of the original dataset, for instance, with images reconstructed after different levels of lossy compression; satellite images would be a use case example, since they are commonly compressed before downloading to the ground. In this situation, the optimization problem consists in finding the smallest images that provide yet sufficient quality to meet the required performance of the deep learning algorithms. Thus, a study with iquaflow is suitable for such case. All this development is wrapped in Mlflow: an interactive tool used to visualize and summarize the results. This document describes different use cases and provides links to their respective repositories. To ease the creation of new studies, we include a cookie-cutter repository. The source code, issue tracker and aforementioned repositories are all hosted on GitHub https://github.com/satellogic/iquaflow.
△ Less
Submitted 24 October, 2022;
originally announced October 2022.
-
QMRNet: Quality Metric Regression for EO Image Quality Assessment and Super-Resolution
Authors:
David Berga,
Pau Gallés,
Katalin Takáts,
Eva Mohedano,
Laura Riordan-Chen,
Clara Garcia-Moll,
David Vilaseca,
Javier Marín
Abstract:
Latest advances in Super-Resolution (SR) have been tested with general purpose images such as faces, landscapes and objects, mainly unused for the task of super-resolving Earth Observation (EO) images. In this research paper, we benchmark state-of-the-art SR algorithms for distinct EO datasets using both Full-Reference and No-Reference Image Quality Assessment (IQA) metrics. We also propose a nove…
▽ More
Latest advances in Super-Resolution (SR) have been tested with general purpose images such as faces, landscapes and objects, mainly unused for the task of super-resolving Earth Observation (EO) images. In this research paper, we benchmark state-of-the-art SR algorithms for distinct EO datasets using both Full-Reference and No-Reference Image Quality Assessment (IQA) metrics. We also propose a novel Quality Metric Regression Network (QMRNet) that is able to predict quality (as a No-Reference metric) by training on any property of the image (i.e. its resolution, its distortions...) and also able to optimize SR algorithms for a specific metric objective. This work is part of the implementation of the framework IQUAFLOW which has been developed for evaluating image quality, detection and classification of objects as well as image compression in EO use cases. We integrated our experimentation and tested our QMRNet algorithm on predicting features like blur, sharpness, snr, rer and ground sampling distance (GSD) and obtain validation medRs below 1.0 (out of N=50) and recall rates above 95\%. Overall benchmark shows promising results for LIIF, CAR and MSRN and also the potential use of QMRNet as Loss for optimizing SR predictions. Due to its simplicity, QMRNet could also be used for other use cases and image domains, as its architecture and data processing is fully scalable.
△ Less
Submitted 14 October, 2022; v1 submitted 12 October, 2022;
originally announced October 2022.
-
Saliency for free: Saliency prediction as a side-effect of object recognition
Authors:
Carola Figueroa-Flores,
David Berga,
Joost van der Weijer,
Bogdan Raducanu
Abstract:
Saliency is the perceptual capacity of our visual system to focus our attention (i.e. gaze) on relevant objects. Neural networks for saliency estimation require ground truth saliency maps for training which are usually achieved via eyetracking experiments. In the current paper, we demonstrate that saliency maps can be generated as a side-effect of training an object recognition deep neural network…
▽ More
Saliency is the perceptual capacity of our visual system to focus our attention (i.e. gaze) on relevant objects. Neural networks for saliency estimation require ground truth saliency maps for training which are usually achieved via eyetracking experiments. In the current paper, we demonstrate that saliency maps can be generated as a side-effect of training an object recognition deep neural network that is endowed with a saliency branch. Such a network does not require any ground-truth saliency maps for training.Extensive experiments carried out on both real and synthetic saliency datasets demonstrate that our approach is able to generate accurate saliency maps, achieving competitive results on both synthetic and real datasets when compared to methods that do require ground truth data.
△ Less
Submitted 20 July, 2021;
originally announced July 2021.
-
Hallucinating Saliency Maps for Fine-Grained Image Classification for Limited Data Domains
Authors:
Carola Figueroa-Flores,
Bogdan Raducanu,
David Berga,
Joost van de Weijer
Abstract:
Most of the saliency methods are evaluated on their ability to generate saliency maps, and not on their functionality in a complete vision pipeline, like for instance, image classification. In the current paper, we propose an approach which does not require explicit saliency maps to improve image classification, but they are learned implicitely, during the training of an end-to-end image classific…
▽ More
Most of the saliency methods are evaluated on their ability to generate saliency maps, and not on their functionality in a complete vision pipeline, like for instance, image classification. In the current paper, we propose an approach which does not require explicit saliency maps to improve image classification, but they are learned implicitely, during the training of an end-to-end image classification task. We show that our approach obtains similar results as the case when the saliency maps are provided explicitely. Combining RGB data with saliency maps represents a significant advantage for object recognition, especially for the case when training data is limited. We validate our method on several datasets for fine-grained classification tasks (Flowers, Birds and Cars). In addition, we show that our saliency estimation method, which is trained without any saliency groundtruth data, obtains competitive results on real image saliency benchmark (Toronto), and outperforms deep saliency models with synthetic images (SID4VAM).
△ Less
Submitted 3 February, 2021; v1 submitted 24 July, 2020;
originally announced July 2020.
-
Disentanglement of Color and Shape Representations for Continual Learning
Authors:
David Berga,
Marc Masana,
Joost Van de Weijer
Abstract:
We hypothesize that disentangled feature representations suffer less from catastrophic forgetting. As a case study we perform explicit disentanglement of color and shape, by adjusting the network architecture. We tested classification accuracy and forgetting in a task-incremental setting with Oxford-102 Flowers dataset. We combine our method with Elastic Weight Consolidation, Learning without Forg…
▽ More
We hypothesize that disentangled feature representations suffer less from catastrophic forgetting. As a case study we perform explicit disentanglement of color and shape, by adjusting the network architecture. We tested classification accuracy and forgetting in a task-incremental setting with Oxford-102 Flowers dataset. We combine our method with Elastic Weight Consolidation, Learning without Forgetting, Synaptic Intelligence and Memory Aware Synapses, and show that feature disentanglement positively impacts continual learning performance.
△ Less
Submitted 13 July, 2020;
originally announced July 2020.
-
MineGAN: effective knowledge transfer from GANs to target domains with few images
Authors:
Yaxing Wang,
Abel Gonzalez-Garcia,
David Berga,
Luis Herranz,
Fahad Shahbaz Khan,
Joost van de Weijer
Abstract:
One of the attractive characteristics of deep neural networks is their ability to transfer knowledge obtained in one domain to other related domains. As a result, high-quality networks can be trained in domains with relatively little training data. This property has been extensively studied for discriminative networks but has received significantly less attention for generative models. Given the o…
▽ More
One of the attractive characteristics of deep neural networks is their ability to transfer knowledge obtained in one domain to other related domains. As a result, high-quality networks can be trained in domains with relatively little training data. This property has been extensively studied for discriminative networks but has received significantly less attention for generative models. Given the often enormous effort required to train GANs, both computationally as well as in the dataset collection, the re-use of pretrained GANs is a desirable objective. We propose a novel knowledge transfer method for generative models based on mining the knowledge that is most beneficial to a specific target domain, either from a single or multiple pretrained GANs. This is done using a miner network that identifies which part of the generative distribution of each pretrained GAN outputs samples closest to the target domain. Mining effectively steers GAN sampling towards suitable regions of the latent space, which facilitates the posterior finetuning and avoids pathologies of other methods such as mode collapse and lack of flexibility. We perform experiments on several complex datasets using various GAN architectures (BigGAN, Progressive GAN) and show that the proposed method, called MineGAN, effectively transfers knowledge to domains with few target images, outperforming existing methods. In addition, MineGAN can successfully transfer knowledge from multiple pretrained GANs. Our code is available at: https://github.com/yaxingwang/MineGAN.
△ Less
Submitted 2 April, 2020; v1 submitted 11 December, 2019;
originally announced December 2019.
-
SID4VAM: A Benchmark Dataset with Synthetic Images for Visual Attention Modeling
Authors:
David Berga,
Xosé R. Fdez-Vidal,
Xavier Otazu,
Xosé M. Pardo
Abstract:
A benchmark of saliency models performance with a synthetic image dataset is provided. Model performance is evaluated through saliency metrics as well as the influence of model inspiration and consistency with human psychophysics. SID4VAM is composed of 230 synthetic images, with known salient regions. Images were generated with 15 distinct types of low-level features (e.g. orientation, brightness…
▽ More
A benchmark of saliency models performance with a synthetic image dataset is provided. Model performance is evaluated through saliency metrics as well as the influence of model inspiration and consistency with human psychophysics. SID4VAM is composed of 230 synthetic images, with known salient regions. Images were generated with 15 distinct types of low-level features (e.g. orientation, brightness, color, size...) with a target-distractor pop-out type of synthetic patterns. We have used Free-Viewing and Visual Search task instructions and 7 feature contrasts for each feature category. Our study reveals that state-of-the-art Deep Learning saliency models do not perform well with synthetic pattern images, instead, models with Spectral/Fourier inspiration outperform others in saliency metrics and are more consistent with human psychophysical experimentation. This study proposes a new way to evaluate saliency models in the forthcoming literature, accounting for synthetic images with uniquely low-level feature contexts, distinct from previous eye tracking image datasets.
△ Less
Submitted 28 October, 2019;
originally announced October 2019.
-
Modeling Bottom-Up and Top-Down Attention with a Neurodynamic Model of V1
Authors:
David Berga,
Xavier Otazu
Abstract:
Previous studies suggested that lateral interactions of V1 cells are responsible, among other visual effects, of bottom-up visual attention (alternatively named visual salience or saliency). Our objective is to mimic these connections with a neurodynamic network of firing-rate neurons in order to predict visual attention. Early visual subcortical processes (i.e. retinal and thalamic) are functiona…
▽ More
Previous studies suggested that lateral interactions of V1 cells are responsible, among other visual effects, of bottom-up visual attention (alternatively named visual salience or saliency). Our objective is to mimic these connections with a neurodynamic network of firing-rate neurons in order to predict visual attention. Early visual subcortical processes (i.e. retinal and thalamic) are functionally simulated. An implementation of the cortical magnification function is included to define the retinotopical projections towards V1, processing neuronal activity for each distinct view during scene observation. Novel computational definitions of top-down inhibition (in terms of inhibition of return and selection mechanisms), are also proposed to predict attention in Free-Viewing and Visual Search tasks. Results show that our model outpeforms other biologically-inpired models of saliency prediction while predicting visual saccade sequences with the same model. We also show how temporal and spatial characteristics of inhibition of return can improve prediction of saccades, as well as how distinct search strategies (in terms of feature-selective or category-specific inhibition) can predict attention at distinct image contexts.
△ Less
Submitted 18 November, 2019; v1 submitted 4 April, 2019;
originally announced April 2019.
-
Psychophysical evaluation of individual low-level feature influences on visual attention
Authors:
David Berga,
Xosé Ramón Fdez-Vidal,
Xavier Otazu,
Víctor Leborán,
Xosé M. Pardo
Abstract:
In this study we provide the analysis of eye movement behavior elicited by low-level feature distinctiveness with a dataset of synthetically-generated image patterns. Design of visual stimuli was inspired by the ones used in previous psychophysical experiments, namely in free-viewing and visual searching tasks, to provide a total of 15 types of stimuli, divided according to the task and feature to…
▽ More
In this study we provide the analysis of eye movement behavior elicited by low-level feature distinctiveness with a dataset of synthetically-generated image patterns. Design of visual stimuli was inspired by the ones used in previous psychophysical experiments, namely in free-viewing and visual searching tasks, to provide a total of 15 types of stimuli, divided according to the task and feature to be analyzed. Our interest is to analyze the influences of low-level feature contrast between a salient region and the rest of distractors, providing fixation localization characteristics and reaction time of landing inside the salient region. Eye-tracking data was collected from 34 participants during the viewing of a 230 images dataset. Results show that saliency is predominantly and distinctively influenced by: 1. feature type, 2. feature contrast, 3. temporality of fixations, 4. task difficulty and 5. center bias. This experimentation proposes a new psychophysical basis for saliency model evaluation using synthetic images.
△ Less
Submitted 15 November, 2018;
originally announced November 2018.
-
A Neurodynamic model of Saliency prediction in V1
Authors:
David Berga,
Xavier Otazu
Abstract:
Lateral connections in the primary visual cortex (V1) have long been hypothesized to be responsible of several visual processing mechanisms such as brightness induction, chromatic induction, visual discomfort and bottom-up visual attention (also named saliency). Many computational models have been developed to independently predict these and other visual processes, but no computational model has b…
▽ More
Lateral connections in the primary visual cortex (V1) have long been hypothesized to be responsible of several visual processing mechanisms such as brightness induction, chromatic induction, visual discomfort and bottom-up visual attention (also named saliency). Many computational models have been developed to independently predict these and other visual processes, but no computational model has been able to reproduce all of them simultaneously. In this work we show that a biologically plausible computational model of lateral interactions of V1 is able to simultaneously predict saliency and all the aforementioned visual processes. Our model's (NSWAM) architecture is based on Pennachio's neurodynamic model of lateral connections of V1. It is defined as a network of firing rate neurons, sensitive to visual features such as brightness, color, orientation and scale. We tested NSWAM saliency predictions using images from several eye tracking datasets. We show that accuracy of predictions, using shuffled metrics, obtained by our architecture is similar to other state-of-the-art computational methods, particularly with synthetic images (CAT2000-Pattern & SID4VAM) which mainly contain low level features. Moreover, we outperform other biologically-inspired saliency models that are specifically designed to exclusively reproduce saliency. Hence, we show that our biologically plausible model of lateral connections can simultaneously explain different visual proceses present in V1 (without applying any type of training or optimization and kee** the same parametrization for all the visual processes). This can be useful for the definition of a unified architecture of the primary visual cortex.
△ Less
Submitted 18 September, 2020; v1 submitted 15 November, 2018;
originally announced November 2018.
