-
3R-INN: How to be climate friendly while consuming/delivering videos?
Authors:
Zoubida Ameur,
Claire-Hélène Demarty,
Daniel Menard,
Olivier Le Meur
Abstract:
The consumption of a video requires a considerable amount of energy during the various stages of its life-cycle. With a billion hours of video consumed daily, this contributes significantly to the greenhouse gas emission. Therefore, reducing the end-to-end carbon footprint of the video chain, while preserving the quality of experience at the user side, is of high importance. To contribute in an im…
▽ More
The consumption of a video requires a considerable amount of energy during the various stages of its life-cycle. With a billion hours of video consumed daily, this contributes significantly to the greenhouse gas emission. Therefore, reducing the end-to-end carbon footprint of the video chain, while preserving the quality of experience at the user side, is of high importance. To contribute in an impactful manner, we propose 3R-INN, a single light invertible network that does three tasks at once: given a high-resolution grainy image, it Rescales it to a lower resolution, Removes film grain and Reduces its power consumption when displayed. Providing such a minimum viable quality content contributes to reducing the energy consumption during encoding, transmission, decoding and display. 3R-INN also offers the possibility to restore either the high-resolution grainy original image or a grain-free version, thanks to its invertibility and the disentanglement of the high frequency, and without transmitting auxiliary data. Experiments show that, while enabling significant energy savings for encoding (78%), decoding (77%) and rendering (5% to 20%), 3R-INN outperforms state-of-the-art film grain synthesis and energy-aware methods and achieves state-of-the-art performance on the rescaling task on different test-sets.
△ Less
Submitted 18 March, 2024;
originally announced March 2024.
-
Overview of The MediaEval 2022 Predicting Video Memorability Task
Authors:
Lorin Sweeney,
Mihai Gabriel Constantin,
Claire-Hélène Demarty,
Camilo Fosco,
Alba G. Seco de Herrera,
Sebastian Halder,
Graham Healy,
Bogdan Ionescu,
Ana Matran-Fernandez,
Alan F. Smeaton,
Mushfika Sultana
Abstract:
This paper describes the 5th edition of the Predicting Video Memorability Task as part of MediaEval2022. This year we have reorganised and simplified the task in order to lubricate a greater depth of inquiry. Similar to last year, two datasets are provided in order to facilitate generalisation, however, this year we have replaced the TRECVid2019 Video-to-Text dataset with the VideoMem dataset in o…
▽ More
This paper describes the 5th edition of the Predicting Video Memorability Task as part of MediaEval2022. This year we have reorganised and simplified the task in order to lubricate a greater depth of inquiry. Similar to last year, two datasets are provided in order to facilitate generalisation, however, this year we have replaced the TRECVid2019 Video-to-Text dataset with the VideoMem dataset in order to remedy underlying data quality issues, and to prioritise short-term memorability prediction by elevating the Memento10k dataset as the primary dataset. Additionally, a fully fledged electroencephalography (EEG)-based prediction sub-task is introduced. In this paper, we outline the core facets of the task and its constituent sub-tasks; describing the datasets, evaluation metrics, and requirements for participant submissions.
△ Less
Submitted 13 December, 2022;
originally announced December 2022.
-
Experiences from the MediaEval Predicting Media Memorability Task
Authors:
Alba García Deco de Herrera,
Mihai Gabriel Constantin,
Chaire-Hélène Demarty,
Camilo Fosco,
Sebastian Halder,
Graham Healy,
Bogdan Ionescu,
Ana Matran-Fernandez,
Alan F. Smeaton,
Mushfika Sultana,
Lorin Sweeney
Abstract:
The Predicting Media Memorability task in the MediaEval evaluation campaign has been running annually since 2018 and several different tasks and data sets have been used in this time. This has allowed us to compare the performance of many memorability prediction techniques on the same data and in a reproducible way and to refine and improve on those techniques. The resources created to compute med…
▽ More
The Predicting Media Memorability task in the MediaEval evaluation campaign has been running annually since 2018 and several different tasks and data sets have been used in this time. This has allowed us to compare the performance of many memorability prediction techniques on the same data and in a reproducible way and to refine and improve on those techniques. The resources created to compute media memorability are now being used by researchers well beyond the actual evaluation campaign. In this paper we present a summary of the task, including the collective lessons we have learned for the research community.
△ Less
Submitted 7 December, 2022;
originally announced December 2022.
-
Overview of The MediaEval 2021 Predicting Media Memorability Task
Authors:
Rukiye Savran Kiziltepe,
Mihai Gabriel Constantin,
Claire-Helene Demarty,
Graham Healy,
Camilo Fosco,
Alba Garcia Seco de Herrera,
Sebastian Halder,
Bogdan Ionescu,
Ana Matran-Fernandez,
Alan F. Smeaton,
Lorin Sweeney
Abstract:
This paper describes the MediaEval 2021 Predicting Media Memorability}task, which is in its 4th edition this year, as the prediction of short-term and long-term video memorability remains a challenging task. In 2021, two datasets of videos are used: first, a subset of the TRECVid 2019 Video-to-Text dataset; second, the Memento10K dataset in order to provide opportunities to explore cross-dataset g…
▽ More
This paper describes the MediaEval 2021 Predicting Media Memorability}task, which is in its 4th edition this year, as the prediction of short-term and long-term video memorability remains a challenging task. In 2021, two datasets of videos are used: first, a subset of the TRECVid 2019 Video-to-Text dataset; second, the Memento10K dataset in order to provide opportunities to explore cross-dataset generalisation. In addition, an Electroencephalography (EEG)-based prediction pilot subtask is introduced. In this paper, we outline the main aspects of the task and describe the datasets, evaluation metrics, and requirements for participants' submissions.
△ Less
Submitted 11 December, 2021;
originally announced December 2021.
-
An Annotated Video Dataset for Computing Video Memorability
Authors:
Rukiye Savran Kiziltepe,
Lorin Sweeney,
Mihai Gabriel Constantin,
Faiyaz Doctor,
Alba Garcia Seco de Herrera,
Claire-Helene Demarty,
Graham Healy,
Bogdan Ionescu,
Alan F. Smeaton
Abstract:
Using a collection of publicly available links to short form video clips of an average of 6 seconds duration each, 1,275 users manually annotated each video multiple times to indicate both long-term and short-term memorability of the videos. The annotations were gathered as part of an online memory game and measured a participant's ability to recall having seen the video previously when shown a co…
▽ More
Using a collection of publicly available links to short form video clips of an average of 6 seconds duration each, 1,275 users manually annotated each video multiple times to indicate both long-term and short-term memorability of the videos. The annotations were gathered as part of an online memory game and measured a participant's ability to recall having seen the video previously when shown a collection of videos. The recognition tasks were performed on videos seen within the previous few minutes for short-term memorability and within the previous 24 to 72 hours for long-term memorability. Data includes the reaction times for each recognition of each video. Associated with each video are text descriptions (captions) as well as a collection of image-level features applied to 3 frames extracted from each video (start, middle and end). Video-level features are also provided. The dataset was used in the Video Memorability task as part of the MediaEval benchmark in 2020.
△ Less
Submitted 4 December, 2021;
originally announced December 2021.
-
Overview of MediaEval 2020 Predicting Media Memorability Task: What Makes a Video Memorable?
Authors:
Alba García Seco De Herrera,
Rukiye Savran Kiziltepe,
Jon Chamberlain,
Mihai Gabriel Constantin,
Claire-Hélène Demarty,
Faiyaz Doctor,
Bogdan Ionescu,
Alan F. Smeaton
Abstract:
This paper describes the MediaEval 2020 \textit{Predicting Media Memorability} task. After first being proposed at MediaEval 2018, the Predicting Media Memorability task is in its 3rd edition this year, as the prediction of short-term and long-term video memorability (VM) remains a challenging task. In 2020, the format remained the same as in previous editions. This year the videos are a subset of…
▽ More
This paper describes the MediaEval 2020 \textit{Predicting Media Memorability} task. After first being proposed at MediaEval 2018, the Predicting Media Memorability task is in its 3rd edition this year, as the prediction of short-term and long-term video memorability (VM) remains a challenging task. In 2020, the format remained the same as in previous editions. This year the videos are a subset of the TRECVid 2019 Video-to-Text dataset, containing more action rich video content as compared with the 2019 task. In this paper a description of some aspects of this task is provided, including its main characteristics, a description of the collection, the ground truth dataset, evaluation metrics and the requirements for participants' run submissions.
△ Less
Submitted 31 December, 2020;
originally announced December 2020.
-
VideoMem: Constructing, Analyzing, Predicting Short-term and Long-term Video Memorability
Authors:
Romain Cohendet,
Claire-Hélène Demarty,
Ngoc Q. K. Duong,
Martin Engilberge
Abstract:
Humans share a strong tendency to memorize/forget some of the visual information they encounter. This paper focuses on providing computational models for the prediction of the intrinsic memorability of visual content. To address this new challenge, we introduce a large scale dataset (VideoMem) composed of 10,000 videos annotated with memorability scores. In contrast to previous work on image memor…
▽ More
Humans share a strong tendency to memorize/forget some of the visual information they encounter. This paper focuses on providing computational models for the prediction of the intrinsic memorability of visual content. To address this new challenge, we introduce a large scale dataset (VideoMem) composed of 10,000 videos annotated with memorability scores. In contrast to previous work on image memorability -- where memorability was measured a few minutes after memorization -- memory performance is measured twice: a few minutes after memorization and again 24-72 hours later. Hence, the dataset comes with short-term and long-term memorability annotations. After an in-depth analysis of the dataset, we investigate several deep neural network based models for the prediction of video memorability. Our best model using a ranking loss achieves a Spearman's rank correlation of 0.494 for short-term memorability prediction, while our proposed model with attention mechanism provides insights of what makes a content memorable. The VideoMem dataset with pre-extracted features is publicly available.
△ Less
Submitted 5 December, 2018;
originally announced December 2018.
-
MediaEval 2018: Predicting Media Memorability Task
Authors:
Romain Cohendet,
Claire-Hélène Demarty,
Ngoc Duong,
Mats Sjöberg,
Bogdan Ionescu,
Thanh-Toan Do,
France Rennes
Abstract:
In this paper, we present the Predicting Media Memorability task, which is proposed as part of the MediaEval 2018 Benchmarking Initiative for Multimedia Evaluation. Participants are expected to design systems that automatically predict memorability scores for videos, which reflect the probability of a video being remembered. In contrast to previous work in image memorability prediction, where memo…
▽ More
In this paper, we present the Predicting Media Memorability task, which is proposed as part of the MediaEval 2018 Benchmarking Initiative for Multimedia Evaluation. Participants are expected to design systems that automatically predict memorability scores for videos, which reflect the probability of a video being remembered. In contrast to previous work in image memorability prediction, where memorability was measured a few minutes after memorization, the proposed dataset comes with short-term and long-term memorability annotations. All task characteristics are described, namely: the task's challenges and breakthrough, the released data set and ground truth, the required participant runs and the evaluation metrics.
△ Less
Submitted 3 July, 2018;
originally announced July 2018.
