-
Live Video Captioning
Authors:
Eduardo Blanco-Fernández,
Carlos Gutiérrez-Álvarez,
Nadia Nasri,
Saturnino Maldonado-Bascón,
Roberto J. López-Sastre
Abstract:
Dense video captioning is the task that involves the detection and description of events within video sequences. While traditional approaches focus on offline solutions where the entire video of analysis is available for the captioning model, in this work we introduce a paradigm shift towards Live Video Captioning (LVC). In LVC, dense video captioning models must generate captions for video stream…
▽ More
Dense video captioning is the task that involves the detection and description of events within video sequences. While traditional approaches focus on offline solutions where the entire video of analysis is available for the captioning model, in this work we introduce a paradigm shift towards Live Video Captioning (LVC). In LVC, dense video captioning models must generate captions for video streams in an online manner, facing important constraints such as having to work with partial observations of the video, the need for temporal anticipation and, of course, ensuring ideally a real-time response. In this work we formally introduce the novel problem of LVC and propose new evaluation metrics tailored for the online scenario, demonstrating their superiority over traditional metrics. We also propose an LVC model integrating deformable transformers and temporal filtering to address the LVC new challenges. Experimental evaluations on the ActivityNet Captions dataset validate the effectiveness of our approach, highlighting its performance in LVC compared to state-of-the-art offline methods. Results of our model as well as an evaluation kit with the novel metrics integrated are made publicly available to encourage further research on LVC.
△ Less
Submitted 20 June, 2024;
originally announced June 2024.
-
Realistic Continual Learning Approach using Pre-trained Models
Authors:
Nadia Nasri,
Carlos Gutiérrez-Álvarez,
Sergio Lafuente-Arroyo,
Saturnino Maldonado-Bascón,
Roberto J. López-Sastre
Abstract:
Continual learning (CL) is crucial for evaluating adaptability in learning solutions to retain knowledge. Our research addresses the challenge of catastrophic forgetting, where models lose proficiency in previously learned tasks as they acquire new ones. While numerous solutions have been proposed, existing experimental setups often rely on idealized class-incremental learning scenarios. We introd…
▽ More
Continual learning (CL) is crucial for evaluating adaptability in learning solutions to retain knowledge. Our research addresses the challenge of catastrophic forgetting, where models lose proficiency in previously learned tasks as they acquire new ones. While numerous solutions have been proposed, existing experimental setups often rely on idealized class-incremental learning scenarios. We introduce Realistic Continual Learning (RealCL), a novel CL paradigm where class distributions across tasks are random, departing from structured setups.
We also present CLARE (Continual Learning Approach with pRE-trained models for RealCL scenarios), a pre-trained model-based solution designed to integrate new knowledge while preserving past learning. Our contributions include pioneering RealCL as a generalization of traditional CL setups, proposing CLARE as an adaptable approach for RealCL tasks, and conducting extensive experiments demonstrating its effectiveness across various RealCL scenarios. Notably, CLARE outperforms existing models on RealCL benchmarks, highlighting its versatility and robustness in unpredictable learning environments.
△ Less
Submitted 11 April, 2024;
originally announced April 2024.
-
Analysis study of time synchronization protocols in wireless sensor networks
Authors:
Salim el Khediri,
Nejah Nasri,
Mounir Samet,
Anne Wei,
Abdennaceur Kachouri
Abstract:
One of the main pervasive problems Wireless Sensor Networks (WSN) encounter is to maintain flawless communication sharing and cooperative processing between sensors via radio links to ensure a reliable treatment of information. Many applications based on these WSNs consider local clocks at each sensor node that need to be synchronized to a common notion of time. In this context, the majority of pr…
▽ More
One of the main pervasive problems Wireless Sensor Networks (WSN) encounter is to maintain flawless communication sharing and cooperative processing between sensors via radio links to ensure a reliable treatment of information. Many applications based on these WSNs consider local clocks at each sensor node that need to be synchronized to a common notion of time. In this context, the majority of previous researches were focused on the study of protocols, and algorithms that address these issues in order to resolve synchronization problems. Previous fforts and empirical studies in wireless sensor network (WSN) proposed several solutions (algorithms). The focus of this this paper is to examine and evaluate the most important synchronization algorithms based on the positions of various quantitative and qualitative synchronization protocols for energy-efficient information processing and routing in WSNs.
△ Less
Submitted 7 June, 2012;
originally announced June 2012.
