-
Smoothness properties related to several commutators of fractional operators for values of $\boldsymbol{p}$ beyond the extreme in the multilinear setting
Authors:
Fabio Berra,
Wilfredo Ramos
Abstract:
We prove continuity properties of higher order commutators of fractional operators on the multilinear setting, between a product of weighted Lebesgue spaces into certain weighted Lipschitz spaces. The considered operators include the multilinear fractional integral function and the main results extend previous estimates known for the unweighted case, as well as those established on the linear case…
▽ More
We prove continuity properties of higher order commutators of fractional operators on the multilinear setting, between a product of weighted Lebesgue spaces into certain weighted Lipschitz spaces. The considered operators include the multilinear fractional integral function and the main results extend previous estimates known for the unweighted case, as well as those established on the linear case.
We give a complete study showing the main properties of the related multilinear weights and the optimal region described by the parameters where we can find nontrivial examples, including the restricted case of the one-weight theory. We also exhibit examples of such weights on the whole region.
△ Less
Submitted 8 September, 2023;
originally announced September 2023.
-
Semantic Segmentation under Adverse Conditions: A Weather and Nighttime-aware Synthetic Data-based Approach
Authors:
Abdulrahman Kerim,
Felipe Chamone,
Washington Ramos,
Leandro Soriano Marcolino,
Erickson R. Nascimento,
Richard Jiang
Abstract:
Recent semantic segmentation models perform well under standard weather conditions and sufficient illumination but struggle with adverse weather conditions and nighttime. Collecting and annotating training data under these conditions is expensive, time-consuming, error-prone, and not always practical. Usually, synthetic data is used as a feasible data source to increase the amount of training data…
▽ More
Recent semantic segmentation models perform well under standard weather conditions and sufficient illumination but struggle with adverse weather conditions and nighttime. Collecting and annotating training data under these conditions is expensive, time-consuming, error-prone, and not always practical. Usually, synthetic data is used as a feasible data source to increase the amount of training data. However, just directly using synthetic data may actually harm the model's performance under normal weather conditions while getting only small gains in adverse situations. Therefore, we present a novel architecture specifically designed for using synthetic training data for domain adaptation. We propose a simple yet powerful addition to DeepLabV3+ by using weather and time-of-the-day supervisors trained with multi-task learning, making it both weather and nighttime aware, which improves its mIoU accuracy by $14$ percentage points on the ACDC dataset while maintaining a score of $75\%$ mIoU on the Cityscapes dataset. Our code is available at https://github.com/lsmcolab/Semantic-Segmentation-under-Adverse-Conditions.
△ Less
Submitted 11 October, 2022;
originally announced October 2022.
-
Leveraging Synthetic Data to Learn Video Stabilization Under Adverse Conditions
Authors:
Abdulrahman Kerim,
Washington L. S. Ramos,
Leandro Soriano Marcolino,
Erickson R. Nascimento,
Richard Jiang
Abstract:
Video stabilization plays a central role to improve videos quality. However, despite the substantial progress made by these methods, they were, mainly, tested under standard weather and lighting conditions, and may perform poorly under adverse conditions. In this paper, we propose a synthetic-aware adverse weather robust algorithm for video stabilization that does not require real data and can be…
▽ More
Video stabilization plays a central role to improve videos quality. However, despite the substantial progress made by these methods, they were, mainly, tested under standard weather and lighting conditions, and may perform poorly under adverse conditions. In this paper, we propose a synthetic-aware adverse weather robust algorithm for video stabilization that does not require real data and can be trained only on synthetic data. We also present Silver, a novel rendering engine to generate the required training data with an automatic ground-truth extraction procedure. Our approach uses our specially generated synthetic data for training an affine transformation matrix estimator avoiding the feature extraction issues faced by current methods. Additionally, since no video stabilization datasets under adverse conditions are available, we propose the novel VSAC105Real dataset for evaluation. We compare our method to five state-of-the-art video stabilization algorithms using two benchmarks. Our results show that current approaches perform poorly in at least one weather condition, and that, even training in a small dataset with synthetic data only, we achieve the best performance in terms of stability average score, distortion score, success rate, and average crop** ratio when considering all weather conditions. Hence, our video stabilization model generalizes well on real-world videos and does not require large-scale synthetic training data to converge.
△ Less
Submitted 26 August, 2022;
originally announced August 2022.
-
Two-weighted estimates of the multilinear fractional integral operator between weighted Lebesgue and Lipschitz spaces with optimal parameters
Authors:
Fabio Berra,
Gladis Pradolini,
Wilfredo Ramos
Abstract:
Given an $m$-tuple of weights $\vec{v}=(v_1,\dots,v_m)$, we characterize the classes of pairs $(w,\vec{v})$ involved with the boundedness properties of the multilinear fractional integral operator from $\prod_{i=1}^mL^{p_i}\left(v_i^{p_i}\right)$ into suitable Lipschitz spaces associated to a parameter $δ$, $\mathcal{L}_w(δ)$. Our results generalize some previous estimates not only for the linear…
▽ More
Given an $m$-tuple of weights $\vec{v}=(v_1,\dots,v_m)$, we characterize the classes of pairs $(w,\vec{v})$ involved with the boundedness properties of the multilinear fractional integral operator from $\prod_{i=1}^mL^{p_i}\left(v_i^{p_i}\right)$ into suitable Lipschitz spaces associated to a parameter $δ$, $\mathcal{L}_w(δ)$. Our results generalize some previous estimates not only for the linear case but also for the unweighted problem in the multilinear context. We emphasize the study related to the range of the parameters involved with the problem described above, which is optimal in the sense that they become trivial outside of the region obtained.
We also exhibit nontrivial examples of pairs of weights in this region.
△ Less
Submitted 23 May, 2022;
originally announced May 2022.
-
Text-Driven Video Acceleration: A Weakly-Supervised Reinforcement Learning Method
Authors:
Washington Ramos,
Michel Silva,
Edson Araujo,
Victor Moura,
Keller Oliveira,
Leandro Soriano Marcolino,
Erickson R. Nascimento
Abstract:
The growth of videos in our digital age and the users' limited time raise the demand for processing untrimmed videos to produce shorter versions conveying the same information. Despite the remarkable progress that summarization methods have made, most of them can only select a few frames or skims, creating visual gaps and breaking the video context. This paper presents a novel weakly-supervised me…
▽ More
The growth of videos in our digital age and the users' limited time raise the demand for processing untrimmed videos to produce shorter versions conveying the same information. Despite the remarkable progress that summarization methods have made, most of them can only select a few frames or skims, creating visual gaps and breaking the video context. This paper presents a novel weakly-supervised methodology based on a reinforcement learning formulation to accelerate instructional videos using text. A novel joint reward function guides our agent to select which frames to remove and reduce the input video to a target length without creating gaps in the final video. We also propose the Extended Visually-guided Document Attention Network (VDAN+), which can generate a highly discriminative embedding space to represent both textual and visual data. Our experiments show that our method achieves the best performance in Precision, Recall, and F1 Score against the baselines while effectively controlling the video's output length. Visit https://www.verlab.dcc.ufmg.br/semantic-hyperlapse/tpami2022/ for code and extra results.
△ Less
Submitted 29 March, 2022;
originally announced March 2022.
-
Optimal parameters related with continuity properties of the multilinear fractional integral operator between Lebesgue and Lipschitz spaces
Authors:
Fabio Berra,
Gladis Pradolini,
Wilfredo Ramos
Abstract:
We deal with the boundedness of the multilinear fractional integral operator $I_{γ,m}$ from a product of weighted Lebesgue spaces into adequate weighted Lipschitz spaces. Our results generalize some previous estimates not only for the linear case but also for the unweighted problem in the multilinear context. We characterize the classes of weights for which the problem described above holds and sh…
▽ More
We deal with the boundedness of the multilinear fractional integral operator $I_{γ,m}$ from a product of weighted Lebesgue spaces into adequate weighted Lipschitz spaces. Our results generalize some previous estimates not only for the linear case but also for the unweighted problem in the multilinear context. We characterize the classes of weights for which the problem described above holds and show the optimal range of the parameters involved. The optimality is understood in the sense that the parameters defining the corresponding spaces belong to a certain region. We further exhibit examples of weights for the class which cover the mentioned area.
△ Less
Submitted 8 March, 2022;
originally announced March 2022.
-
A Sparse Sampling-based framework for Semantic Fast-Forward of First-Person Videos
Authors:
Michel Melo Silva,
Washington Luis Souza Ramos,
Mario Fernando Montenegro Campos,
Erickson Rangel Nascimento
Abstract:
Technological advances in sensors have paved the way for digital cameras to become increasingly ubiquitous, which, in turn, led to the popularity of the self-recording culture. As a result, the amount of visual data on the Internet is moving in the opposite direction of the available time and patience of the users. Thus, most of the uploaded videos are doomed to be forgotten and unwatched stashed…
▽ More
Technological advances in sensors have paved the way for digital cameras to become increasingly ubiquitous, which, in turn, led to the popularity of the self-recording culture. As a result, the amount of visual data on the Internet is moving in the opposite direction of the available time and patience of the users. Thus, most of the uploaded videos are doomed to be forgotten and unwatched stashed away in some computer folder or website. In this paper, we address the problem of creating smooth fast-forward videos without losing the relevant content. We present a new adaptive frame selection formulated as a weighted minimum reconstruction problem. Using a smoothing frame transition and filling visual gaps between segments, our approach accelerates first-person videos emphasizing the relevant segments and avoids visual discontinuities. Experiments conducted on controlled videos and also on an unconstrained dataset of First-Person Videos (FPVs) show that, when creating fast-forward videos, our method is able to retain as much relevant information and smoothness as the state-of-the-art techniques, but in less processing time.
△ Less
Submitted 21 September, 2020;
originally announced September 2020.
-
Straight to the Point: Fast-forwarding Videos via Reinforcement Learning Using Textual Data
Authors:
Washington Ramos,
Michel Silva,
Edson Araujo,
Leandro Soriano Marcolino,
Erickson Nascimento
Abstract:
The rapid increase in the amount of published visual data and the limited time of users bring the demand for processing untrimmed videos to produce shorter versions that convey the same information. Despite the remarkable progress that has been made by summarization methods, most of them can only select a few frames or skims, which creates visual gaps and breaks the video context. In this paper, w…
▽ More
The rapid increase in the amount of published visual data and the limited time of users bring the demand for processing untrimmed videos to produce shorter versions that convey the same information. Despite the remarkable progress that has been made by summarization methods, most of them can only select a few frames or skims, which creates visual gaps and breaks the video context. In this paper, we present a novel methodology based on a reinforcement learning formulation to accelerate instructional videos. Our approach can adaptively select frames that are not relevant to convey the information without creating gaps in the final video. Our agent is textually and visually oriented to select which frames to remove to shrink the input video. Additionally, we propose a novel network, called Visually-guided Document Attention Network (VDAN), able to generate a highly discriminative embedding space to represent both textual and visual data. Our experiments show that our method achieves the best performance in terms of F1 Score and coverage at the video segment level.
△ Less
Submitted 31 March, 2020;
originally announced March 2020.
-
Personalizing Fast-Forward Videos Based on Visual and Textual Features from Social Network
Authors:
Washington L. S. Ramos,
Michel M. Silva,
Edson R. Araujo,
Alan C. Neves,
Erickson R. Nascimento
Abstract:
The growth of Social Networks has fueled the habit of people logging their day-to-day activities, and long First-Person Videos (FPVs) are one of the main tools in this new habit. Semantic-aware fast-forward methods are able to decrease the watch time and select meaningful moments, which is key to increase the chances of these videos being watched. However, these methods can not handle semantics in…
▽ More
The growth of Social Networks has fueled the habit of people logging their day-to-day activities, and long First-Person Videos (FPVs) are one of the main tools in this new habit. Semantic-aware fast-forward methods are able to decrease the watch time and select meaningful moments, which is key to increase the chances of these videos being watched. However, these methods can not handle semantics in terms of personalization. In this work, we present a new approach to automatically creating personalized fast-forward videos for FPVs. Our approach explores the availability of text-centric data from the user's social networks such as status updates to infer her/his topics of interest and assigns scores to the input frames according to her/his preferences. Extensive experiments are conducted on three different datasets with simulated and real-world users as input, achieving an average F1 score of up to 12.8 percentage points higher than the best competitors. We also present a user study to demonstrate the effectiveness of our method.
△ Less
Submitted 29 December, 2019;
originally announced December 2019.
-
Musielak Orlicz bumps and Bloom type estimates for commutators of Calderón Zygmund and fractional integral operators on variable Lebesgue spaces via sparse operators
Authors:
Luciana Melchiori,
Gladis Pradolini,
Wilfredo Ramos
Abstract:
We obtain Musielak Orlicz bumps conditions on a pair of weights for the boundedness of Calderón Zygmund operators and their commutators between variable Lebesgue spaces with different weights. The symbols of the commutators belong to a wider class of functions. We also give Bloom type estimates for commutators of Calderón Zygmund and fractional integral operators in the variable Lebesgue context.…
▽ More
We obtain Musielak Orlicz bumps conditions on a pair of weights for the boundedness of Calderón Zygmund operators and their commutators between variable Lebesgue spaces with different weights. The symbols of the commutators belong to a wider class of functions. We also give Bloom type estimates for commutators of Calderón Zygmund and fractional integral operators in the variable Lebesgue context. The techniques involved in both type of results are related with the theory of sparse domination.
△ Less
Submitted 23 October, 2019;
originally announced October 2019.
-
On two weighted problems for commutators of classical operators with optimal behaviour on the parameters involved and extrapolation results
Authors:
Gladis Pradolini,
Wilfredo Ramos,
Jorgelina Recchi
Abstract:
We give two weighted norm estimates for higher order commutator of classical operators such as singular integral and fractional type operators, between weighted $L^p$ and certain spaces that include Lipschitz, BMO and Morrey spaces. We also give the optimal parameters involved with these results, where the optimality is understood in the sense that the parameters defining the corresponding spaces…
▽ More
We give two weighted norm estimates for higher order commutator of classical operators such as singular integral and fractional type operators, between weighted $L^p$ and certain spaces that include Lipschitz, BMO and Morrey spaces. We also give the optimal parameters involved with these results, where the optimality is understood in the sense that the parameters defining the corresponding spaces belong to certain region out of which the classes of weights are satisfied by trivial weights. We also exhibit pairs of non-trivial weights in the optimal region satisfying the conditions required. Finally, we exhibit an extrapolation result that allows us to obtain boundedness results of the type described above in the variable setting and for a great variety of operators, by starting from analogous inequalities in the classical context. In order to get this result we prove a Calderón-Scott type inequality with weights that connects adequately the spaces involved.
△ Less
Submitted 27 September, 2020; v1 submitted 22 October, 2019;
originally announced October 2019.
-
Commutators of potential type operators with Lipschitz symbols on variable Lebesgue spaces with different weights
Authors:
Luciana Melchiori,
Gladis Pradolini,
Wilfredo Ramos
Abstract:
We prove that a generalized Fefferman-Phong type condition on a pair of weights $u$ and $v$ is sufficient for the boundedness of the commutators of potential type operators from $L^{p(\cdot)}_v$ into $L^{q(\cdot)}_u$. We also give an improvement of this result in the sense that we not only consider a variable version of power bump conditions, but also weaker norms related to Musielak-Orlicz functi…
▽ More
We prove that a generalized Fefferman-Phong type condition on a pair of weights $u$ and $v$ is sufficient for the boundedness of the commutators of potential type operators from $L^{p(\cdot)}_v$ into $L^{q(\cdot)}_u$. We also give an improvement of this result in the sense that we not only consider a variable version of power bump conditions, but also weaker norms related to Musielak-Orlicz functions. We consider a wider class of symbols including Lipschitz symbols and some generalizations.
△ Less
Submitted 12 July, 2019;
originally announced July 2019.
-
A Weighted Sparse Sampling and Smoothing Frame Transition Approach for Semantic Fast-Forward First-Person Videos
Authors:
Michel Melo Silva,
Washington Luis Souza Ramos,
Joao Klock Ferreira,
Felipe Cadar Chamone,
Mario Fernando Montenegro Campos,
Erickson Rangel Nascimento
Abstract:
Thanks to the advances in the technology of low-cost digital cameras and the popularity of the self-recording culture, the amount of visual data on the Internet is going to the opposite side of the available time and patience of the users. Thus, most of the uploaded videos are doomed to be forgotten and unwatched in a computer folder or website. In this work, we address the problem of creating smo…
▽ More
Thanks to the advances in the technology of low-cost digital cameras and the popularity of the self-recording culture, the amount of visual data on the Internet is going to the opposite side of the available time and patience of the users. Thus, most of the uploaded videos are doomed to be forgotten and unwatched in a computer folder or website. In this work, we address the problem of creating smooth fast-forward videos without losing the relevant content. We present a new adaptive frame selection formulated as a weighted minimum reconstruction problem, which combined with a smoothing frame transition method accelerates first-person videos emphasizing the relevant segments and avoids visual discontinuities. The experiments show that our method is able to fast-forward videos to retain as much relevant information and smoothness as the state-of-the-art techniques in less time. We also present a new 80-hour multimodal (RGB-D, IMU, and GPS) dataset of first-person videos with annotations for recorder profile, frame scene, activities, interaction, and attention.
△ Less
Submitted 4 April, 2019; v1 submitted 23 February, 2018;
originally announced February 2018.
-
Making a long story short: A Multi-Importance fast-forwarding egocentric videos with the emphasis on relevant objects
Authors:
Michel Melo Silva,
Washington Luis Souza Ramos,
Felipe Cadar Chamone,
João Pedro Klock Ferreira,
Mario Fernando Montenegro Campos,
Erickson Rangel Nascimento
Abstract:
The emergence of low-cost high-quality personal wearable cameras combined with the increasing storage capacity of video-sharing websites have evoked a growing interest in first-person videos, since most videos are composed of long-running unedited streams which are usually tedious and unpleasant to watch. State-of-the-art semantic fast-forward methods currently face the challenge of providing an a…
▽ More
The emergence of low-cost high-quality personal wearable cameras combined with the increasing storage capacity of video-sharing websites have evoked a growing interest in first-person videos, since most videos are composed of long-running unedited streams which are usually tedious and unpleasant to watch. State-of-the-art semantic fast-forward methods currently face the challenge of providing an adequate balance between smoothness in visual flow and the emphasis on the relevant parts. In this work, we present the Multi-Importance Fast-Forward (MIFF), a fully automatic methodology to fast-forward egocentric videos facing these challenges. The dilemma of defining what is the semantic information of a video is addressed by a learning process based on the preferences of the user. Results show that the proposed method keeps over $3$ times more semantic content than the state-of-the-art fast-forward. Finally, we discuss the need of a particular video stabilization technique for fast-forward egocentric videos.
△ Less
Submitted 7 March, 2018; v1 submitted 9 November, 2017;
originally announced November 2017.
-
Fast-Forward Video Based on Semantic Extraction
Authors:
Washington Luis Souza Ramos,
Michel Melo Silva,
Mario Fernando Montenegro Campos,
Erickson Rangel Nascimento
Abstract:
Thanks to the low operational cost and large storage capacity of smartphones and wearable devices, people are recording many hours of daily activities, sport actions and home videos. These videos, also known as egocentric videos, are generally long-running streams with unedited content, which make them boring and visually unpalatable, bringing up the challenge to make egocentric videos more appeal…
▽ More
Thanks to the low operational cost and large storage capacity of smartphones and wearable devices, people are recording many hours of daily activities, sport actions and home videos. These videos, also known as egocentric videos, are generally long-running streams with unedited content, which make them boring and visually unpalatable, bringing up the challenge to make egocentric videos more appealing. In this work we propose a novel methodology to compose the new fast-forward video by selecting frames based on semantic information extracted from images. The experiments show that our approach outperforms the state-of-the-art as far as semantic information is concerned and that it is also able to produce videos that are more pleasant to be watched.
△ Less
Submitted 16 August, 2017; v1 submitted 14 August, 2017;
originally announced August 2017.
-
Towards Semantic Fast-Forward and Stabilized Egocentric Videos
Authors:
Michel Melo Silva,
Washington Luis Souza Ramos,
Joao Pedro Klock Ferreira,
Mario Fernando Montenegro Campos,
Erickson Rangel Nascimento
Abstract:
The emergence of low-cost personal mobiles devices and wearable cameras and the increasing storage capacity of video-sharing websites have pushed forward a growing interest towards first-person videos. Since most of the recorded videos compose long-running streams with unedited content, they are tedious and unpleasant to watch. The fast-forward state-of-the-art methods are facing challenges of bal…
▽ More
The emergence of low-cost personal mobiles devices and wearable cameras and the increasing storage capacity of video-sharing websites have pushed forward a growing interest towards first-person videos. Since most of the recorded videos compose long-running streams with unedited content, they are tedious and unpleasant to watch. The fast-forward state-of-the-art methods are facing challenges of balancing the smoothness of the video and the emphasis in the relevant frames given a speed-up rate. In this work, we present a methodology capable of summarizing and stabilizing egocentric videos by extracting the semantic information from the frames. This paper also describes a dataset collection with several semantically labeled videos and introduces a new smoothness evaluation metric for egocentric videos that is used to test our method.
△ Less
Submitted 16 August, 2017; v1 submitted 14 August, 2017;
originally announced August 2017.
-
The effect of the smoothness of fractional type operators over their commutators with Lipschitz symbols on weighted spaces
Authors:
Estefanía Dalmasso,
Gladis Pradolini,
Wilfredo Ramos
Abstract:
We prove boundedness results for integral operators of fractional type and their higher order commutators between weighted spaces, including $L^p$-$L^q$, $L^p$-$BMO$ and $L^p$-Lipschitz estimates. The kernels of such operators satisfy certain size condition and a Lipschitz type regularity, and the symbol of the commutator belongs to a Lipschitz class. We also deal with commutators of fractional ty…
▽ More
We prove boundedness results for integral operators of fractional type and their higher order commutators between weighted spaces, including $L^p$-$L^q$, $L^p$-$BMO$ and $L^p$-Lipschitz estimates. The kernels of such operators satisfy certain size condition and a Lipschitz type regularity, and the symbol of the commutator belongs to a Lipschitz class. We also deal with commutators of fractional type operators with less regular kernels satisfying a Hörmander's type inequality. As far as we know, these last results are new even in the unweighted case. Moreover, we give a characterization result involving symbols of the commutators and continuity results for extreme values of $p$.
△ Less
Submitted 17 March, 2017;
originally announced March 2017.
