Showing 1–37 of 37 results for author: Moreira, D

Impact of Network Deployment on the Performance of NCR-assisted Networks

Authors: Gabriel C. M. da Silva, Diego A. Sousa, Victor F. Monteiro, Darlan C. Moreira, Tarcisio F. Maciel, Fco. Rafael M. Lima, Behrooz Makki

Abstract: To address the need of coverage enhancement in the fifth generation (5G) of wireless cellular telecommunications, while taking into account possible bottlenecks related to deploying fiber based backhaul (e.g., required cost and time), the 3rd generation partnership project (3GPP) proposed in Release 18 the concept of network-controlled repeaters (NCRs). NCRs enhance previous radio frequency (RF) r… ▽ More To address the need of coverage enhancement in the fifth generation (5G) of wireless cellular telecommunications, while taking into account possible bottlenecks related to deploying fiber based backhaul (e.g., required cost and time), the 3rd generation partnership project (3GPP) proposed in Release 18 the concept of network-controlled repeaters (NCRs). NCRs enhance previous radio frequency (RF) repeaters by exploring beamforming transmissions controlled by the network through side control information. In this context, this paper introduces the concept of NCR. Furthermore, we present a system level model that allows the performance evaluation of an NCR-assisted network. Finally, we evaluate the network deployment impact on the performance of NCR-assisted networks. As we show, with proper network planning, NCRs can boost the signal to interference-plus-noise ratio (SINR) of the user equipments (UEs) in a poor coverage of a macro base station. Furthermore, celledge UEs and uplink (UL) communications are the ones that benefit the most from the presence of NCRs. △ Less

Submitted 2 July, 2024; originally announced July 2024.

Comments: Paper accepted for publication in the conference proceedings of "19th International Symposium on Wireless Communication Systems" (ISWCS)

Network-Controlled Repeater -- An Introduction

Authors: Fco. Italo G. Carvalho, Raul Victor de O. Paiva, Tarcisio F. Maciel, Victor F. Monteiro, Fco. Rafael M. Lima, Darlan C. Moreira, Diego A. Sousa, Behrooz Makki, Magnus Astrom, Lei Bao

Abstract: In fifth generation (5G) wireless cellular networks, millimeter wave spectrum opens room for several potential improvements in throughput, reliability, latency, among other aspects. However, it also brings challenges, such as a higher influence of blockage which may significantly limit the coverage. In this context, network-controlled repeaters (NCRs) are network nodes with low complexity that rep… ▽ More In fifth generation (5G) wireless cellular networks, millimeter wave spectrum opens room for several potential improvements in throughput, reliability, latency, among other aspects. However, it also brings challenges, such as a higher influence of blockage which may significantly limit the coverage. In this context, network-controlled repeaters (NCRs) are network nodes with low complexity that represent a technique to overcome coverage problems. In this paper, we introduce the NCR concept and study its performance gains and deployment options. Particularly, presenting the main specifications of NCR as agreed in 3rd generation partnership project (3GPP) Rel-18, we analyze different NCR deployments in an urban scenario and compare its performance with alternative deployments. As demonstrated, with a proper network planning and beamforming design, NCR is an attractive solution to cover blind spots the base stations (BSs) may have. △ Less

Submitted 14 March, 2024; originally announced March 2024.

Comments: Submmited to IEEE Communications Standards Magazine

Exploring Saliency Bias in Manipulation Detection

Authors: Joshua Krinsky, Alan Bettis, Qiuyu Tang, Daniel Moreira, Aparna Bharati

Abstract: The social media-fuelled explosion of fake news and misinformation supported by tampered images has led to growth in the development of models and datasets for image manipulation detection. However, existing detection methods mostly treat media objects in isolation, without considering the impact of specific manipulations on viewer perception. Forensic datasets are usually analyzed based on the ma… ▽ More The social media-fuelled explosion of fake news and misinformation supported by tampered images has led to growth in the development of models and datasets for image manipulation detection. However, existing detection methods mostly treat media objects in isolation, without considering the impact of specific manipulations on viewer perception. Forensic datasets are usually analyzed based on the manipulation operations and corresponding pixel-based masks, but not on the semantics of the manipulation, i.e., type of scene, objects, and viewers' attention to scene content. The semantics of the manipulation play an important role in spreading misinformation through manipulated images. In an attempt to encourage further development of semantic-aware forensic approaches to understand visual misinformation, we propose a framework to analyze the trends of visual and semantic saliency in popular image manipulation datasets and their impact on detection. △ Less

Submitted 15 February, 2024; v1 submitted 11 February, 2024; originally announced February 2024.

LlamaRec: Two-Stage Recommendation using Large Language Models for Ranking

Authors: Zhenrui Yue, Sara Rabhi, Gabriel de Souza Pereira Moreira, Dong Wang, Even Oldridge

Abstract: Recently, large language models (LLMs) have exhibited significant progress in language understanding and generation. By leveraging textual features, customized LLMs are also applied for recommendation and demonstrate improvements across diverse recommendation scenarios. Yet the majority of existing methods perform training-free recommendation that heavily relies on pretrained knowledge (e.g., movi… ▽ More Recently, large language models (LLMs) have exhibited significant progress in language understanding and generation. By leveraging textual features, customized LLMs are also applied for recommendation and demonstrate improvements across diverse recommendation scenarios. Yet the majority of existing methods perform training-free recommendation that heavily relies on pretrained knowledge (e.g., movie recommendation). In addition, inference on LLMs is slow due to autoregressive generation, rendering existing methods less effective for real-time recommendation. As such, we propose a two-stage framework using large language models for ranking-based recommendation (LlamaRec). In particular, we use small-scale sequential recommenders to retrieve candidates based on the user interaction history. Then, both history and retrieved items are fed to the LLM in text via a carefully designed prompt template. Instead of generating next-item titles, we adopt a verbalizer-based approach that transforms output logits into probability distributions over the candidate items. Therefore, the proposed LlamaRec can efficiently rank items without generating long text. To validate the effectiveness of the proposed framework, we compare against state-of-the-art baseline methods on benchmark datasets. Our experimental results demonstrate the performance of LlamaRec, which consistently achieves superior performance in both recommendation performance and efficiency. △ Less

Submitted 25 October, 2023; originally announced November 2023.

Comments: Accepted to PGAI@CIKM 2023

CAPIVARA: Cost-Efficient Approach for Improving Multilingual CLIP Performance on Low-Resource Languages

Authors: Gabriel Oliveira dos Santos, Diego A. B. Moreira, Alef Iury Ferreira, Jhessica Silva, Luiz Pereira, Pedro Bueno, Thiago Sousa, Helena Maia, Nádia Da Silva, Esther Colombini, Helio Pedrini, Sandra Avila

Abstract: This work introduces CAPIVARA, a cost-efficient framework designed to enhance the performance of multilingual CLIP models in low-resource languages. While CLIP has excelled in zero-shot vision-language tasks, the resource-intensive nature of model training remains challenging. Many datasets lack linguistic diversity, featuring solely English descriptions for images. CAPIVARA addresses this by augm… ▽ More This work introduces CAPIVARA, a cost-efficient framework designed to enhance the performance of multilingual CLIP models in low-resource languages. While CLIP has excelled in zero-shot vision-language tasks, the resource-intensive nature of model training remains challenging. Many datasets lack linguistic diversity, featuring solely English descriptions for images. CAPIVARA addresses this by augmenting text data using image captioning and machine translation to generate multiple synthetic captions in low-resource languages. We optimize the training pipeline with LiT, LoRA, and gradient checkpointing to alleviate the computational cost. Through extensive experiments, CAPIVARA emerges as state of the art in zero-shot tasks involving images and Portuguese texts. We show the potential for significant improvements in other low-resource languages, achieved by fine-tuning the pre-trained multilingual CLIP using CAPIVARA on a single GPU for 2 hours. Our model and code is available at https://github.com/hiaac-nlp/CAPIVARA. △ Less

Submitted 23 October, 2023; v1 submitted 20 October, 2023; originally announced October 2023.

The Age of Synthetic Realities: Challenges and Opportunities

Authors: João Phillipe Cardenuto, **g Yang, Rafael Padilha, Renjie Wan, Daniel Moreira, Haoliang Li, Shiqi Wang, Fernanda Andaló, Sébastien Marcel, Anderson Rocha

Abstract: Synthetic realities are digital creations or augmentations that are contextually generated through the use of Artificial Intelligence (AI) methods, leveraging extensive amounts of data to construct new narratives or realities, regardless of the intent to deceive. In this paper, we delve into the concept of synthetic realities and their implications for Digital Forensics and society at large within… ▽ More Synthetic realities are digital creations or augmentations that are contextually generated through the use of Artificial Intelligence (AI) methods, leveraging extensive amounts of data to construct new narratives or realities, regardless of the intent to deceive. In this paper, we delve into the concept of synthetic realities and their implications for Digital Forensics and society at large within the rapidly advancing field of AI. We highlight the crucial need for the development of forensic techniques capable of identifying harmful synthetic creations and distinguishing them from reality. This is especially important in scenarios involving the creation and dissemination of fake news, disinformation, and misinformation. Our focus extends to various forms of media, such as images, videos, audio, and text, as we examine how synthetic realities are crafted and explore approaches to detecting these malicious creations. Additionally, we shed light on the key research challenges that lie ahead in this area. This study is of paramount importance due to the rapid progress of AI generative techniques and their impact on the fundamental principles of Forensic Science. △ Less

Submitted 9 June, 2023; originally announced June 2023.

E Pluribus Unum: Guidelines on Multi-Objective Evaluation of Recommender Systems

Authors: Patrick John Chia, Giuseppe Attanasio, Jacopo Tagliabue, Federico Bianchi, Ciro Greco, Gabriel de Souza P. Moreira, Davide Eynard, Fahd Husain

Abstract: Recommender Systems today are still mostly evaluated in terms of accuracy, with other aspects beyond the immediate relevance of recommendations, such as diversity, long-term user retention and fairness, often taking a back seat. Moreover, reconciling multiple performance perspectives is by definition indeterminate, presenting a stumbling block to those in the pursuit of rounded evaluation of Recom… ▽ More Recommender Systems today are still mostly evaluated in terms of accuracy, with other aspects beyond the immediate relevance of recommendations, such as diversity, long-term user retention and fairness, often taking a back seat. Moreover, reconciling multiple performance perspectives is by definition indeterminate, presenting a stumbling block to those in the pursuit of rounded evaluation of Recommender Systems. EvalRS 2022 -- a data challenge designed around Multi-Objective Evaluation -- was a first practical endeavour, providing many insights into the requirements and challenges of balancing multiple objectives in evaluation. In this work, we reflect on EvalRS 2022 and expound upon crucial learnings to formulate a first-principles approach toward Multi-Objective model selection, and outline a set of guidelines for carrying out a Multi-Objective Evaluation challenge, with potential applicability to the problem of rounded evaluation of competing models in real-world deployments. △ Less

Submitted 20 April, 2023; originally announced April 2023.

Comments: 15 pages, under submission

On the Effectiveness of Image Manipulation Detection in the Age of Social Media

Authors: Rosaura G. VidalMata, Priscila Saboia, Daniel Moreira, Grant Jensen, Jason Schlessman, Walter J. Scheirer

Abstract: Image manipulation detection algorithms designed to identify local anomalies often rely on the manipulated regions being ``sufficiently'' different from the rest of the non-tampered regions in the image. However, such anomalies might not be easily identifiable in high-quality manipulations, and their use is often based on the assumption that certain image phenomena are associated with the use of s… ▽ More Image manipulation detection algorithms designed to identify local anomalies often rely on the manipulated regions being ``sufficiently'' different from the rest of the non-tampered regions in the image. However, such anomalies might not be easily identifiable in high-quality manipulations, and their use is often based on the assumption that certain image phenomena are associated with the use of specific editing tools. This makes the task of manipulation detection hard in and of itself, with state-of-the-art detectors only being able to detect a limited number of manipulation types. More importantly, in cases where the anomaly assumption does not hold, the detection of false positives in otherwise non-manipulated images becomes a serious problem. To understand the current state of manipulation detection, we present an in-depth analysis of deep learning-based and learning-free methods, assessing their performance on different benchmark datasets containing tampered and non-tampered samples. We provide a comprehensive study of their suitability for detecting different manipulations as well as their robustness when presented with non-tampered data. Furthermore, we propose a novel deep learning-based pre-processing technique that accentuates the anomalies present in manipulated regions to make them more identifiable by a variety of manipulation detection methods. To this end, we introduce an anomaly enhancement loss that, when used with a residual architecture, improves the performance of different detection algorithms with a minimal introduction of false positives on the non-manipulated data. Lastly, we introduce an open-source manipulation detection toolkit comprising a number of standard detection algorithms. △ Less

Submitted 19 April, 2023; originally announced April 2023.

Analysis and Compilation of Normal Map Generation Techniques for Pixel Art

Authors: Rodrigo D. Moreira, Flávio Coutinho, Luiz Chaimowicz

Abstract: Pixel art is a popular artistic style adopted in the gaming industry, and nowadays, it is often accompanied by modern rendering techniques. One example is dynamic lighting for the game sprites, for which normal map** defines how the light interacts with the material represented by each pixel. Although there are different methods to generate normal maps for 3D games, applying them for pixel art m… ▽ More Pixel art is a popular artistic style adopted in the gaming industry, and nowadays, it is often accompanied by modern rendering techniques. One example is dynamic lighting for the game sprites, for which normal map** defines how the light interacts with the material represented by each pixel. Although there are different methods to generate normal maps for 3D games, applying them for pixel art may not yield correct results due to the style specificities. Therefore, this work compiles different normal map generation methods and study their applicability for pixel art, reducing the scarcity of existing material on the techniques and contributing to a qualitative analysis of the behavior of these methods in different case studies. △ Less

Submitted 19 December, 2022; originally announced December 2022.

Comments: Published in SBGames 2022

Human Saliency-Driven Patch-based Matching for Interpretable Post-mortem Iris Recognition

Authors: Aidan Boyd, Daniel Moreira, Andrey Kuehlkamp, Kevin Bowyer, Adam Czajka

Abstract: Forensic iris recognition, as opposed to live iris recognition, is an emerging research area that leverages the discriminative power of iris biometrics to aid human examiners in their efforts to identify deceased persons. As a machine learning-based technique in a predominantly human-controlled task, forensic recognition serves as "back-up" to human expertise in the task of post-mortem identificat… ▽ More Forensic iris recognition, as opposed to live iris recognition, is an emerging research area that leverages the discriminative power of iris biometrics to aid human examiners in their efforts to identify deceased persons. As a machine learning-based technique in a predominantly human-controlled task, forensic recognition serves as "back-up" to human expertise in the task of post-mortem identification. As such, the machine learning model must be (a) interpretable, and (b) post-mortem-specific, to account for changes in decaying eye tissue. In this work, we propose a method that satisfies both requirements, and that approaches the creation of a post-mortem-specific feature extractor in a novel way employing human perception. We first train a deep learning-based feature detector on post-mortem iris images, using annotations of image regions highlighted by humans as salient for their decision making. In effect, the method learns interpretable features directly from humans, rather than purely data-driven features. Second, regional iris codes (again, with human-driven filtering kernels) are used to pair detected iris patches, which are translated into pairwise, patch-based comparison scores. In this way, our method presents human examiners with human-understandable visual cues in order to justify the identification decision and corresponding confidence score. When tested on a dataset of post-mortem iris images collected from 259 deceased subjects, the proposed method places among the three best iris matchers, demonstrating better results than the commercial (non-human-interpretable) VeriEye approach. We propose a unique post-mortem iris recognition method trained with human saliency to give fully-interpretable comparison outcomes for use in the context of forensic examination, achieving state-of-the-art recognition performance. △ Less

Submitted 3 August, 2022; originally announced August 2022.

EvalRS: a Rounded Evaluation of Recommender Systems

Authors: Jacopo Tagliabue, Federico Bianchi, Tobias Schnabel, Giuseppe Attanasio, Ciro Greco, Gabriel de Souza P. Moreira, Patrick John Chia

Abstract: Much of the complexity of Recommender Systems (RSs) comes from the fact that they are used as part of more complex applications and affect user experience through a varied range of user interfaces. However, research focused almost exclusively on the ability of RSs to produce accurate item rankings while giving little attention to the evaluation of RS behavior in real-world scenarios. Such narrow f… ▽ More Much of the complexity of Recommender Systems (RSs) comes from the fact that they are used as part of more complex applications and affect user experience through a varied range of user interfaces. However, research focused almost exclusively on the ability of RSs to produce accurate item rankings while giving little attention to the evaluation of RS behavior in real-world scenarios. Such narrow focus has limited the capacity of RSs to have a lasting impact in the real world and makes them vulnerable to undesired behavior, such as reinforcing data biases. We propose EvalRS as a new type of challenge, in order to foster this discussion among practitioners and build in the open new methodologies for testing RSs "in the wild". △ Less

Submitted 12 August, 2022; v1 submitted 12 July, 2022; originally announced July 2022.

Comments: CIKM 2022 Data Challenge Paper

Motif Mining: Finding and Summarizing Remixed Image Content

Authors: William Theisen, Daniel Gonzalez Cedre, Zachariah Carmichael, Daniel Moreira, Tim Weninger, Walter Scheirer

Abstract: On the internet, images are no longer static; they have become dynamic content. Thanks to the availability of smartphones with cameras and easy-to-use editing software, images can be remixed (i.e., redacted, edited, and recombined with other content) on-the-fly and with a world-wide audience that can repeat the process. From digital art to memes, the evolution of images through time is now an impo… ▽ More On the internet, images are no longer static; they have become dynamic content. Thanks to the availability of smartphones with cameras and easy-to-use editing software, images can be remixed (i.e., redacted, edited, and recombined with other content) on-the-fly and with a world-wide audience that can repeat the process. From digital art to memes, the evolution of images through time is now an important topic of study for digital humanists, social scientists, and media forensics specialists. However, because typical data sets in computer vision are composed of static content, the development of automated algorithms to analyze remixed content has been limited. In this paper, we introduce the idea of Motif Mining - the process of finding and summarizing remixed image content in large collections of unlabeled and unsorted data. In this paper, this idea is formalized and a reference implementation is introduced. Experiments are conducted on three meme-style data sets, including a newly collected set associated with the information war in the Russo-Ukrainian conflict. The proposed motif mining approach is able to identify related remixed content that, when compared to similar approaches, more closely aligns with the preferences and expectations of human observers. △ Less

Submitted 17 March, 2022; v1 submitted 15 March, 2022; originally announced March 2022.

Comments: 41 pages, 21 figures

${\tt simwave}$ -- A Finite Difference Simulator for Acoustic Waves Propagation

Authors: Jaime Freire de Souza, João Baptista Dias Moreira, Keith Jared Roberts, Roussian di Ramos Alves Gaioso, Edson Satoshi Gomi, Emílio Carlos Nelli Silva, Hermes Senger

Abstract: ${\tt simwave}$ is an open-source Python package to perform wave simulations in 2D or 3D domains. It solves the constant and variable density acoustic wave equation with the finite difference method and has support for domain truncation techniques, several boundary conditions, and the modeling of sources and receivers given a user-defined acquisition geometry. The architecture of ${\tt simwave}$ i… ▽ More ${\tt simwave}$ is an open-source Python package to perform wave simulations in 2D or 3D domains. It solves the constant and variable density acoustic wave equation with the finite difference method and has support for domain truncation techniques, several boundary conditions, and the modeling of sources and receivers given a user-defined acquisition geometry. The architecture of ${\tt simwave}$ is designed for applications with geophysical exploration in mind. Its Python front-end enables straightforward integration with many existing Python scientific libraries for the composition of more complex workflows and applications (e.g., migration and inversion problems). The back-end is implemented in C enabling performance portability across a range of computing hardware and compilers including both CPUs and GPUs. △ Less

Submitted 13 January, 2022; originally announced January 2022.

Synthetic Data and Simulators for Recommendation Systems: Current State and Future Directions

Authors: Adam Lesnikowski, Gabriel de Souza Pereira Moreira, Sara Rabhi, Karl Byleen-Higley

Abstract: Synthetic data and simulators have the potential to markedly improve the performance and robustness of recommendation systems. These approaches have already had a beneficial impact in other machine-learning driven fields. We identify and discuss a key trade-off between data fidelity and privacy in the past work on synthetic data and simulators for recommendation systems. For the important use case… ▽ More Synthetic data and simulators have the potential to markedly improve the performance and robustness of recommendation systems. These approaches have already had a beneficial impact in other machine-learning driven fields. We identify and discuss a key trade-off between data fidelity and privacy in the past work on synthetic data and simulators for recommendation systems. For the important use case of predicting algorithm rankings on real data from synthetic data, we provide motivation and current successes versus limitations. Finally we outline a number of exciting future directions for recommendation systems that we believe deserve further attention and work, including mixing real and synthetic data, feedback in dataset generation, robust simulations, and privacy-preserving methods. △ Less

Submitted 21 December, 2021; originally announced December 2021.

Comments: 7 pages, included in SimuRec 2021: Workshop on Simulation Methods for Recommender Systems at ACM RecSys 2021, October 2nd, 2021, Amsterdam, NL and online

Forensic Analysis of Synthetically Generated Western Blot Images

Authors: Sara Mandelli, Davide Cozzolino, Edoardo D. Cannas, Joao P. Cardenuto, Daniel Moreira, Paolo Bestagini, Walter J. Scheirer, Anderson Rocha, Luisa Verdoliva, Stefano Tubaro, Edward J. Delp

Abstract: The widespread diffusion of synthetically generated content is a serious threat that needs urgent countermeasures. As a matter of fact, the generation of synthetic content is not restricted to multimedia data like videos, photographs or audio sequences, but covers a significantly vast area that can include biological images as well, such as western blot and microscopic images. In this paper, we fo… ▽ More The widespread diffusion of synthetically generated content is a serious threat that needs urgent countermeasures. As a matter of fact, the generation of synthetic content is not restricted to multimedia data like videos, photographs or audio sequences, but covers a significantly vast area that can include biological images as well, such as western blot and microscopic images. In this paper, we focus on the detection of synthetically generated western blot images. These images are largely explored in the biomedical literature and it has been already shown they can be easily counterfeited with few hopes to spot manipulations by visual inspection or by using standard forensics detectors. To overcome the absence of publicly available data for this task, we create a new dataset comprising more than 14K original western blot images and 24K synthetic western blot images, generated using four different state-of-the-art generation methods. We investigate different strategies to detect synthetic western blots, exploring binary classification methods as well as one-class detectors. In both scenarios, we never exploit synthetic western blot images at training stage. The achieved results show that synthetically generated western blot images can be spot with good accuracy, even though the exploited detectors are not optimized over synthetic versions of these scientific images. We also test the robustness of the developed detectors against post-processing operations commonly performed on scientific images, showing that we can be robust to JPEG compression and that some generative models are easily recognizable, despite the application of editing might alter the artifacts they leave. △ Less

Submitted 1 June, 2022; v1 submitted 16 December, 2021; originally announced December 2021.

Transformers with multi-modal features and post-fusion context for e-commerce session-based recommendation

Authors: Gabriel de Souza P. Moreira, Sara Rabhi, Ronay Ak, Md Yasin Kabir, Even Oldridge

Abstract: Session-based recommendation is an important task for e-commerce services, where a large number of users browse anonymously or may have very distinct interests for different sessions. In this paper we present one of the winning solutions for the Recommendation task of the SIGIR 2021 Workshop on E-commerce Data Challenge. Our solution was inspired by NLP techniques and consists of an ensemble of tw… ▽ More Session-based recommendation is an important task for e-commerce services, where a large number of users browse anonymously or may have very distinct interests for different sessions. In this paper we present one of the winning solutions for the Recommendation task of the SIGIR 2021 Workshop on E-commerce Data Challenge. Our solution was inspired by NLP techniques and consists of an ensemble of two Transformer architectures - Transformer-XL and XLNet - trained with autoregressive and autoencoding approaches. To leverage most of the rich dataset made available for the competition, we describe how we prepared multi-model features by combining tabular events with textual and image vectors. We also present a model prediction analysis to better understand the effectiveness of our architectures for the session-based recommendation. △ Less

Submitted 11 July, 2021; originally announced July 2021.

Comments: In Proceedings of SIGIR eCom'21 - SIGIR eCommerce Workshop Data Challenge 2021. https://sigir-ecom.github.io/

Social Distancing and the Internet: What Can Network Performance Measurements Tell Us?

Authors: Jessica De Oliveira Moreira, Amey Praveen Pasarkar, Wenjun Chen, Wenkai Hu, Jan Janak, Henning Schulzrinne

Abstract: The COVID-19 pandemic and related restrictions forced many to work, learn, and socialize from home over the internet. There appears to be consensus that internet infrastructure in the developed world handled the resulting traffic surge well. In this paper, we study network measurement data collected by the Federal Communications Commission's Measuring Broadband America program before and during th… ▽ More The COVID-19 pandemic and related restrictions forced many to work, learn, and socialize from home over the internet. There appears to be consensus that internet infrastructure in the developed world handled the resulting traffic surge well. In this paper, we study network measurement data collected by the Federal Communications Commission's Measuring Broadband America program before and during the pandemic in the United States (US). We analyze the data to understand the impact of lockdown orders on the performance of fixed broadband internet infrastructure across the US, and also attempt to correlate internet usage patterns with the changing behavior of users during lockdown. We found the key metrics such as change in data usage to be generally consistent with the literature. Through additional analysis, we found differences between metro and rural areas, changes in weekday, weekend, and hourly internet usage patterns, and indications of network congestion for some users. △ Less

Submitted 13 January, 2021; v1 submitted 17 December, 2020; originally announced December 2020.

Comments: 12 pages, submitted to TPRC48

A Multiperiod Workforce Scheduling and Routing Problem with Dependent Tasks

Authors: Dilson Lucas Pereira, Júlio César Alves, Mayron César de Oliveira Moreira

Abstract: In this paper, we study a new Workforce Scheduling and Routing Problem, denoted Multiperiod Workforce Scheduling and Routing Problem with Dependent Tasks. In this problem, customers request services from a company. Each service is composed of dependent tasks, which are executed by teams of varying skills along one or more days. Tasks belonging to a service may be executed by different teams, and c… ▽ More In this paper, we study a new Workforce Scheduling and Routing Problem, denoted Multiperiod Workforce Scheduling and Routing Problem with Dependent Tasks. In this problem, customers request services from a company. Each service is composed of dependent tasks, which are executed by teams of varying skills along one or more days. Tasks belonging to a service may be executed by different teams, and customers may be visited more than once a day, as long as precedences are not violated. The objective is to schedule and route teams so that the makespan is minimized, i.e., all services are completed in the minimum number of days. In order to solve this problem, we propose a Mixed-Integer Programming model, a constructive algorithm and heuristic algorithms based on the Ant Colony Optimization (ACO) metaheuristic. The presence of precedence constraints makes it difficult to develop efficient local search algorithms. This motivates the choice of the ACO metaheuristic, which is effective in guiding the construction process towards good solutions. Computational results show that the model is capable of consistently solving problems with up to about 20 customers and 60 tasks. In most cases, the best performing ACO algorithm was able to match the best solution provided by the model in a fraction of its computational time. △ Less

Submitted 6 August, 2020; originally announced August 2020.

Journal ref: Computers & Operations Research, Volume 118, 2020, 104930, ISSN 0305-0548

Hybrid Session-based News Recommendation using Recurrent Neural Networks

Authors: Gabriel de Souza P. Moreira, Dietmar Jannach, Adilson Marques da Cunha

Abstract: We describe a hybrid meta-architecture -- the CHAMELEON -- for session-based news recommendation that is able to leverage a variety of information types using Recurrent Neural Networks. We evaluated our approach on two public datasets, using a temporal evaluation protocol that simulates the dynamics of a news portal in a realistic way. Our results confirm the benefits of modeling the sequence of s… ▽ More We describe a hybrid meta-architecture -- the CHAMELEON -- for session-based news recommendation that is able to leverage a variety of information types using Recurrent Neural Networks. We evaluated our approach on two public datasets, using a temporal evaluation protocol that simulates the dynamics of a news portal in a realistic way. Our results confirm the benefits of modeling the sequence of session clicks with RNNs and leveraging side information about users and articles, resulting in significantly higher recommendation accuracy and catalog coverage than other session-based algorithms. △ Less

Submitted 22 June, 2020; originally announced June 2020.

Comments: From the Proceeding of the LatinX in AI Research (LXAI) at ICML 2020. arXiv admin note: text overlap with arXiv:1904.10367

Automatic Discovery of Political Meme Genres with Diverse Appearances

Authors: William Theisen, Joel Brogan, Pamela Bilo Thomas, Daniel Moreira, Pascal Phoa, Tim Weninger, Walter Scheirer

Abstract: Forms of human communication are not static -- we expect some evolution in the way information is conveyed over time because of advances in technology. One example of this phenomenon is the image-based meme, which has emerged as a dominant form of political messaging in the past decade. While originally used to spread jokes on social media, memes are now having an outsized impact on public percept… ▽ More Forms of human communication are not static -- we expect some evolution in the way information is conveyed over time because of advances in technology. One example of this phenomenon is the image-based meme, which has emerged as a dominant form of political messaging in the past decade. While originally used to spread jokes on social media, memes are now having an outsized impact on public perception of world events. A significant challenge in automatic meme analysis has been the development of a strategy to match memes from within a single genre when the appearances of the images vary. Such variation is especially common in memes exhibiting mimicry. For example, when voters perform a common hand gesture to signal their support for a candidate. In this paper we introduce a scalable automated visual recognition pipeline for discovering political meme genres of diverse appearance. This pipeline can ingest meme images from a social network, apply computer vision-based techniques to extract local features and index new images into a database, and then organize the memes into related genres. To validate this approach, we perform a large case study on the 2019 Indonesian Presidential Election using a new dataset of over two million images collected from Twitter and Instagram. Results show that this approach can discover new meme genres with visually diverse images that share common stylistic elements, paving the way forward for further work in semantic analysis and content attribution. △ Less

Submitted 10 September, 2020; v1 submitted 16 January, 2020; originally announced January 2020.

Comments: 13 pages, 14 figures

CHAMELEON: A Deep Learning Meta-Architecture for News Recommender Systems [Phd. Thesis]

Authors: Gabriel de Souza Pereira Moreira

Abstract: Recommender Systems (RS) have became a popular research topic and, since 2016, Deep Learning methods and techniques have been increasingly explored in this area. News RS are aimed to personalize users experiences and help them discover relevant articles from a large and dynamic search space. The main contribution of this research was named CHAMELEON, a Deep Learning meta-architecture designed to t… ▽ More Recommender Systems (RS) have became a popular research topic and, since 2016, Deep Learning methods and techniques have been increasingly explored in this area. News RS are aimed to personalize users experiences and help them discover relevant articles from a large and dynamic search space. The main contribution of this research was named CHAMELEON, a Deep Learning meta-architecture designed to tackle the specific challenges of news recommendation. It consists of a modular reference architecture which can be instantiated using different neural building blocks. As information about users' past interactions is scarce in the news domain, the user context can be leveraged to deal with the user cold-start problem. Articles' content is also important to tackle the item cold-start problem. Additionally, the temporal decay of items (articles) relevance is very accelerated in the news domain. Furthermore, external breaking events may temporally attract global readership attention, a phenomenon generally known as concept drift in machine learning. All those characteristics are explicitly modeled on this research by a contextual hybrid session-based recommendation approach using Recurrent Neural Networks. The task addressed by this research is session-based news recommendation, i.e., next-click prediction using only information available in the current user session. A method is proposed for a realistic temporal offline evaluation of such task, replaying the stream of user clicks and fresh articles being continuously published in a news portal. Experiments performed with two large datasets have shown the effectiveness of the CHAMELEON for news recommendation on many quality factors such as accuracy, item coverage, novelty, and reduced item cold-start problem, when compared to other traditional and state-of-the-art session-based recommendation algorithms. △ Less

Submitted 29 December, 2019; originally announced January 2020.

Comments: Phd. Thesis presented on Dec. 09, 2019 to the Instituto Tecnológico de Aeronáutica (ITA), in partial fulfillment of the requirements for the degree of Doctor of Science in the Graduate Program of Electronics and Computing Engineering, Field of Informatics

Report number: DCTA/ITA/TD-035/2019

Learning Transformation-Aware Embeddings for Image Forensics

Authors: Aparna Bharati, Daniel Moreira, Patrick Flynn, Anderson Rocha, Kevin Bowyer, Walter Scheirer

Abstract: A dramatic rise in the flow of manipulated image content on the Internet has led to an aggressive response from the media forensics research community. New efforts have incorporated increased usage of techniques from computer vision and machine learning to detect and profile the space of image manipulations. This paper addresses Image Provenance Analysis, which aims at discovering relationships am… ▽ More A dramatic rise in the flow of manipulated image content on the Internet has led to an aggressive response from the media forensics research community. New efforts have incorporated increased usage of techniques from computer vision and machine learning to detect and profile the space of image manipulations. This paper addresses Image Provenance Analysis, which aims at discovering relationships among different manipulated image versions that share content. One of the main sub-problems for provenance analysis that has not yet been addressed directly is the edit ordering of images that share full content or are near-duplicates. The existing large networks that generate image descriptors for tasks such as object recognition may not encode the subtle differences between these image covariates. This paper introduces a novel deep learning-based approach to provide a plausible ordering to images that have been generated from a single image through transformations. Our approach learns transformation-aware descriptors using weak supervision via composited transformations and a rank-based quadruplet loss. To establish the efficacy of the proposed approach, comparisons with state-of-the-art handcrafted and deep learning-based descriptors, and image matching approaches are made. Further experimentation validates the proposed approach in the context of image provenance analysis. △ Less

Submitted 13 January, 2020; originally announced January 2020.

Comments: Supplemental material for this paper is available at https://drive.google.com/file/d/1covDhaTN24zkmyQf1XCTZHNrUZdZqGyo/view?usp=sharing

On the Importance of News Content Representation in Hybrid Neural Session-based Recommender Systems

Authors: Gabriel de Souza P. Moreira, Dietmar Jannach, Adilson Marques da Cunha

Abstract: News recommender systems are designed to surface relevant information for online readers by personalizing their user experiences. A particular problem in that context is that online readers are often anonymous, which means that this personalization can only be based on the last few recorded interactions with the user, a setting named session-based recommendation. Another particularity of the news… ▽ More News recommender systems are designed to surface relevant information for online readers by personalizing their user experiences. A particular problem in that context is that online readers are often anonymous, which means that this personalization can only be based on the last few recorded interactions with the user, a setting named session-based recommendation. Another particularity of the news domain is that constantly fresh articles are published, which should be immediately considered for recommendation. To deal with this item cold-start problem, it is important to consider the actual content of items when recommending. Hybrid approaches are therefore often considered as the method of choice in such settings. In this work, we analyze the importance of considering content information in a hybrid neural news recommender system. We contrast content-aware and content-agnostic techniques and also explore the effects of using different content encodings. Experiments on two public datasets confirm the importance of adopting a hybrid approach. Furthermore, we show that the choice of the content encoding can have an impact on the resulting performance. △ Less

Submitted 6 September, 2019; v1 submitted 12 July, 2019; originally announced July 2019.

Comments: Short paper. In 7th International Workshop on News Recommendation and Analytics (INRA 2019), in conjunction with RecSys 2019, September 19, 2019, Copenhagen, Denmark. arXiv admin note: text overlap with arXiv:1904.10367

Contextual Hybrid Session-based News Recommendation with Recurrent Neural Networks

Authors: Gabriel de Souza Pereira Moreira, Dietmar Jannach, Adilson Marques da Cunha

Abstract: Recommender systems help users deal with information overload by providing tailored item suggestions to them. The recommendation of news is often considered to be challenging, since the relevance of an article for a user can depend on a variety of factors, including the user's short-term reading interests, the reader's context, or the recency or popularity of an article. Previous work has shown th… ▽ More Recommender systems help users deal with information overload by providing tailored item suggestions to them. The recommendation of news is often considered to be challenging, since the relevance of an article for a user can depend on a variety of factors, including the user's short-term reading interests, the reader's context, or the recency or popularity of an article. Previous work has shown that the use of Recurrent Neural Networks is promising for the next-in-session prediction task, but has certain limitations when only recorded item click sequences are used as input. In this work, we present a contextual hybrid, deep learning based approach for session-based news recommendation that is able to leverage a variety of information types. We evaluated our approach on two public datasets, using a temporal evaluation protocol that simulates the dynamics of a news portal in a realistic way. Our results confirm the benefits of considering additional types of information, including article popularity and recency, in the proposed way, resulting in significantly higher recommendation accuracy and catalog coverage than other session-based algorithms. Additional experiments show that the proposed parameterizable loss function used in our method also allows us to balance two usually conflicting quality factors, accuracy and novelty. Keywords: Artificial Neural Networks, Context-Aware Recommender Systems, Hybrid Recommender Systems, News Recommender Systems, Session-based Recommendation △ Less

Submitted 8 December, 2019; v1 submitted 15 April, 2019; originally announced April 2019.

Comments: 20 pgs. Published at IEEE Access, Volume 7, 2019. https://ieeexplore.ieee.org/document/8908688

Journal ref: IEEE Access 7 (2019): 169185-169203

Dynamic Spatial Verification for Large-Scale Object-Level Image Retrieval

Authors: Joel Brogan, Aparna Bharati, Daniel Moreira, Kevin Bowyer, Patrick Flynn, Anderson Rocha, Walter Scheirer

Abstract: Images from social media can reflect diverse viewpoints, heated arguments, and expressions of creativity, adding new complexity to retrieval tasks. Researchers working onContent-Based Image Retrieval (CBIR) have traditionally tuned their algorithms to match filtered results with user search intent. However, we are now bombarded with composite images of unknown origin, authenticity, and even meanin… ▽ More Images from social media can reflect diverse viewpoints, heated arguments, and expressions of creativity, adding new complexity to retrieval tasks. Researchers working onContent-Based Image Retrieval (CBIR) have traditionally tuned their algorithms to match filtered results with user search intent. However, we are now bombarded with composite images of unknown origin, authenticity, and even meaning. With such uncertainty, users may not have an initial idea of what the results of a search query should look like. For instance, hidden people, spliced objects, and subtly altered scenes can be difficult for a user to detect initially in a meme image, but may contribute significantly to its composition. We propose a new approach for spatial verification that aims at modeling object-level regions dynamically clustering keypoints in a 2D Hough space, which are then used to accurately weight small contributing objects within the results, without the need for costly object detection steps. We call this method Objects in Scene to Objects in Scene (OS2OS) score, and it is optimized for fast matrix operations on CPUs. OS2OS performs comparably to state-of-the-art methods in classic CBIR problems, on the Oxford5K, Paris 6K, and Google-Landmarks datasets, without the need for bounding boxes. It also succeeds in emerging retrieval tasks such as image composite matching in the NIST MFC2018 dataset and meme-style composite imagery fromReddit. △ Less

Submitted 2 December, 2019; v1 submitted 24 March, 2019; originally announced March 2019.

News Session-Based Recommendations using Deep Neural Networks

Authors: Gabriel de Souza P. Moreira, Felipe Ferreira, Adilson Marques da Cunha

Abstract: News recommender systems are aimed to personalize users experiences and help them to discover relevant articles from a large and dynamic search space. Therefore, news domain is a challenging scenario for recommendations, due to its sparse user profiling, fast growing number of items, accelerated item's value decay, and users preferences dynamic shift. Some promising results have been recently achi… ▽ More News recommender systems are aimed to personalize users experiences and help them to discover relevant articles from a large and dynamic search space. Therefore, news domain is a challenging scenario for recommendations, due to its sparse user profiling, fast growing number of items, accelerated item's value decay, and users preferences dynamic shift. Some promising results have been recently achieved by the usage of Deep Learning techniques on Recommender Systems, specially for item's feature extraction and for session-based recommendations with Recurrent Neural Networks. In this paper, it is proposed an instantiation of the CHAMELEON -- a Deep Learning Meta-Architecture for News Recommender Systems. This architecture is composed of two modules, the first responsible to learn news articles representations, based on their text and metadata, and the second module aimed to provide session-based recommendations using Recurrent Neural Networks. The recommendation task addressed in this work is next-item prediction for users sessions: "what is the next most likely article a user might read in a session?" Users sessions context is leveraged by the architecture to provide additional information in such extreme cold-start scenario of news recommendation. Users' behavior and item features are both merged in an hybrid recommendation approach. A temporal offline evaluation method is also proposed as a complementary contribution, for a more realistic evaluation of such task, considering dynamic factors that affect global readership interests like popularity, recency, and seasonality. Experiments with an extensive number of session-based recommendation methods were performed and the proposed instantiation of CHAMELEON meta-architecture obtained a significant relative improvement in top-n accuracy and ranking metrics (10% on Hit Rate and 13% on MRR) over the best benchmark methods. △ Less

Submitted 16 September, 2018; v1 submitted 31 July, 2018; originally announced August 2018.

Comments: Accepted for the Third Workshop on Deep Learning for Recommender Systems - DLRS 2018, October 02-07, 2018, Vancouver, Canada. https://recsys.acm.org/recsys18/dlrs/

Domain-Specific Human-Inspired Binarized Statistical Image Features for Iris Recognition

Authors: Adam Czajka, Daniel Moreira, Kevin W. Bowyer, Patrick J. Flynn

Abstract: Binarized statistical image features (BSIF) have been successfully used for texture analysis in many computer vision tasks, including iris recognition and biometric presentation attack detection. One important point is that all applications of BSIF in iris recognition have used the original BSIF filters, which were trained on image patches extracted from natural images. This paper tests the questi… ▽ More Binarized statistical image features (BSIF) have been successfully used for texture analysis in many computer vision tasks, including iris recognition and biometric presentation attack detection. One important point is that all applications of BSIF in iris recognition have used the original BSIF filters, which were trained on image patches extracted from natural images. This paper tests the question of whether domain-specific BSIF can give better performance than the default BSIF. The second important point is in the selection of image patches to use in training for BSIF. Can image patches derived from eye-tracking experiments, in which humans perform an iris recognition task, give better performance than random patches? Our results say that (1) domain-specific BSIF features can out-perform the default BSIF features, and (2) selecting image patches in a task-specific manner guided by human performance can out-perform selecting random patches. These results are important because BSIF is often regarded as a generic texture tool that does not need any domain adaptation, and human-task-guided selection of patches for training has never (to our knowledge) been done. This paper follows the reproducible research requirements, and the new iris-domain-specific BSIF filters, the patches used in filter training, the database used in testing and the source codes of the designed iris recognition method are made available along with this paper to facilitate applications of this concept. △ Less

Submitted 17 November, 2018; v1 submitted 13 July, 2018; originally announced July 2018.

Comments: Paper accepted for WACV 2019, Hawaii, USA

Performance of Humans in Iris Recognition: The Impact of Iris Condition and Annotation-driven Verification

Authors: Daniel Moreira, Mateusz Trokielewicz, Adam Czajka, Kevin W. Bowyer, Patrick J. Flynn

Abstract: This paper advances the state of the art in human examination of iris images by (1) assessing the impact of different iris conditions in identity verification, and (2) introducing an annotation step that improves the accuracy of people's decisions. In a first experimental session, 114 subjects were asked to decide if pairs of iris images depict the same eye (genuine pairs) or two distinct eyes (im… ▽ More This paper advances the state of the art in human examination of iris images by (1) assessing the impact of different iris conditions in identity verification, and (2) introducing an annotation step that improves the accuracy of people's decisions. In a first experimental session, 114 subjects were asked to decide if pairs of iris images depict the same eye (genuine pairs) or two distinct eyes (impostor pairs). The image pairs sampled six conditions: (1) easy for algorithms to classify, (2) difficult for algorithms to classify, (3) large difference in pupil dilation, (4) disease-affected eyes, (5) identical twins, and (6) post-mortem samples. In a second session, 85 of the 114 subjects were asked to annotate matching and non-matching regions that supported their decisions. Subjects were allowed to change their initial classification as a result of the annotation process. Results suggest that: (a) people improve their identity verification accuracy when asked to annotate matching and non-matching regions between the pair of images, (b) images depicting the same eye with large difference in pupil dilation were the most challenging to subjects, but benefited well from the annotation-driven classification, (c) humans performed better than iris recognition algorithms when verifying genuine pairs of post-mortem and disease-affected eyes (i.e., samples showing deformations that go beyond the distortions of a healthy iris due to pupil dilation), and (d) annotation does not improve accuracy of analyzing images from identical twins, which remain confusing for people. △ Less

Submitted 20 November, 2018; v1 submitted 13 July, 2018; originally announced July 2018.

Comments: Paper accepted for WACV 2019, Hawaii, USA

Beyond Pixels: Image Provenance Analysis Leveraging Metadata

Authors: Aparna Bharati, Daniel Moreira, Joel Brogan, Patricia Hale, Kevin W. Bowyer, Patrick J. Flynn, Anderson Rocha, Walter J. Scheirer

Abstract: Creative works, whether paintings or memes, follow unique journeys that result in their final form. Understanding these journeys, a process known as "provenance analysis", provides rich insights into the use, motivation, and authenticity underlying any given work. The application of this type of study to the expanse of unregulated content on the Internet is what we consider in this paper. Provenan… ▽ More Creative works, whether paintings or memes, follow unique journeys that result in their final form. Understanding these journeys, a process known as "provenance analysis", provides rich insights into the use, motivation, and authenticity underlying any given work. The application of this type of study to the expanse of unregulated content on the Internet is what we consider in this paper. Provenance analysis provides a snapshot of the chronology and validity of content as it is uploaded, re-uploaded, and modified over time. Although still in its infancy, automated provenance analysis for online multimedia is already being applied to different types of content. Most current works seek to build provenance graphs based on the shared content between images or videos. This can be a computationally expensive task, especially when considering the vast influx of content that the Internet sees every day. Utilizing non-content-based information, such as timestamps, geotags, and camera IDs can help provide important insights into the path a particular image or video has traveled during its time on the Internet without large computational overhead. This paper tests the scope and applicability of metadata-based inferences for provenance graph construction in two different scenarios: digital image forensics and cultural analytics. △ Less

Submitted 6 March, 2019; v1 submitted 9 July, 2018; originally announced July 2018.

Comments: Supplemental material for this paper can be found at https://drive.google.com/file/d/1Tbs2CQg_VQAc2PdztW5twVaiXD0G12-H/view?usp=sharing

Getting the subtext without the text: Scalable multimodal sentiment classification from visual and acoustic modalities

Authors: Nathaniel Blanchard, Daniel Moreira, Aparna Bharati, Walter J. Scheirer

Abstract: In the last decade, video blogs (vlogs) have become an extremely popular method through which people express sentiment. The ubiquitousness of these videos has increased the importance of multimodal fusion models, which incorporate video and audio features with traditional text features for automatic sentiment detection. Multimodal fusion offers a unique opportunity to build models that learn from… ▽ More In the last decade, video blogs (vlogs) have become an extremely popular method through which people express sentiment. The ubiquitousness of these videos has increased the importance of multimodal fusion models, which incorporate video and audio features with traditional text features for automatic sentiment detection. Multimodal fusion offers a unique opportunity to build models that learn from the full depth of expression available to human viewers. In the detection of sentiment in these videos, acoustic and video features provide clarity to otherwise ambiguous transcripts. In this paper, we present a multimodal fusion model that exclusively uses high-level video and audio features to analyze spoken sentences for sentiment. We discard traditional transcription features in order to minimize human intervention and to maximize the deployability of our model on at-scale real-world data. We select high-level features for our model that have been successful in nonaffect domains in order to test their generalizability in the sentiment detection domain. We train and test our model on the newly released CMU Multimodal Opinion Sentiment and Emotion Intensity (CMUMOSEI) dataset, obtaining an F1 score of 0.8049 on the validation set and an F1 score of 0.6325 on the held-out challenge test set. △ Less

Submitted 3 July, 2018; originally announced July 2018.

Comments: Published in the First Workshop on Computational Modeling of Human Multimodal Language - ACL 2018

Image Provenance Analysis at Scale

Authors: Daniel Moreira, Aparna Bharati, Joel Brogan, Allan Pinto, Michael Parowski, Kevin W. Bowyer, Patrick J. Flynn, Anderson Rocha, Walter J. Scheirer

Abstract: Prior art has shown it is possible to estimate, through image processing and computer vision techniques, the types and parameters of transformations that have been applied to the content of individual images to obtain new images. Given a large corpus of images and a query image, an interesting further step is to retrieve the set of original images whose content is present in the query image, as we… ▽ More Prior art has shown it is possible to estimate, through image processing and computer vision techniques, the types and parameters of transformations that have been applied to the content of individual images to obtain new images. Given a large corpus of images and a query image, an interesting further step is to retrieve the set of original images whose content is present in the query image, as well as the detailed sequences of transformations that yield the query image given the original images. This is a problem that recently has received the name of image provenance analysis. In these times of public media manipulation ( e.g., fake news and meme sharing), obtaining the history of image transformations is relevant for fact checking and authorship verification, among many other applications. This article presents an end-to-end processing pipeline for image provenance analysis, which works at real-world scale. It employs a cutting-edge image filtering solution that is custom-tailored for the problem at hand, as well as novel techniques for obtaining the provenance graph that expresses how the images, as nodes, are ancestrally connected. A comprehensive set of experiments for each stage of the pipeline is provided, comparing the proposed solution with state-of-the-art results, employing previously published datasets. In addition, this work introduces a new dataset of real-world provenance cases from the social media site Reddit, along with baseline results. △ Less

Submitted 23 January, 2018; v1 submitted 19 January, 2018; originally announced January 2018.

Comments: 13 pages, 6 figures

Non-locality of the meet levels of the Trotter-Weil Hierarchy

Authors: João Daniel Moreira

Abstract: We prove that the meet level $m$ of the Trotter-Weil, $\mathsf{V}_m$ is not local for all $m \geq 1$, as conjectured in a paper by Kufleitner and Lauser. In order to show this, we explicitly provide a language whose syntactic semigroup is in $L \mathsf{V}_m$ and not in $\mathsf{V}_m*\mathsf{D}$. We prove that the meet level $m$ of the Trotter-Weil, $\mathsf{V}_m$ is not local for all $m \geq 1$, as conjectured in a paper by Kufleitner and Lauser. In order to show this, we explicitly provide a language whose syntactic semigroup is in $L \mathsf{V}_m$ and not in $\mathsf{V}_m*\mathsf{D}$. △ Less

Submitted 14 July, 2017; originally announced July 2017.

Provenance Filtering for Multimedia Phylogeny

Authors: Allan Pinto, Daniel Moreira, Aparna Bharati, Joel Brogan, Kevin Bowyer, Patrick Flynn, Walter Scheirer, Anderson Rocha

Abstract: Departing from traditional digital forensics modeling, which seeks to analyze single objects in isolation, multimedia phylogeny analyzes the evolutionary processes that influence digital objects and collections over time. One of its integral pieces is provenance filtering, which consists of searching a potentially large pool of objects for the most related ones with respect to a given query, in te… ▽ More Departing from traditional digital forensics modeling, which seeks to analyze single objects in isolation, multimedia phylogeny analyzes the evolutionary processes that influence digital objects and collections over time. One of its integral pieces is provenance filtering, which consists of searching a potentially large pool of objects for the most related ones with respect to a given query, in terms of possible ancestors (donors or contributors) and descendants. In this paper, we propose a two-tiered provenance filtering approach to find all the potential images that might have contributed to the creation process of a given query $q$. In our solution, the first (coarse) tier aims to find the most likely "host" images --- the major donor or background --- contributing to a composite/doctored image. The search is then refined in the second tier, in which we search for more specific (potentially small) parts of the query that might have been extracted from other images and spliced into the query image. Experimental results with a dataset containing more than a million images show that the two-tiered solution underpinned by the context of the query is highly useful for solving this difficult task. △ Less

Submitted 1 June, 2017; originally announced June 2017.

Comments: 5 pages, Accepted in IEEE International Conference on Image Processing (ICIP), 2017

U-Phylogeny: Undirected Provenance Graph Construction in the Wild

Authors: Aparna Bharati, Daniel Moreira, Allan Pinto, Joel Brogan, Kevin Bowyer, Patrick Flynn, Walter Scheirer, Anderson Rocha

Abstract: Deriving relationships between images and tracing back their history of modifications are at the core of Multimedia Phylogeny solutions, which aim to combat misinformation through doctored visual media. Nonetheless, most recent image phylogeny solutions cannot properly address cases of forged composite images with multiple donors, an area known as multiple parenting phylogeny (MPP). This paper pre… ▽ More Deriving relationships between images and tracing back their history of modifications are at the core of Multimedia Phylogeny solutions, which aim to combat misinformation through doctored visual media. Nonetheless, most recent image phylogeny solutions cannot properly address cases of forged composite images with multiple donors, an area known as multiple parenting phylogeny (MPP). This paper presents a preliminary undirected graph construction solution for MPP, without any strict assumptions. The algorithm is underpinned by robust image representative keypoints and different geometric consistency checks among matching regions in both images to provide regions of interest for direct comparison. The paper introduces a novel technique to geometrically filter the most promising matches as well as to aid in the shared region localization task. The strength of the approach is corroborated by experiments with real-world cases, with and without image distractors (unrelated cases). △ Less

Submitted 31 May, 2017; originally announced May 2017.

Comments: 5 pages, Accepted in International Conference on Image Processing, 2017

Spotting the Difference: Context Retrieval and Analysis for Improved Forgery Detection and Localization

Authors: Joel Brogan, Paolo Bestagini, Aparna Bharati, Allan Pinto, Daniel Moreira, Kevin Bowyer, Patrick Flynn, Anderson Rocha, Walter Scheirer

Abstract: As image tampering becomes ever more sophisticated and commonplace, the need for image forensics algorithms that can accurately and quickly detect forgeries grows. In this paper, we revisit the ideas of image querying and retrieval to provide clues to better localize forgeries. We propose a method to perform large-scale image forensics on the order of one million images using the help of an image… ▽ More As image tampering becomes ever more sophisticated and commonplace, the need for image forensics algorithms that can accurately and quickly detect forgeries grows. In this paper, we revisit the ideas of image querying and retrieval to provide clues to better localize forgeries. We propose a method to perform large-scale image forensics on the order of one million images using the help of an image search algorithm and database to gather contextual clues as to where tampering may have taken place. In this vein, we introduce five new strongly invariant image comparison methods and test their effectiveness under heavy noise, rotation, and color space changes. Lastly, we show the effectiveness of these methods compared to passive image forensics using Nimble [https://www.nist.gov/itl/iad/mig/nimble-challenge], a new, state-of-the-art dataset from the National Institute of Standards and Technology (NIST). △ Less

Submitted 1 May, 2017; originally announced May 2017.

Comments: 5 pages, 5 figures

Social Participation Ontology: community documentation, enhancements and use examples

Authors: Renato Fabbri, Henrique Parra Parra Filho, Rodrigo Bandeira de Luna, Ricardo Augusto Poppi Martins, Flor Karina Mamani Amanqui, Dilvan de Abreu Moreira, Osvaldo Novais de Oliveira Junior

Abstract: Participatory democracy advances in virtually all governments and especially in South America which exhibits a mixed culture and social predisposition. This article presents the "Social Participation Ontology" (OPS from the Brazilian name \emph{Ontologia de Participação Social}) implemented in compliance with the Web Ontology Language standard (OWL) for fostering social participation, specially in… ▽ More Participatory democracy advances in virtually all governments and especially in South America which exhibits a mixed culture and social predisposition. This article presents the "Social Participation Ontology" (OPS from the Brazilian name \emph{Ontologia de Participação Social}) implemented in compliance with the Web Ontology Language standard (OWL) for fostering social participation, specially in virtual platforms. The entities and links of OPS were defined based on an extensive collaboration of specialists. It is shown that OPS is instrumental for information retrieval from the contents of the portal, both in terms of the actors (at various levels) as well as mechanisms and activities. Significantly, OPS is linked to other OWL ontologies as an upper ontology and via FOAF and BFO as higher upper ontologies, which yields sound organization and access of knowledge and data. In order to illustrate the usefulness of OPS, we present results on ontological expansion and integration with other ontologies and data. Ongoing work involves further adoption of OPS by the official Brazilian federal portal for social participation and NGO s, and further linkage to other ontologies for social participation. △ Less

Submitted 30 October, 2017; v1 submitted 12 January, 2015; originally announced January 2015.

Comments: See ancillary for table of terms, OPS code and figures. Further information is at https://github.com/ttm/ops

Parallel WiSARD object tracker: a ram-based tracking system

Authors: Rodrigo da Silva Moreira, Nelson Francisco Favilla Ebecken

Abstract: This paper proposes the Parallel WiSARD Object Tracker (PWOT), a new object tracker based on the WiSARD weightless neural network that is robust against quantization errors. Object tracking in video is an important and challenging task in many applications. Difficulties can arise due to weather conditions, target trajectory and appearance, occlusions, lighting conditions and noise. Tracking is a h… ▽ More This paper proposes the Parallel WiSARD Object Tracker (PWOT), a new object tracker based on the WiSARD weightless neural network that is robust against quantization errors. Object tracking in video is an important and challenging task in many applications. Difficulties can arise due to weather conditions, target trajectory and appearance, occlusions, lighting conditions and noise. Tracking is a high-level application and requires the object location frame by frame in real time. This paper proposes a fast hybrid image segmentation (threshold and edge detection) in YcbCr color model and a parallel RAM based discriminator that improves efficiency when quantization errors occur. The original WiSARD training algorithm was changed to allow the tracking. △ Less

Submitted 12 March, 2014; originally announced March 2014.

Comments: 15 pages, 7 figures

Journal ref: Computer Science & Engineering: An International Journal (CSEIJ), Vol. 4, No. 1, February 2014