-
The NES Video-Music Database: A Dataset of Symbolic Video Game Music Paired with Gameplay Videos
Authors:
Igor Cardoso,
Rubens O. Moraes,
Lucas N. Ferreira
Abstract:
Neural models are one of the most popular approaches for music generation, yet there aren't standard large datasets tailored for learning music directly from game data. To address this research gap, we introduce a novel dataset named NES-VMDB, containing 98,940 gameplay videos from 389 NES games, each paired with its original soundtrack in symbolic format (MIDI). NES-VMDB is built upon the Nintend…
▽ More
Neural models are one of the most popular approaches for music generation, yet there aren't standard large datasets tailored for learning music directly from game data. To address this research gap, we introduce a novel dataset named NES-VMDB, containing 98,940 gameplay videos from 389 NES games, each paired with its original soundtrack in symbolic format (MIDI). NES-VMDB is built upon the Nintendo Entertainment System Music Database (NES-MDB), encompassing 5,278 music pieces from 397 NES games. Our approach involves collecting long-play videos for 389 games of the original dataset, slicing them into 15-second-long clips, and extracting the audio from each clip. Subsequently, we apply an audio fingerprinting algorithm (similar to Shazam) to automatically identify the corresponding piece in the NES-MDB dataset. Additionally, we introduce a baseline method based on the Controllable Music Transformer to generate NES music conditioned on gameplay clips. We evaluated this approach with objective metrics, and the results showed that the conditional CMT improves musical structural quality when compared to its unconditional counterpart. Moreover, we used a neural classifier to predict the game genre of the generated pieces. Results showed that the CMT generator can learn correlations between gameplay videos and game genres, but further research has to be conducted to achieve human-level performance.
△ Less
Submitted 5 April, 2024;
originally announced April 2024.
-
Choosing Well Your Opponents: How to Guide the Synthesis of Programmatic Strategies
Authors:
Rubens O. Moraes,
David S. Aleixo,
Lucas N. Ferreira,
Levi H. S. Lelis
Abstract:
This paper introduces Local Learner (2L), an algorithm for providing a set of reference strategies to guide the search for programmatic strategies in two-player zero-sum games. Previous learning algorithms, such as Iterated Best Response (IBR), Fictitious Play (FP), and Double-Oracle (DO), can be computationally expensive or miss important information for guiding search algorithms. 2L actively sel…
▽ More
This paper introduces Local Learner (2L), an algorithm for providing a set of reference strategies to guide the search for programmatic strategies in two-player zero-sum games. Previous learning algorithms, such as Iterated Best Response (IBR), Fictitious Play (FP), and Double-Oracle (DO), can be computationally expensive or miss important information for guiding search algorithms. 2L actively selects a set of reference strategies to improve the search signal. We empirically demonstrate the advantages of our approach while guiding a local search algorithm for synthesizing strategies in three games, including MicroRTS, a challenging real-time strategy game. Results show that 2L learns reference strategies that provide a stronger search signal than IBR, FP, and DO. We also simulate a tournament of MicroRTS, where a synthesizer using 2L outperformed the winners of the two latest MicroRTS competitions, which were programmatic strategies written by human programmers.
△ Less
Submitted 23 July, 2023; v1 submitted 10 July, 2023;
originally announced July 2023.
-
Beyond the shortest path: the path length index as a distribution
Authors:
Leonardo B. L. Santos,
Luiz Max Carvalho,
Giovanni G. Soares,
Leonardo N. Ferreira,
Igor M. Sokolov
Abstract:
The traditional complex network approach considers only the shortest paths from one node to another, not taking into account several other possible paths. This limitation is significant, for example, in urban mobility studies. In this short report, as the first steps, we present an exhaustive approach to address that problem and show we can go beyond the shortest path, but we do not need to go so…
▽ More
The traditional complex network approach considers only the shortest paths from one node to another, not taking into account several other possible paths. This limitation is significant, for example, in urban mobility studies. In this short report, as the first steps, we present an exhaustive approach to address that problem and show we can go beyond the shortest path, but we do not need to go so far: we present an interactive procedure and an early stop possibility. After presenting some fundamental concepts in graph theory, we presented an analytical solution for the problem of counting the number of possible paths between two nodes in complete graphs, and a depth-limited approach to get all possible paths between each pair of nodes in a general graph (an NP-hard problem). We do not collapse the distribution of path lengths between a pair of nodes into a scalar number, we look at the distribution itself - taking all paths up to a pre-defined path length (considering a truncated distribution), and show the impact of that approach on the most straightforward distance-based graph index: the walk/path length.
△ Less
Submitted 6 October, 2022;
originally announced October 2022.
-
From Time Series to Networks in R with the ts2net Package
Authors:
Leonardo N. Ferreira
Abstract:
Network science established itself as a prominent tool for modeling time series and complex systems. This modeling process consists of transforming a set or a single time series into a network. Nodes may represent complete time series, segments, or single values, while links define associations or similarities between the represented parts. R is one of the main programming languages used in data s…
▽ More
Network science established itself as a prominent tool for modeling time series and complex systems. This modeling process consists of transforming a set or a single time series into a network. Nodes may represent complete time series, segments, or single values, while links define associations or similarities between the represented parts. R is one of the main programming languages used in data science, statistics, and machine learning, with many packages available. However, no single package provides the necessary methods to transform time series into networks. This paper presents ts2net, an R package for modeling one or multiple time series into networks. The package provides the time series distance functions that can be easily computed in parallel and in supercomputers to process larger data sets and methods to transform distance matrices into networks. Ts2net also provides methods to transform a single time series into a network, such as recurrence networks, visibility graphs, and transition networks. Together with other packages, ts2net permits using network science and graph mining tools to extract information from time series.
△ Less
Submitted 20 August, 2022;
originally announced August 2022.
-
Controlling Perceived Emotion in Symbolic Music Generation with Monte Carlo Tree Search
Authors:
Lucas N. Ferreira,
Lili Mou,
Jim Whitehead,
Levi H. S. Lelis
Abstract:
This paper presents a new approach for controlling emotion in symbolic music generation with Monte Carlo Tree Search. We use Monte Carlo Tree Search as a decoding mechanism to steer the probability distribution learned by a language model towards a given emotion. At every step of the decoding process, we use Predictor Upper Confidence for Trees (PUCT) to search for sequences that maximize the aver…
▽ More
This paper presents a new approach for controlling emotion in symbolic music generation with Monte Carlo Tree Search. We use Monte Carlo Tree Search as a decoding mechanism to steer the probability distribution learned by a language model towards a given emotion. At every step of the decoding process, we use Predictor Upper Confidence for Trees (PUCT) to search for sequences that maximize the average values of emotion and quality as given by an emotion classifier and a discriminator, respectively. We use a language model as PUCT's policy and a combination of the emotion classifier and the discriminator as its value function. To decode the next token in a piece of music, we sample from the distribution of node visits created during the search. We evaluate the quality of the generated samples with respect to human-composed pieces using a set of objective metrics computed directly from the generated samples. We also perform a user study to evaluate how human subjects perceive the generated samples' quality and emotion. We compare PUCT against Stochastic Bi-Objective Beam Search (SBBS) and Conditional Sampling (CS). Results suggest that PUCT outperforms SBBS and CS in almost all metrics of music quality and emotion.
△ Less
Submitted 1 September, 2022; v1 submitted 10 August, 2022;
originally announced August 2022.
-
Método de Monte Carlo aplicado ao Cálculo Fracionário
Authors:
Luverci N. Ferreira,
Matheus J. Lazo
Abstract:
This article analyzes and develops a method to solve fractional ordinary differential equations using the Monte Carlo Method. A numerical simulation is performed for some differential equations, comparing the results with what exists in the mathematical literature. The Python language is used to create computational models.
This article analyzes and develops a method to solve fractional ordinary differential equations using the Monte Carlo Method. A numerical simulation is performed for some differential equations, comparing the results with what exists in the mathematical literature. The Python language is used to create computational models.
△ Less
Submitted 15 October, 2021;
originally announced October 2021.
-
Learning to Generate Music With Sentiment
Authors:
Lucas N. Ferreira,
Jim Whitehead
Abstract:
Deep Learning models have shown very promising results in automatically composing polyphonic music pieces. However, it is very hard to control such models in order to guide the compositions towards a desired goal. We are interested in controlling a model to automatically generate music with a given sentiment. This paper presents a generative Deep Learning model that can be directed to compose musi…
▽ More
Deep Learning models have shown very promising results in automatically composing polyphonic music pieces. However, it is very hard to control such models in order to guide the compositions towards a desired goal. We are interested in controlling a model to automatically generate music with a given sentiment. This paper presents a generative Deep Learning model that can be directed to compose music with a given sentiment. Besides music generation, the same model can be used for sentiment analysis of symbolic music. We evaluate the accuracy of the model in classifying sentiment of symbolic music using a new dataset of video game soundtracks. Results show that our model is able to obtain good prediction accuracy. A user study shows that human subjects agreed that the generated music has the intended sentiment, however negative pieces can be ambiguous.
△ Less
Submitted 8 March, 2021;
originally announced March 2021.
-
Dynamic Community Detection into Analyzing of Wildfires Events
Authors:
Alessandra Marli,
Didier A Vega-Oliveros,
Moshé Cotacallapa,
Leonardo N Ferreira,
Elbert EN Macau,
Marcos G Quiles
Abstract:
The study and comprehension of complex systems are crucial intellectual and scientific challenges of the 21st century. In this scenario, network science has emerged as a mathematical tool to support the study of such systems. Examples include environmental processes such as wildfires, which are known for their considerable impact on human life. However, there is a considerable lack of studies of w…
▽ More
The study and comprehension of complex systems are crucial intellectual and scientific challenges of the 21st century. In this scenario, network science has emerged as a mathematical tool to support the study of such systems. Examples include environmental processes such as wildfires, which are known for their considerable impact on human life. However, there is a considerable lack of studies of wildfire from a network science perspective. Here, employing the chronological network concept -- a temporal network where nodes are linked if two consecutive events occur between them -- we investigate the information that dynamic community structures reveal about the wildfires' dynamics. Particularly, we explore a two-phase dynamic community detection approach, i.e., we applied the Louvain algorithm on a series of snapshots. Then we used the Jaccard similarity coefficient to match communities across adjacent snapshots. Experiments with the MODIS dataset of fire events in the Amazon basing were conducted. Our results show that the dynamic communities can reveal wildfire patterns observed throughout the year.
△ Less
Submitted 2 November, 2020;
originally announced November 2020.
-
Computer-Generated Music for Tabletop Role-Playing Games
Authors:
Lucas N. Ferreira,
Levi H. S. Lelis,
Jim Whitehead
Abstract:
In this paper we present Bardo Composer, a system to generate background music for tabletop role-playing games. Bardo Composer uses a speech recognition system to translate player speech into text, which is classified according to a model of emotion. Bardo Composer then uses Stochastic Bi-Objective Beam Search, a variant of Stochastic Beam Search that we introduce in this paper, with a neural mode…
▽ More
In this paper we present Bardo Composer, a system to generate background music for tabletop role-playing games. Bardo Composer uses a speech recognition system to translate player speech into text, which is classified according to a model of emotion. Bardo Composer then uses Stochastic Bi-Objective Beam Search, a variant of Stochastic Beam Search that we introduce in this paper, with a neural model to generate musical pieces conveying the desired emotion. We performed a user study with 116 participants to evaluate whether people are able to correctly identify the emotion conveyed in the pieces generated by the system. In our study we used pieces generated for Call of the Wild, a Dungeons and Dragons campaign available on YouTube. Our results show that human subjects could correctly identify the emotion of the generated music pieces as accurately as they were able to identify the emotion of pieces written by humans.
△ Less
Submitted 16 August, 2020;
originally announced August 2020.
-
Spatiotemporal data analysis with chronological networks
Authors:
Leonardo N. Ferreira,
Didier A. Vega-Oliveros,
Moshe Cotacallapa,
Manoel F. Cardoso,
Marcos G. Quiles,
Liang Zhao,
Elbert E. N. Macau
Abstract:
The amount and size of spatiotemporal data sets from different domains have been rapidly increasing in the last years, which demands the development of robust and fast methods to analyze and extract information from them. In this paper, we propose a network-based model for spatiotemporal data analysis called chronnet. It consists of dividing a geometrical space into grid cells represented by nodes…
▽ More
The amount and size of spatiotemporal data sets from different domains have been rapidly increasing in the last years, which demands the development of robust and fast methods to analyze and extract information from them. In this paper, we propose a network-based model for spatiotemporal data analysis called chronnet. It consists of dividing a geometrical space into grid cells represented by nodes connected chronologically. The main goal of this model is to represent consecutive recurrent events between cells with strong links in the network. This representation permits the use of network science and graphing mining tools to extract information from spatiotemporal data. The chronnet construction process is fast, which makes it suitable for large data sets. In this paper, we describe how to use our model considering artificial and real data. For this purpose, we propose an artificial spatiotemporal data set generator to show how chronnets capture not just simple statistics, but also frequent patterns, spatial changes, outliers, and spatiotemporal clusters. Additionally, we analyze a real-world data set composed of global fire detections, in which we describe the frequency of fire events, outlier fire detections, and the seasonal activity, using a single chronnet.
△ Less
Submitted 12 August, 2020; v1 submitted 23 April, 2020;
originally announced April 2020.
-
Measuring the engagement level in encrypted group conversations by using temporal networks
Authors:
Moshe Cotacallapa,
Lilian Berton,
Leonardo N. Ferreira,
Marcos G. Quiles,
Liang Zhao,
Elbert E. N. Macau,
Didier A. Vega-Oliveros
Abstract:
Chat groups are well-known for their capacity to promote viral political and marketing campaigns, spread fake news, and create rallies by hundreds of thousands on the streets. Also, with the increasing public awareness regarding privacy and surveillance, many platforms have started to deploy end-to-end encrypted protocols. In this context, the group's conversations are not accessible in plain text…
▽ More
Chat groups are well-known for their capacity to promote viral political and marketing campaigns, spread fake news, and create rallies by hundreds of thousands on the streets. Also, with the increasing public awareness regarding privacy and surveillance, many platforms have started to deploy end-to-end encrypted protocols. In this context, the group's conversations are not accessible in plain text or readable format by third-party organizations or even the platform owner. Then, the main challenge that emerges is related to getting insights from users' activity of those groups, but without accessing the messages. Previous approaches evaluated the user engagement by assessing user's activity, however, on limited conditions where the data is encrypted, they cannot be applied. In this work, we present a framework for measuring the level of engagement of group conversations and users, without reading the messages. Our framework creates an ensemble of interaction networks that represent the temporal evolution of the conversation, then, we apply the proposed Engagement Index (EI) for each interval of conversations to assess users' participation. Our results in five datasets from real-world WhatsApp Groups indicate that, based on the EI, it is possible to identify the most engaged users within a time interval, create rankings, and group users according to their engagement and monitor their performance over time.
△ Less
Submitted 15 June, 2020; v1 submitted 20 June, 2019;
originally announced June 2019.
-
Global Fire Season Severity Analysis and Forecasting
Authors:
Leonardo N. Ferreira,
Didier A. Vega-Oliveros,
Liang Zhao,
Manoel F. Cardoso,
Elbert E. N. Macau
Abstract:
In this paper, we divide the globe into a hexagonal grid and we extracted time series of daily fire counts from each cell to estimate and analyze worldwide fire season severity (FSS), here defined as the accumulated fire detections in a season. The central question here is evaluating the accuracy of time series forecasting methods to estimate short-term (months) and medium-term (seasons) using onl…
▽ More
In this paper, we divide the globe into a hexagonal grid and we extracted time series of daily fire counts from each cell to estimate and analyze worldwide fire season severity (FSS), here defined as the accumulated fire detections in a season. The central question here is evaluating the accuracy of time series forecasting methods to estimate short-term (months) and medium-term (seasons) using only historical data of active fire detections. This approach is simple, fast, and use globally available data, making it easier for large scale prediction. Our results comprehend descriptive and predictive analyses of the worldwide seasonal fire activity. We verified that in 99% of the cells, the fire seasons have lengths shorter than seven months and that 57% have their lengths decrease. We also observed a declining tendency in the number of active fire counts during the seasons in 61% cells. However, some regions like the Northeast Brazil and the West Coast of the USA present an increasing trend. We verified that the forecasting error is lower than the mean FSS in 95% of the cells, indicating clear predictability in the FSS.
△ Less
Submitted 27 October, 2019; v1 submitted 11 March, 2019;
originally announced March 2019.
-
The Effect of Time Series Distance Functions on Functional Climate Networks
Authors:
Leonardo N. Ferreira,
Nicole C. R. Ferreira,
Elbert E. N. Macau,
Reik V. Donner
Abstract:
Complex network theory provides an important tool for the analysis of complex systems such as the Earth's climate. In this context, functional climate networks can be constructed using a spatiotemporal climate dataset and a suitable time series distance function. The resulting coarse-grained view on climate variability consists of representing distinct areas on the globe (i.e., grid cells) by node…
▽ More
Complex network theory provides an important tool for the analysis of complex systems such as the Earth's climate. In this context, functional climate networks can be constructed using a spatiotemporal climate dataset and a suitable time series distance function. The resulting coarse-grained view on climate variability consists of representing distinct areas on the globe (i.e., grid cells) by nodes and connecting pairs of nodes that present similar time series. One fundamental concern when constructing such a functional climate network is the definition of a metric that captures the mutual similarity between time series. Here we study systematically the effect of 29 time series distance functions on functional climate network construction based on global temperature data. We observe that the distance functions previously used in the literature commonly generate very similar networks while alternative ones result in rather distinct network structures and reveal different long-distance connection patterns. These patterns are highly important for the study of climate dynamics since they generally represent pathways for the long-distance transportation of energy and can be used to forecast climate variability on subseasonal to interannual or even decadal scales. Therefore, we propose the measures studied here as alternatives for the analysis of climate variability and to further exploit their complementary capability of capturing different aspects of the underlying dynamics that may help gaining a more holistic empirical understanding of the global climate system.
△ Less
Submitted 21 September, 2021; v1 submitted 8 February, 2019;
originally announced February 2019.
-
From spatio-temporal data to chronological networks: An application to wildfire analysis
Authors:
Didier A. Vega-Oliveros,
Moshé Cotacallapa,
Leonardo N. Ferreira,
Marcos Quiles,
Liang Zhao,
Elbert E. N. Macau,
Manoel F. Cardoso
Abstract:
Network theory has established itself as an appropriate tool for complex systems analysis and pattern recognition. In the context of spatiotemporal data analysis, correlation networks are used in the vast majority of works. However, the Pearson correlation coefficient captures only linear relationships and does not correctly capture recurrent events. This missed information is essential for tempor…
▽ More
Network theory has established itself as an appropriate tool for complex systems analysis and pattern recognition. In the context of spatiotemporal data analysis, correlation networks are used in the vast majority of works. However, the Pearson correlation coefficient captures only linear relationships and does not correctly capture recurrent events. This missed information is essential for temporal pattern recognition. In this work, we propose a chronological network construction process that is capable of capturing various events. Similar to the previous methods, we divide the area of study into grid cells and represent them by nodes. In our approach, links are established if two consecutive events occur in two different nodes. Our method is computationally efficient, adaptable to different time windows and can be applied to any spatiotemporal data set. As a proof-of-concept, we evaluated the proposed approach by constructing chronological networks from the MODIS dataset for fire events in the Amazon basin. We explore two data analytic approaches: one static and another temporal. The results show some activity patterns on the fire events and a displacement phenomenon over the year. The validity of the analyses in this application indicates that our data modeling approach is very promising for spatio-temporal data mining.
△ Less
Submitted 10 December, 2018; v1 submitted 4 December, 2018;
originally announced December 2018.
-
Long-range correlations and fractal dynamics in C. elegans: changes with aging and stress
Authors:
Luiz G. A. Alves,
Peter B. Winter,
Leonardo N. Ferreira,
Renée M. Brielmann,
Richard I. Morimoto,
Luís A. N. Amaral
Abstract:
Reduced motor control is one of the most frequent features associated with aging and disease. Nonlinear and fractal analyses have proved to be useful in investigating human physiological alterations with age and disease. Similar findings have not been established for any of the model organisms typically studied by biologists, though. If the physiology of a simpler model organism displays the same…
▽ More
Reduced motor control is one of the most frequent features associated with aging and disease. Nonlinear and fractal analyses have proved to be useful in investigating human physiological alterations with age and disease. Similar findings have not been established for any of the model organisms typically studied by biologists, though. If the physiology of a simpler model organism displays the same characteristics, this fact would open a new research window on the control mechanisms that organisms use to regulate physiological processes during aging and stress. Here, we use a recently introduced animal tracking technology to simultaneously follow tens of Caenorhabdits elegans for several hours and use tools from fractal physiology to quantitatively evaluate the effects of aging and temperature stress on nematode motility. Similarly to human physiological signals, scaling analysis reveals long-range correlations in numerous motility variables, fractal properties in behavioral shifts, and fluctuation dynamics over a wide range of timescales. These properties change as a result of a superposition of age and stress-related adaptive mechanisms that regulate motility.
△ Less
Submitted 15 August, 2017; v1 submitted 3 May, 2017;
originally announced May 2017.
-
Time Series Clustering via Community Detection in Networks
Authors:
Leonardo N. Ferreira,
Liang Zhao
Abstract:
In this paper, we propose a technique for time series clustering using community detection in complex networks. Firstly, we present a method to transform a set of time series into a network using different distance functions, where each time series is represented by a vertex and the most similar ones are connected. Then, we apply community detection algorithms to identify groups of strongly connec…
▽ More
In this paper, we propose a technique for time series clustering using community detection in complex networks. Firstly, we present a method to transform a set of time series into a network using different distance functions, where each time series is represented by a vertex and the most similar ones are connected. Then, we apply community detection algorithms to identify groups of strongly connected vertices (called a community) and, consequently, identify time series clusters. Still in this paper, we make a comprehensive analysis on the influence of various combinations of time series distance functions, network generation methods and community detection techniques on clustering results. Experimental study shows that the proposed network-based approach achieves better results than various classic or up-to-date clustering techniques under consideration. Statistical tests confirm that the proposed method outperforms some classic clustering algorithms, such as $k$-medoids, diana, median-linkage and centroid-linkage in various data sets. Interestingly, the proposed method can effectively detect shape patterns presented in time series due to the topological structure of the underlying network constructed in the clustering process. At the same time, other techniques fail to identify such patterns. Moreover, the proposed method is robust enough to group time series presenting similar pattern but with time shifts and/or amplitude variations. In summary, the main point of the proposed method is the transformation of time series from time-space domain to topological domain. Therefore, we hope that our approach contributes not only for time series clustering, but also for general time series analysis tasks.
△ Less
Submitted 19 August, 2015;
originally announced August 2015.
