Showing 1–12 of 12 results for author: Figueiredo, D

Exact Recovery and Bregman Hard Clustering of Node-Attributed Stochastic Block Model

Authors: Maximilien Dreveton, Felipe S. Fernandes, Daniel R. Figueiredo

Abstract: Network clustering tackles the problem of identifying sets of nodes (communities) that have similar connection patterns. However, in many scenarios, nodes also have attributes that are correlated with the clustering structure. Thus, network information (edges) and node information (attributes) can be jointly leveraged to design high-performance clustering algorithms. Under a general model for the… ▽ More Network clustering tackles the problem of identifying sets of nodes (communities) that have similar connection patterns. However, in many scenarios, nodes also have attributes that are correlated with the clustering structure. Thus, network information (edges) and node information (attributes) can be jointly leveraged to design high-performance clustering algorithms. Under a general model for the network and node attributes, this work establishes an information-theoretic criterion for the exact recovery of community labels and characterizes a phase transition determined by the Chernoff-Hellinger divergence of the model. The criterion shows how network and attribute information can be exchanged in order to have exact recovery (e.g., more reliable network information requires less reliable attribute information). This work also presents an iterative clustering algorithm that maximizes the joint likelihood, assuming that the probability distribution of network interactions and node attributes belong to exponential families. This covers a broad range of possible interactions (e.g., edges with weights) and attributes (e.g., non-Gaussian models), as well as sparse networks, while also exploring the connection between exponential families and Bregman divergences. Extensive numerical experiments using synthetic data indicate that the proposed algorithm outperforms classic algorithms that leverage only network or only attribute information as well as state-of-the-art algorithms that also leverage both sources of information. The contributions of this work provide insights into the fundamental limits and practical techniques for inferring community labels on node-attributed networks. △ Less

Submitted 30 October, 2023; originally announced October 2023.

MSC Class: 62H30; 62F12

Journal ref: NeurIPS 2023

Optimizing Connectivity through Network Gradients for Restricted Boltzmann Machines

Authors: A. C. N. de Oliveira, D. R. Figueiredo

Abstract: Leveraging sparse networks to connect successive layers in deep neural networks has recently been shown to provide benefits to large scale state-of-the-art models. However, network connectivity also plays a significant role on the learning performance of shallow networks, such as the classic Restricted Boltzmann Machines (RBM). Efficiently finding sparse connectivity patterns that improve the lear… ▽ More Leveraging sparse networks to connect successive layers in deep neural networks has recently been shown to provide benefits to large scale state-of-the-art models. However, network connectivity also plays a significant role on the learning performance of shallow networks, such as the classic Restricted Boltzmann Machines (RBM). Efficiently finding sparse connectivity patterns that improve the learning performance of shallow networks is a fundamental problem. While recent principled approaches explicitly include network connections as model parameters that must be optimized, they often rely on explicit penalization or have network sparsity as a hyperparameter. This work presents the Network Connectivity Gradients (NCG), a method to find optimal connectivity patterns for RBMs based on the idea of network gradients: computing the gradient of every possible connection, given a specific connection pattern, and using the gradient to drive a continuous connection strength parameter that in turn is used to determine the connection pattern. Thus, learning RBM parameters and learning network connections is truly jointly performed, albeit with different learning rates, and without changes to the model's classic objective function. The method is applied to the MNIST and other data sets showing that better RBM models are found for the benchmark tasks of sample generation and input classification. Results also show that NCG is robust to network initialization, both adding and removing network connections while learning. △ Less

Submitted 3 December, 2022; v1 submitted 14 September, 2022; originally announced September 2022.

A transient equivalence between Aldous-Broder and Wilson's algorithms and a two-stage framework for generating uniform spanning trees

Authors: Igor Nunes, Giulio Iacobelli, Daniel Ratton Figueiredo

Abstract: The $Aldous\text{-}Broder$ and $Wilson$ are two well-known algorithms to generate uniform spanning trees (USTs) based on random walks. This work studies their relationship while they construct random trees with the goal of reducing the total time required to build the spanning tree. Using the notion of $branches$ $-$ paths generated by the two algorithms on particular stop** times, we show that… ▽ More The $Aldous\text{-}Broder$ and $Wilson$ are two well-known algorithms to generate uniform spanning trees (USTs) based on random walks. This work studies their relationship while they construct random trees with the goal of reducing the total time required to build the spanning tree. Using the notion of $branches$ $-$ paths generated by the two algorithms on particular stop** times, we show that the trees built by the two algorithms when running on a complete graph are statistically equivalent on these stop** times. This leads to a hybrid algorithm that can generate uniform spanning trees of complete graphs faster than either of the two algorithms. An efficient two-stage framework is also proposed to explore this hybrid approach beyond complete graphs, showing its feasibility in various examples, including transitive graphs where it requires 25% less time than $Wilson$ to generate a UST. △ Less

Submitted 24 June, 2022; originally announced June 2022.

Heterogeneous download times in bandwidth-homogeneous BitTorrent swarms

Authors: Fabricio Murai, Antonio A. de A. Rocha, Daniel R. Figueiredo, Edmundo A. de Souza e Silva

Abstract: Modeling and understanding BitTorrent (BT) dynamics is a recurrent research topic mainly due to its high complexity and tremendous practical efficiency. Over the years, different models have uncovered various phenomena exhibited by the system, many of which have direct impact on its performance. In this paper we identify and characterize a phenomenon that has not been previously observed: homogene… ▽ More Modeling and understanding BitTorrent (BT) dynamics is a recurrent research topic mainly due to its high complexity and tremendous practical efficiency. Over the years, different models have uncovered various phenomena exhibited by the system, many of which have direct impact on its performance. In this paper we identify and characterize a phenomenon that has not been previously observed: homogeneous peers (with respect to their upload capacities) experience heterogeneous download times. This behavior has direct impact on peer and system performance, such as high variability of download times, unfairness with respect to peer arrival order, bursty departures and content synchronization. Detailed packet-level simulations and prototype-based experiments on the Internet were performed to characterize this phenomenon. We also develop a mathematical model that accurately predicts the heterogeneous download rates of the homogeneous peers as a function of their content. In addition, we apply the model to calculate lower and upper bounds to the number of departures that occur in a burst. The heterogeneous download rates are more prevalent in unpopular swarms (very few peers). Although few works have addressed this kind of swarm, these by far represent the most common type of swarm in BT. △ Less

Submitted 17 September, 2021; originally announced September 2021.

Comments: Published in Computer Networks. arXiv admin note: substantial text overlap with arXiv:1102.3610

ACM Class: C.4; I.6

A Contextual Hierarchical Graph Model for Generating Random Sequences of Objects with Application to Music Playlists

Authors: Igor de Oliveira Nunes, Gabriel Matos Cardoso Leite, Daniel Ratton Figueiredo

Abstract: Recommending the right content in large scale multimedia streaming services is an important and challenging problem that has received much attention in the past decade. A key ingredient for successful recommendations is an effective similarity metric between two objects, and models that leverage the current context to constrain the recommendations. This work proposes a model for random object gene… ▽ More Recommending the right content in large scale multimedia streaming services is an important and challenging problem that has received much attention in the past decade. A key ingredient for successful recommendations is an effective similarity metric between two objects, and models that leverage the current context to constrain the recommendations. This work proposes a model for random object generation that introduces two key novel elements: (i) a similarity metric based on the distance between objects in a given object sequence, that is also used to measure similarity between meta-data associated with the objects, such as artists and genres; (ii) a hierarchical graph model with different graphs each associated with a different meta-data. A biased random walk in each graph that are coupled and synchronized dictate the random generation of objects, leveraging the current context to constrain randomness. The proposed model is fully parameterized from sequences of objects, requiring no external parameters or tuning. The model is applied to a large music dataset with over 1 million playlists generating a hierarchy with three layers (genre, artist, track). Results indicate its superiority in generating actual full playlists against two baseline models. △ Less

Submitted 11 November, 2019; originally announced November 2019.

struc2vec: Learning Node Representations from Structural Identity

Authors: Leonardo F. R. Ribeiro, Pedro H. P. Savarese, Daniel R. Figueiredo

Abstract: Structural identity is a concept of symmetry in which network nodes are identified according to the network structure and their relationship to other nodes. Structural identity has been studied in theory and practice over the past decades, but only recently has it been addressed with representational learning techniques. This work presents struc2vec, a novel and flexible framework for learning lat… ▽ More Structural identity is a concept of symmetry in which network nodes are identified according to the network structure and their relationship to other nodes. Structural identity has been studied in theory and practice over the past decades, but only recently has it been addressed with representational learning techniques. This work presents struc2vec, a novel and flexible framework for learning latent representations for the structural identity of nodes. struc2vec uses a hierarchy to measure node similarity at different scales, and constructs a multilayer graph to encode structural similarities and generate structural context for nodes. Numerical experiments indicate that state-of-the-art techniques for learning node representations fail in capturing stronger notions of structural identity, while struc2vec exhibits much superior performance in this task, as it overcomes limitations of prior approaches. As a consequence, numerical experiments indicate that struc2vec improves performance on classification tasks that depend more on structural identity. △ Less

Submitted 3 July, 2017; v1 submitted 11 April, 2017; originally announced April 2017.

Comments: 10 pages, KDD2017, Research Track

Learning Identity Map**s with Residual Gates

Authors: Pedro H. P. Savarese, Leonardo O. Mazza, Daniel R. Figueiredo

Abstract: We propose a new layer design by adding a linear gating mechanism to shortcut connections. By using a scalar parameter to control each gate, we provide a way to learn identity map**s by optimizing only one parameter. We build upon the motivation behind Residual Networks, where a layer is reformulated in order to make learning identity map**s less problematic to the optimizer. The augmentation… ▽ More We propose a new layer design by adding a linear gating mechanism to shortcut connections. By using a scalar parameter to control each gate, we provide a way to learn identity map**s by optimizing only one parameter. We build upon the motivation behind Residual Networks, where a layer is reformulated in order to make learning identity map**s less problematic to the optimizer. The augmentation introduces only one extra parameter per layer, and provides easier optimization by making degeneration into identity map**s simpler. We propose a new model, the Gated Residual Network, which is the result when augmenting Residual Networks. Experimental results show that augmenting layers provides better optimization, increased performance, and more layer independence. We evaluate our method on MNIST using fully-connected networks, showing empirical indications that our augmentation facilitates the optimization of deep models, and that it provides high tolerance to full layer removal: the model retains over 90% of its performance even after half of its layers have been randomly removed. We also evaluate our model on CIFAR-10 and CIFAR-100 using Wide Gated ResNets, achieving 3.65% and 18.27% error, respectively. △ Less

Submitted 28 December, 2016; v1 submitted 4 November, 2016; originally announced November 2016.

On the Emergence of Shortest Paths by Reinforced Random Walks

Authors: Daniel R. Figueiredo, Michele Garetto

Abstract: The co-evolution between network structure and functional performance is a fundamental and challenging problem whose complexity emerges from the intrinsic interdependent nature of structure and function. Within this context, we investigate the interplay between the efficiency of network navigation (i.e., path lengths) and network structure (i.e., edge weights). We propose a simple and tractable mo… ▽ More The co-evolution between network structure and functional performance is a fundamental and challenging problem whose complexity emerges from the intrinsic interdependent nature of structure and function. Within this context, we investigate the interplay between the efficiency of network navigation (i.e., path lengths) and network structure (i.e., edge weights). We propose a simple and tractable model based on iterative biased random walks where edge weights increase over time as function of the traversed path length. Under mild assumptions, we prove that biased random walks will eventually only traverse shortest paths in their journey towards the destination. We further characterize the transient regime proving that the probability to traverse non-shortest paths decays according to a power-law. We also highlight various properties in this dynamic, such as the trade-off between exploration and convergence, and preservation of initial network plasticity. We believe the proposed model and results can be of interest to various domains where biased random walks and decentralized navigation have been applied. △ Less

Submitted 9 May, 2016; originally announced May 2016.

On the Duration and Intensity of Competitions in Nonlinear Pólya Urn Processes with Fitness

Authors: Bo Jiang, Daniel R. Figueiredo, Bruno Ribeiro, Don Towsley

Abstract: Cumulative advantage (CA) refers to the notion that accumulated resources foster the accumulation of further resources in competitions, a phenomenon that has been empirically observed in various contexts. The oldest and arguably simplest mathematical model that embodies this general principle is the Pólya urn process, which finds applications in a myriad of problems. The original model captures th… ▽ More Cumulative advantage (CA) refers to the notion that accumulated resources foster the accumulation of further resources in competitions, a phenomenon that has been empirically observed in various contexts. The oldest and arguably simplest mathematical model that embodies this general principle is the Pólya urn process, which finds applications in a myriad of problems. The original model captures the dynamics of competitions between two equally fit agents under linear CA effects, which can be readily generalized to incorporate different fitnesses and nonlinear CA effects. We study two statistics of competitions under the generalized model, namely duration (i.e., time of the last tie) and intensity (i.e., number of ties). We give rigorous mathematical characterizations of the tail distributions of both duration and intensity under the various regimes for fitness and nonlinearity, which reveal very interesting behaviors. For example, fitness superiority induces much shorter competitions in the sublinear regime while much longer competitions in the superlinear regime. Our findings can shed light on the application of Pólya urn processes in more general contexts where fitness and nonlinearity may be present. △ Less

Submitted 7 April, 2017; v1 submitted 5 April, 2016; originally announced April 2016.

Power-law decay of the degree-sequence probabilities of multiple random graphs with application to graph isomorphism

Authors: Jefferson Elbert Simões, Daniel R. Figueiredo, Valmir C. Barbosa

Abstract: We consider events over the probability space generated by the degree sequences of multiple independent Erdős-Rényi random graphs, and consider an approximation probability space where such degree sequences are deemed to be sequences of i.i.d. random variables. We show that, for any sequence of events with probabilities asymptotically smaller than some power law in the approximation model, the sam… ▽ More We consider events over the probability space generated by the degree sequences of multiple independent Erdős-Rényi random graphs, and consider an approximation probability space where such degree sequences are deemed to be sequences of i.i.d. random variables. We show that, for any sequence of events with probabilities asymptotically smaller than some power law in the approximation model, the same upper bound also holds in the original model. We accomplish this by extending an approximation framework proposed in a seminal paper by McKay and Wormald. Finally, as an example, we apply the developed framework to bound the probability of isomorphism-related events over multiple independent random graphs. △ Less

Submitted 31 January, 2017; v1 submitted 11 January, 2016; originally announced January 2016.

Comments: (changes in v4: updated for multiple random graphs) (changes in v3: improvements in title and supporting material) (changes in v2: typo corrections)

MSC Class: 05C80

Journal ref: ESAIM: Probability and Statistics 21 (2017), 235-250

Characterizing Continuous Time Random Walks on Time Varying Graphs

Authors: Daniel Figueiredo, Philippe Nain, Bruno Ribeiro, Edmundo de Souza e Silva, Don Towsley

Abstract: In this paper we study the behavior of a continuous time random walk (CTRW) on a stationary and ergodic time varying dynamic graph. We establish conditions under which the CTRW is a stationary and ergodic process. In general, the stationary distribution of the walker depends on the walker rate and is difficult to characterize. However, we characterize the stationary distribution in the following c… ▽ More In this paper we study the behavior of a continuous time random walk (CTRW) on a stationary and ergodic time varying dynamic graph. We establish conditions under which the CTRW is a stationary and ergodic process. In general, the stationary distribution of the walker depends on the walker rate and is difficult to characterize. However, we characterize the stationary distribution in the following cases: i) the walker rate is significantly larger or smaller than the rate in which the graph changes (time-scale separation), ii) the walker rate is proportional to the degree of the node that it resides on (coupled dynamics), and iii) the degrees of node belonging to the same connected component are identical (structural constraints). We provide examples that illustrate our theoretical findings. △ Less

Submitted 2 December, 2012; v1 submitted 24 December, 2011; originally announced December 2011.

Report number: UM-CS-2012-011v2

Heterogeneous download times in a homogeneous BitTorrent swarm

Authors: Fabricio Murai, Antonio A de A Rocha, Daniel R. Figueiredo, Edmundo de Souza e Silva

Abstract: Modeling and understanding BitTorrent (BT) dynamics is a recurrent research topic mainly due to its high complexity and tremendous practical efficiency. Over the years, different models have uncovered various phenomena exhibited by the system, many of which have direct impact on its performance. In this paper we identify and characterize a phenomenon that has not been previously observed: homogene… ▽ More Modeling and understanding BitTorrent (BT) dynamics is a recurrent research topic mainly due to its high complexity and tremendous practical efficiency. Over the years, different models have uncovered various phenomena exhibited by the system, many of which have direct impact on its performance. In this paper we identify and characterize a phenomenon that has not been previously observed: homogeneous peers (with respect to their upload capacities) experience heterogeneous download rates. The consequences of this phenomenon have direct impact on peer and system performance, such as high variability of download times, unfairness with respect to peer arrival order, bursty departures and content synchronization. Detailed packet-level simulations and prototype-based experiments on the Internet were performed to characterize this phenomenon. We also develop a mathematical model that accurately predicts the heterogeneous download rates of the homogeneous peers as a function of their content. Although this phenomenon is more prevalent in unpopular swarms (very few peers), these by far represent the most common type of swarm in BT. △ Less

Submitted 18 February, 2011; v1 submitted 17 February, 2011; originally announced February 2011.

ACM Class: C.2.2; I.6.3; I.6.4