-
Gransformer: Transformer-based Graph Generation
Authors:
Ahmad Khajenezhad,
Seyed Ali Osia,
Mahmood Karimian,
Hamid Beigy
Abstract:
Transformers have become widely used in various tasks, such as natural language processing and machine vision. This paper proposes Gransformer, an algorithm based on Transformer for generating graphs. We modify the Transformer encoder to exploit the structural information of the given graph. The attention mechanism is adapted to consider the presence or absence of edges between each pair of nodes.…
▽ More
Transformers have become widely used in various tasks, such as natural language processing and machine vision. This paper proposes Gransformer, an algorithm based on Transformer for generating graphs. We modify the Transformer encoder to exploit the structural information of the given graph. The attention mechanism is adapted to consider the presence or absence of edges between each pair of nodes. We also introduce a graph-based familiarity measure between node pairs that applies to both the attention and the positional encoding. This measure of familiarity is based on message-passing algorithms and contains structural information about the graph. Also, this measure is autoregressive, which allows our model to acquire the necessary conditional probabilities in a single forward pass. In the output layer, we also use a masked autoencoder for density estimation to efficiently model the sequential generation of dependent edges connected to each node. In addition, we propose a technique to prevent the model from generating isolated nodes without connection to preceding nodes by using BFS node orderings. We evaluate this method using synthetic and real-world datasets and compare it with related ones, including recurrent models and graph convolutional networks. Experimental results show that the proposed method performs comparatively to these methods.
△ Less
Submitted 30 May, 2024; v1 submitted 25 March, 2022;
originally announced March 2022.
-
Effects of Synaptic and Myelin Plasticity on Learning in a Network of Kuramoto Phase Oscillators
Authors:
Maryam Karimian,
Domenica Dibenedetto,
Michelle Moerel,
Thomas Burwick,
Ronald Westra,
Peter De Weerd,
Mario Senden
Abstract:
Models of learning typically focus on synaptic plasticity. However, learning is the result of both synaptic and myelin plasticity. Specifically, synaptic changes often co-occur and interact with myelin changes, leading to complex dynamic interactions between these processes. Here, we investigate the implications of these interactions for the coupling behavior of a system of Kuramoto oscillators. T…
▽ More
Models of learning typically focus on synaptic plasticity. However, learning is the result of both synaptic and myelin plasticity. Specifically, synaptic changes often co-occur and interact with myelin changes, leading to complex dynamic interactions between these processes. Here, we investigate the implications of these interactions for the coupling behavior of a system of Kuramoto oscillators. To that end, we construct a fully connected, one-dimensional ring network of phase oscillators whose coupling strength (reflecting synaptic strength) as well as conduction velocity (reflecting myelination) are each regulated by a Hebbian learning rule. We evaluate the behavior of the system in terms of structural (pairwise connection strength and conduction velocity) and functional connectivity (local and global synchronization behavior). We find that for conditions in which a system limited to synaptic plasticity develops two distinct clusters both structurally and functionally, additional adaptive myelination allows for functional communication across these structural clusters. Hence, dynamic conduction velocity permits the functional integration of structurally segregated clusters. Our results confirm that network states following learning may be different when myelin plasticity is considered in addition to synaptic plasticity, pointing towards the relevance of integrating both factors in computational models of learning.
△ Less
Submitted 17 February, 2019;
originally announced February 2019.
-
The effects of noise and time delay on the synchronization of the Kuramoto model in small-world networks
Authors:
Sara Ameli,
Farhad Shahbazi,
Maryam Karimian,
Tahereh Malakoutikhah
Abstract:
We study the synchronization of a small-world network of identical coupled phase oscillators with Kuramoto interaction. First, we consider the model with instantaneous mutual interaction and the normalized coupling constant to the degree of each node. For this model, similar to the constant coupling studied before, we find the existence of various attractors corresponding to the different defect p…
▽ More
We study the synchronization of a small-world network of identical coupled phase oscillators with Kuramoto interaction. First, we consider the model with instantaneous mutual interaction and the normalized coupling constant to the degree of each node. For this model, similar to the constant coupling studied before, we find the existence of various attractors corresponding to the different defect patterns and also the noise enhanced synchronization when driven by an external uncorrelated white noise. We also investigate the synchronization of the model with homogenous time-delay in the phase couplings. For a given intrinsic frequency and coupling constant, upon varying the time delay we observe the existence a partially synchronized state with defect patterns which transforms to an incoherent phase characterized by randomly phase locked states. By further increasing of the time delay, this phase again undergoes a transition to another patterned partially synchronized state. We show that the transition between theses phases are discontinuous and moreover in each phase the average frequency of the oscillators decreases by increasing the time delay and shows an upward jump at the transition points.
△ Less
Submitted 22 May, 2017;
originally announced May 2017.
