-
Control Theoretic Approach to Fine-Tuning and Transfer Learning
Authors:
Erkan Bayram,
Shenyu Liu,
Mohamed-Ali Belabbas,
Tamer Başar
Abstract:
Given a training set in the form of a paired $(\mathcal{X},\mathcal{Y})$, we say that the control system $\dot x = f(x,u)$ has learned the paired set via the control $u^*$ if the system steers each point of $\mathcal{X}$ to its corresponding target in $\mathcal{Y}$. If the training set is expanded, most existing methods for finding a new control $u^*$ require starting from scratch, resulting in a…
▽ More
Given a training set in the form of a paired $(\mathcal{X},\mathcal{Y})$, we say that the control system $\dot x = f(x,u)$ has learned the paired set via the control $u^*$ if the system steers each point of $\mathcal{X}$ to its corresponding target in $\mathcal{Y}$. If the training set is expanded, most existing methods for finding a new control $u^*$ require starting from scratch, resulting in a quadratic increase in complexity with the number of points. To overcome this limitation, we introduce the concept of $\textit{ tuning without forgetting}$. We develop $\textit{an iterative algorithm}$ to tune the control $u^*$ when the training set expands, whereby points already in the paired set are still matched, and new training samples are learned. At each update of our method, the control $u^*$ is projected onto the kernel of the end-point map** generated by the controlled dynamics at the learned samples. It ensures kee** the end-points for the previously learned samples constant while iteratively learning additional samples.
△ Less
Submitted 19 May, 2024; v1 submitted 16 April, 2024;
originally announced April 2024.
-
Age of $(k,n)$-Threshold Signature Scheme on a Gossip Network
Authors:
Erkan Bayram,
Melih Bastopcu,
Mohamed-Ali Belabbas,
Tamer Başar
Abstract:
We consider information update systems on a gossip network, which consists of a single source and $n$ receiver nodes. The source encrypts the information into $n$ distinct keys with version stamps, sending a unique key to each node. For decryption in a $(k, n)$-Threshold Signature Scheme, each receiver node requires at least $k+1$ different keys with the same version, shared over peer-to-peer conn…
▽ More
We consider information update systems on a gossip network, which consists of a single source and $n$ receiver nodes. The source encrypts the information into $n$ distinct keys with version stamps, sending a unique key to each node. For decryption in a $(k, n)$-Threshold Signature Scheme, each receiver node requires at least $k+1$ different keys with the same version, shared over peer-to-peer connections. We consider two different schemes: a memory scheme (in which the nodes keep the source's current and previous encrypted messages) and a memoryless scheme (in which the nodes are allowed to only keep the source's current message). We measure the ''timeliness'' of information updates by using the version age of information. Our work focuses on determining closed-form expressions for the time average age of information in a heterogeneous random graph. Our work not only allows to verify the expected outcome that a memory scheme results in a lower average age compared to a memoryless scheme, but also provides the quantitative difference between the two. In our numerical results, we quantify the value of memory and demonstrate that the advantages of memory diminish with infrequent source updates, frequent gossip** between nodes, or a decrease in $k$ for a fixed number of nodes.
△ Less
Submitted 18 February, 2024;
originally announced February 2024.
-
Vector-Valued Gossip over $w$-Holonomic Networks
Authors:
Erkan Bayram,
Mohamed-Ali Belabbas,
Tamer Başar
Abstract:
We study the weighted average consensus problem for a gossip network of agents with vector-valued states. For a given matrix-weighted graph, the gossip process is described by a sequence of pairs of adjacent agents communicating and updating their states based on the edge matrix weight. Our key contribution is providing conditions for the convergence of this non-homogeneous Markov process as well…
▽ More
We study the weighted average consensus problem for a gossip network of agents with vector-valued states. For a given matrix-weighted graph, the gossip process is described by a sequence of pairs of adjacent agents communicating and updating their states based on the edge matrix weight. Our key contribution is providing conditions for the convergence of this non-homogeneous Markov process as well as the characterization of its limit set. To this end, we introduce the notion of "$w$-holonomy" of a set of stochastic matrices, which enables the characterization of sequences of gossi** pairs resulting in reaching a desired consensus in a decentralized manner. Stated otherwise, our result characterizes the limiting behavior of infinite products of (non-commuting, possibly with absorbing states) stochastic matrices.
△ Less
Submitted 7 November, 2023;
originally announced November 2023.
-
Propagation on Multi-relational Graphs for Node Regression
Authors:
Eda Bayram
Abstract:
Recent years have witnessed a rise in real-world data captured with rich structural information that can be conveniently depicted by multi-relational graphs. While inference of continuous node features across a simple graph is rather under-studied by the current relational learning research, we go one step further and focus on node regression problem on multi-relational graphs. We take inspiration…
▽ More
Recent years have witnessed a rise in real-world data captured with rich structural information that can be conveniently depicted by multi-relational graphs. While inference of continuous node features across a simple graph is rather under-studied by the current relational learning research, we go one step further and focus on node regression problem on multi-relational graphs. We take inspiration from the well-known label propagation algorithm aiming at completing categorical features across a simple graph and propose a novel propagation framework for completing missing continuous features at the nodes of a multi-relational and directed graph. Our multi-relational propagation algorithm is composed of iterative neighborhood aggregations which originate from a relational local generative model. Our findings show the benefit of exploiting the multi-relational structure of the data in several node regression scenarios in different settings.
△ Less
Submitted 15 October, 2021;
originally announced October 2021.
-
Node Attribute Completion in Knowledge Graphs with Multi-Relational Propagation
Authors:
Eda Bayram,
Alberto Garcia-Duran,
Robert West
Abstract:
The existing literature on knowledge graph completion mostly focuses on the link prediction task. However, knowledge graphs have an additional incompleteness problem: their nodes possess numerical attributes, whose values are often missing. Our approach, denoted as MrAP, imputes the values of missing attributes by propagating information across the multi-relational structure of a knowledge graph.…
▽ More
The existing literature on knowledge graph completion mostly focuses on the link prediction task. However, knowledge graphs have an additional incompleteness problem: their nodes possess numerical attributes, whose values are often missing. Our approach, denoted as MrAP, imputes the values of missing attributes by propagating information across the multi-relational structure of a knowledge graph. It employs regression functions for predicting one node attribute from another depending on the relationship between the nodes and the type of the attributes. The propagation mechanism operates iteratively in a message passing scheme that collects the predictions at every iteration and updates the value of the node attributes. Experiments over two benchmark datasets show the effectiveness of our approach.
△ Less
Submitted 10 November, 2020;
originally announced November 2020.
-
Mask Combination of Multi-layer Graphs for Global Structure Inference
Authors:
Eda Bayram,
Dorina Thanou,
Elif Vural,
Pascal Frossard
Abstract:
Structure inference is an important task for network data processing and analysis in data science. In recent years, quite a few approaches have been developed to learn the graph structure underlying a set of observations captured in a data space. Although real-world data is often acquired in settings where relationships are influenced by a priori known rules, such domain knowledge is still not wel…
▽ More
Structure inference is an important task for network data processing and analysis in data science. In recent years, quite a few approaches have been developed to learn the graph structure underlying a set of observations captured in a data space. Although real-world data is often acquired in settings where relationships are influenced by a priori known rules, such domain knowledge is still not well exploited in structure inference problems. In this paper, we identify the structure of signals defined in a data space whose inner relationships are encoded by multi-layer graphs. We aim at properly exploiting the information originating from each layer to infer the global structure underlying the signals. We thus present a novel method for combining the multiple graphs into a global graph using mask matrices, which are estimated through an optimization problem that accommodates the multi-layer graph information and a signal representation model. The proposed mask combination method also estimates the contribution of each graph layer in the structure of signals. The experiments conducted both on synthetic and real-world data suggest that integrating the multi-layer graph representation of the data in the structure inference framework enhances the learning procedure considerably by adapting to the quality and the quantity of the input data.
△ Less
Submitted 15 April, 2020; v1 submitted 22 October, 2019;
originally announced October 2019.
