-
Practical implementation of geometric quasi-cyclic LDPC codes
Authors:
Simeon Ball,
Tomàs Ortega
Abstract:
We detail for the first time a complete explicit description of the quasi-cyclic structure of all classical finite generalized quadrangles. Using these descriptions we construct families of quasi-cyclic LDPC codes derived from the point-line incidence matrix of the quadrangles by explicitly calculating quasi-cyclic generator and parity check matrices for these codes. This allows us to construct pa…
▽ More
We detail for the first time a complete explicit description of the quasi-cyclic structure of all classical finite generalized quadrangles. Using these descriptions we construct families of quasi-cyclic LDPC codes derived from the point-line incidence matrix of the quadrangles by explicitly calculating quasi-cyclic generator and parity check matrices for these codes. This allows us to construct parity check and generator matrices of all such codes of length up to 400000. These codes cover a wide range of transmission rates, are easy and fast to implement and perform close to Shannon's limit with no visible error floors. We also include some performance data for these codes. Furthermore, we include a complete explicit description of the quasi-cyclic structure of the point-line and point-hyperplane incidences of the finite projective and affine spaces.
△ Less
Submitted 30 May, 2024;
originally announced May 2024.
-
Decentralized Optimization in Time-Varying Networks with Arbitrary Delays
Authors:
Tomas Ortega,
Hamid Jafarkhani
Abstract:
We consider a decentralized optimization problem for networks affected by communication delays. Examples of such networks include collaborative machine learning, sensor networks, and multi-agent systems. To mimic communication delays, we add virtual non-computing nodes to the network, resulting in directed graphs. This motivates investigating decentralized optimization solutions on directed graphs…
▽ More
We consider a decentralized optimization problem for networks affected by communication delays. Examples of such networks include collaborative machine learning, sensor networks, and multi-agent systems. To mimic communication delays, we add virtual non-computing nodes to the network, resulting in directed graphs. This motivates investigating decentralized optimization solutions on directed graphs. Existing solutions assume nodes know their out-degrees, resulting in limited applicability. To overcome this limitation, we introduce a novel gossip-based algorithm, called DT-GO, that does not need to know the out-degrees. The algorithm is applicable in general directed networks, for example networks with delays or limited acknowledgment capabilities. We derive convergence rates for both convex and non-convex objectives, showing that our algorithm achieves the same complexity order as centralized Stochastic Gradient Descent. In other words, the effects of the graph topology and delays are confined to higher-order terms. Additionally, we extend our analysis to accommodate time-varying network topologies. Numerical simulations are provided to support our theoretical findings.
△ Less
Submitted 29 May, 2024;
originally announced May 2024.
-
Decentralized Optimization in Networks with Arbitrary Delays
Authors:
Tomas Ortega,
Hamid Jafarkhani
Abstract:
We consider the problem of decentralized optimization in networks with communication delays. To accommodate delays, we need decentralized optimization algorithms that work on directed graphs. Existing approaches require nodes to know their out-degree to achieve convergence. We propose a novel gossip-based algorithm that circumvents this requirement, allowing decentralized optimization in networks…
▽ More
We consider the problem of decentralized optimization in networks with communication delays. To accommodate delays, we need decentralized optimization algorithms that work on directed graphs. Existing approaches require nodes to know their out-degree to achieve convergence. We propose a novel gossip-based algorithm that circumvents this requirement, allowing decentralized optimization in networks with communication delays. We prove that our algorithm converges on non-convex objectives, with the same main complexity order term as centralized Stochastic Gradient Descent (SGD), and show that the graph topology and the delays only affect the higher order terms. We provide numerical simulations that illustrate our theoretical results.
△ Less
Submitted 20 January, 2024;
originally announced January 2024.
-
Asynchronous Federated Learning with Bidirectional Quantized Communications and Buffered Aggregation
Authors:
Tomas Ortega,
Hamid Jafarkhani
Abstract:
Asynchronous Federated Learning with Buffered Aggregation (FedBuff) is a state-of-the-art algorithm known for its efficiency and high scalability. However, it has a high communication cost, which has not been examined with quantized communications. To tackle this problem, we present a new algorithm (QAFeL), with a quantization scheme that establishes a shared "hidden" state between the server and…
▽ More
Asynchronous Federated Learning with Buffered Aggregation (FedBuff) is a state-of-the-art algorithm known for its efficiency and high scalability. However, it has a high communication cost, which has not been examined with quantized communications. To tackle this problem, we present a new algorithm (QAFeL), with a quantization scheme that establishes a shared "hidden" state between the server and clients to avoid the error propagation caused by direct quantization. This approach allows for high precision while significantly reducing the data transmitted during client-server interactions. We provide theoretical convergence guarantees for QAFeL and corroborate our analysis with experiments on a standard benchmark.
△ Less
Submitted 31 July, 2023;
originally announced August 2023.
-
Gossiped and Quantized Online Multi-Kernel Learning
Authors:
Tomas Ortega,
Hamid Jafarkhani
Abstract:
In instances of online kernel learning where little prior information is available and centralized learning is unfeasible, past research has shown that distributed and online multi-kernel learning provides sub-linear regret as long as every pair of nodes in the network can communicate (i.e., the communications network is a complete graph). In addition, to manage the communication load, which is of…
▽ More
In instances of online kernel learning where little prior information is available and centralized learning is unfeasible, past research has shown that distributed and online multi-kernel learning provides sub-linear regret as long as every pair of nodes in the network can communicate (i.e., the communications network is a complete graph). In addition, to manage the communication load, which is often a performance bottleneck, communications between nodes can be quantized. This letter expands on these results to non-fully connected graphs, which is often the case in wireless sensor networks. To address this challenge, we propose a gossip algorithm and provide a proof that it achieves sub-linear regret. Experiments with real datasets confirm our findings.
△ Less
Submitted 28 April, 2023; v1 submitted 24 January, 2023;
originally announced January 2023.
