Showing 1–1 of 1 results for author: Neto, A d S D

Communication-Efficient Byzantine-Resilient Federated Zero-Order Optimization

Authors: Afonso de Sá Delgado Neto, Maximilian Egger, Mayank Bakshi, Rawad Bitar

Abstract: We introduce CYBER-0, the first zero-order optimization algorithm for memory-and-communication efficient Federated Learning, resilient to Byzantine faults. We show through extensive numerical experiments on the MNIST dataset and finetuning RoBERTa-Large that CYBER-0 outperforms state-of-the-art algorithms in terms of communication and memory efficiency while reaching similar accuracy. We provide t… ▽ More We introduce CYBER-0, the first zero-order optimization algorithm for memory-and-communication efficient Federated Learning, resilient to Byzantine faults. We show through extensive numerical experiments on the MNIST dataset and finetuning RoBERTa-Large that CYBER-0 outperforms state-of-the-art algorithms in terms of communication and memory efficiency while reaching similar accuracy. We provide theoretical guarantees on its convergence for convex loss functions. △ Less

Submitted 20 June, 2024; originally announced June 2024.