FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracy

arXiv2302.10429MaRDI QIDQ6427132

Author name not available (Why is that?)

Publication date: 20 February 2023

Abstract: Federated learning is an emerging distributed machine learning framework which jointly trains a global model via a large number of local devices with data privacy protections. Its performance suffers from the non-vanishing biases introduced by the local inconsistent optimal and the rugged client-drifts by the local over-fitting. In this paper, we propose a novel and practical method, FedSpeed, to alleviate the negative impacts posed by these problems. Concretely, FedSpeed applies the prox-correction term on the current local updates to efficiently reduce the biases introduced by the prox-term, a necessary regularizer to maintain the strong local consistency. Furthermore, FedSpeed merges the vanilla stochastic gradient with a perturbation computed from an extra gradient ascent step in the neighborhood, thereby alleviating the issue of local over-fitting. Our theoretical analysis indicates that the convergence rate is related to both the communication rounds $T$ and local intervals $K$ with a upper bound $smallmathcalO(1/T)$ if setting a proper local interval. Moreover, we conduct extensive experiments on the real-world dataset to demonstrate the efficiency of our proposed FedSpeed, which performs significantly faster and achieves the state-of-the-art (SOTA) performance on the general FL experimental settings than several baselines. Our code is available at url{https://github.com/woodenchild95/FL-Simulator.git}.

Has companion code repository: https://github.com/NAVER-INTEL-Co-Lab/gaudi-byzantine

This page was built for publication: FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracy

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6427132)