Communication-Efficient Distributed SVD via Local Power Iterations

arXiv2002.08014MaRDI QIDQ6335115

Author name not available (Why is that?)

Publication date: 19 February 2020

Abstract: We study distributed computing of the truncated singular value decomposition problem. We develop an algorithm that we call exttt{LocalPower} for improving communication efficiency. Specifically, we uniformly partition the dataset among $m$ nodes and alternate between multiple (precisely $p$) local power iterations and one global aggregation. In the aggregation, we propose to weight each local eigenvector matrix with orthogonal Procrustes transformation (OPT). As a practical surrogate of OPT, sign-fixing, which uses a diagonal matrix with $pm1$ entries as weights, has better computation complexity and stability in experiments. We theoretically show that under certain assumptions exttt{LocalPower} lowers the required number of communications by a factor of $p$ to reach a constant accuracy. We also show that the strategy of periodically decaying $p$ helps obtain high-precision solutions. We conduct experiments to demonstrate the effectiveness of exttt{LocalPower}.

Has companion code repository: https://github.com/lx10077/LocalPower

This page was built for publication: Communication-Efficient Distributed SVD via Local Power Iterations

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6335115)