Gradient descent for deep matrix factorization: dynamics and implicit bias towards low rank

DOI10.1016/j.acha.2023.101595arXiv2011.13772OpenAlexW4386484876MaRDI QIDQ6185686

Holger Rauhut, Carsten Gieshoff, Johannes Maly, Hung-Hsu Chou

Publication date: 30 January 2024

Published in: Applied and Computational Harmonic Analysis (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2011.13772

zbMATH Keywords

neural networksmatrix factorizationgradient descentimplicit bias/regularization

Mathematics Subject Classification ID

Computer science (68-XX) Game theory, economics, finance, and other social and behavioral sciences (91-XX)

Cites Work

• On early stopping in gradient descent learning
• Stable low-rank matrix recovery via null space properties
• Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizers
• Gradient Descent with Identity Initialization Efficiently Learns Positive-Definite Linear Transformations by Deep Residual Networks

This page was built for publication: Gradient descent for deep matrix factorization: dynamics and implicit bias towards low rank