Truth or Backpropaganda? An Empirical Investigation of Deep Learning Theory
From MaRDI portal
Publication:6326317
arXiv1910.00359MaRDI QIDQ6326317
Author name not available (Why is that?)
Publication date: 1 October 2019
Abstract: We empirically evaluate common assumptions about neural networks that are widely held by practitioners and theorists alike. In this work, we: (1) prove the widespread existence of suboptimal local minima in the loss landscape of neural networks, and we use our theory to find examples; (2) show that small-norm parameters are not optimal for generalization; (3) demonstrate that ResNets do not conform to wide-network theories, such as the neural tangent kernel, and that the interaction between skip connections and batch normalization plays a role; (4) find that rank does not correlate with generalization or robustness in a practical setting.
Has companion code repository: https://github.com/goldblum/TruthOrBackpropaganda
This page was built for publication: Truth or Backpropaganda? An Empirical Investigation of Deep Learning Theory
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6326317)
