Universal statistics of Fisher information in deep neural networks: mean field approach*

DOI10.1088/1742-5468/ABC62EOpenAlexW3116019732MaRDI QIDQ5857452

Ryo Karakida, Shotaro Akaho, Shun-ichi Amari

Publication date: 1 April 2021

Published in: Journal of Statistical Mechanics: Theory and Experiment (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1806.01316

zbMATH Keywords

deep learning

Mathematics Subject Classification ID

Statistical mechanics, structure of matter (82-XX)

Related Items (3)

Geometry on degradation models and mis-specification analysis by using \(\alpha\)-divergence ⋮ Impact of barren plateaus countermeasures on the quantum neural network capacity to learn ⋮ Asymptotic freeness of layerwise Jacobians caused by invariance of multilayer perceptron: the Haar orthogonal case

Cites Work

• Unnamed Item
• Unnamed Item
• Information geometry and its applications
• The landscape of empirical risk for nonconvex losses
• Flat Minima
• Riemannian metrics for neural networks I: feedforward networks
• Overcoming catastrophic forgetting in neural networks

This page was built for publication: Universal statistics of Fisher information in deep neural networks: mean field approach^*