Approximate spectral decomposition of Fisher information matrix for simple ReLU networks

DOI10.1016/j.neunet.2023.05.017zbMath1546.68027MaRDI QIDQ6534979

Jun'ichi Takeuchi, Masazumi Iida, Yoshinari Takeishi

Publication date: 26 October 2023

Published in: Neural Networks (Search for Journal in Brave)

zbMATH Keywords

neural networksFisher informationmachine learning

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Numerical computation of eigenvalues and eigenvectors of matrices (65F15) Measures of information, entropy (94A17)

Cites Work

• Modeling by shortest data description
• Minimum complexity density estimation
• High-Dimensional Statistics
• Two Models of Double Descent for Weak Features
• Minimum Description Length Principle in Supervised Learning With Application to Lasso
• Any Target Function Exists in a Neighborhood of Any Sufficiently Wide Random Network: A Geometrical Perspective
• Reconciling modern machine-learning practice and the classical bias–variance trade-off
• Universal statistics of Fisher information in deep neural networks: mean field approach^*
• Unnamed Item
• Unnamed Item
• Unnamed Item

This page was built for publication: Approximate spectral decomposition of Fisher information matrix for simple ReLU networks