A spin glass model for the loss surfaces of generative adversarial networks

DOI10.1007/s10955-022-02875-wOpenAlexW3118365541MaRDI QIDQ2076047

Francesco Mezzadri, Joseph Najnudel, Nicholas P. Baskerville, Jonathan P. Keating

Publication date: 18 February 2022

Published in: Journal of Statistical Physics (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2101.02524

zbMATH Keywords

neural networksspin glassesrandom matrix theorydeep learninggenerative adversarial networks

Mathematics Subject Classification ID

Time-dependent statistical mechanics (dynamic and nonequilibrium) (82Cxx) Special matrices (15Bxx) Applications of statistical mechanics to specific types of physical systems (82Dxx)

Related Items

Landscape complexity beyond invariance and the elastic manifold ⋮ Local convexity of the TAP free energy and AMP convergence for \(\mathbb{Z}_2\)-synchronization ⋮ Exponential growth of random determinants beyond invariance ⋮ Universal characteristics of deep neural network loss surfaces from random matrix theory

Uses Software

• CIFAR
• GitHub
• CycleGAN
• pix2pix
• BERT
• StyleGAN
• Wasserstein GAN
• Entropy-SGD

Cites Work

• Unnamed Item
• Concentration of the spectral measure for large matrices
• The complexity of spherical \(p\)-spin models: a second moment approach
• Replica symmetry breaking condition exposed by random matrix calculation of landscape complexity
• Statistical Mechanics of Learning
• Complexity of Random Energy Landscapes, Glass Transition, and Absolute Value of the Spectral Determinant of Random Matrices
• Flat Minima
• Dyson’s correlation functions and graded symmetry
• The loss surfaces of neural networks with general activation functions
• Statistical Physics of Spin Glasses and Information Processing
• Random Matrices and Complexity of Spin Glasses
• Geometry and Temperature Chaos in Mixed Spherical Spin Glasses at Low Temperature: The Perturbative Regime
• The Landscape of the Spiked Tensor Model
• Random Fields and Geometry
• The space of interactions in neural network models
• Nonlinear random matrix theory for deep learning
• Comparing dynamics: deep neural networks versus glassy systems
• Entropy-SGD: biasing gradient descent into wide valleys
• Shaping the learning landscape in neural networks around wide flat minima
• Aging of spherical spin glasses

This page was built for publication: A spin glass model for the loss surfaces of generative adversarial networks