Comparing dynamics: deep neural networks versus glassy systems

DOI10.1088/1742-5468/ab3281zbMath1459.82317arXiv1803.06969OpenAlexW3103509423MaRDI QIDQ5854115

Levent Sagun, Matthieu Wyart, Marco Baity-Jesi, Chiara Cammarota, Mario Geiger, Stefano Spigler, Yann LeCun, Gérard Ben Arous, Giulio Biroli

Publication date: 16 March 2021

Published in: Journal of Statistical Mechanics: Theory and Experiment (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1803.06969

Mathematics Subject Classification ID

Statistical mechanics of random media, disordered materials (including liquid crystals and spin glasses) (82D30) Neural nets applied to problems in time-dependent statistical mechanics (82C32) Dynamics of disordered systems (random Ising systems, etc.) in time-dependent statistical mechanics (82C44)

Related Items (15)

Gradient descent dynamics and the jamming transition in infinite dimensions ⋮ Archetypal landscapes for deep neural networks ⋮ Ensemble Inference Methods for Models With Noisy and Expensive Likelihoods ⋮ Black holes and the loss landscape in machine learning ⋮ An introduction to machine learning: a perspective from statistical physics ⋮ Landscape and training regimes in deep learning ⋮ Unnamed Item ⋮ An emergent autonomous flow for mean-field spin glasses ⋮ The loss surfaces of neural networks with general activation functions ⋮ A spin glass model for the loss surfaces of generative adversarial networks ⋮ Maximal relevance and optimal learning machines ⋮ Scaling description of generalization with number of parameters in deep learning ⋮ Mean-field dynamics of infinite-dimensional particle systems: global shear versus random local forcing ⋮ An analytic theory of shallow networks dynamics for hinge loss classification* ⋮ Universal characteristics of deep neural network loss surfaces from random matrix theory

Uses Software

• Entropy-SGD

Cites Work

• Unnamed Item
• Rigorous inequalities between length and time scales in glassy systems
• Spectral gap estimates in mean field spin glasses
• Cugliandolo-Kurchan equations for dynamics of spin-glasses
• Phase space geometry and slow dynamics
• Activated aging dynamics and effective trap model description in the random energy model
• Spin-glass theory for pedestrians
• Determining computational complexity from characteristic ‘phase transitions’
• Gibbs states and the set of solutions of random constraint satisfaction problems

This page was built for publication: Comparing dynamics: deep neural networks versus glassy systems