Shaping the learning landscape in neural networks around wide flat minima

DOI10.1073/pnas.1908636117zbMath1456.92009arXiv1905.07833OpenAlexW2998438489WikidataQ92217545 ScholiaQ92217545MaRDI QIDQ5854796

Riccardo Zecchina, Carlo Baldassi, Fabrizio Pittorino

Publication date: 12 March 2021

Published in: Proceedings of the National Academy of Sciences (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1905.07833

Mathematics Subject Classification ID

Applications of mathematical programming (90C90) Nonconvex programming, global optimization (90C26) Learning and adaptive systems in artificial intelligence (68T05) Neural networks for/in biological studies, artificial life and related topics (92B20) Statistical mechanics of random media, disordered materials (including liquid crystals and spin glasses) (82D30)

Related Items (8)

On Gaussian spin glass with P-wise interactions ⋮ Deep networks on toroids: removing symmetries reveals the structure of flat regions in the landscape geometry* ⋮ The inverse variance–flatness relation in stochastic gradient descent is critical for finding flat minima ⋮ Replica symmetry breaking in dense Hebbian neural networks ⋮ A spin glass model for the loss surfaces of generative adversarial networks ⋮ Clustering of solutions in the symmetric binary perceptron ⋮ Wide flat minima and optimal generalization in classifying high-dimensional Gaussian mixtures ⋮ Entropic gradient descent algorithms and wide flat minima*

Uses Software

• Fashion-MNIST

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Generalization learning in a perceptron with binary synapses
• Bounds on the learning capacity of some multi-layer networks
• Local entropy as a measure for sampling solutions in constraint satisfaction problems
• Weight Distribution of Low-Density Parity-Check Codes
• Observation of a hexatic vortex glass in flux lattices of the high-<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mrow><mml:msub><mml:mrow><mml:mi mathvariant="italic">T</mml:mi></mml:mrow><mml:mrow><mml:mi mathvariant="italic">c</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:math>superconductor<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mrow><mml:msub><mml:mrow><mml:mi mathvariant="normal">Bi</mml:mi></mml:mrow><mml:mrow><mml:…
• Capacity lower bound for the Ising perceptron

This page was built for publication: Shaping the learning landscape in neural networks around wide flat minima