Entropic gradient descent algorithms and wide flat minima*

DOI10.1088/1742-5468/ac3ae8OpenAlexW4205795626MaRDI QIDQ5020063

Christoph Feinauer, Gabriele Perugini, Elizaveta Demyanenko, Riccardo Zecchina, Carlo Baldassi, Carlo Lucibello, Fabrizio Pittorino

Publication date: 3 January 2022

Published in: Journal of Statistical Mechanics: Theory and Experiment (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2006.07897

zbMATH Keywords

machine learningdeep learningmessage-passing algorithms

Mathematics Subject Classification ID

Statistical mechanics, structure of matter (82-XX)

Related Items

Deep networks on toroids: removing symmetries reveals the structure of flat regions in the landscape geometry*

Uses Software

• PyTorch
• AutoAugment
• EfficientNet
• Entropy-SGD

Cites Work

• Deep relaxation: partial differential equations for optimizing deep neural networks
• Flat Minima
• Local entropy as a measure for sampling solutions in constraint satisfaction problems
• Information, Physics, and Computation
• Entropy-SGD: biasing gradient descent into wide valleys
• Shaping the learning landscape in neural networks around wide flat minima
• Unnamed Item
• Unnamed Item