The committee machine: computational to statistical gaps in learning a two-layers neural network

DOI10.1088/1742-5468/ab43d2zbMath1459.82248arXiv1806.05451OpenAlexW3100762814MaRDI QIDQ5854128

Lenka Zdeborová, Benjamin Aubin, Jean Barbier, Antoine Maillard, Florent Krzakala, Nicolas Macris

Publication date: 16 March 2021

Published in: Journal of Statistical Mechanics: Theory and Experiment (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1806.05451

Mathematics Subject Classification ID

Neural nets applied to problems in time-dependent statistical mechanics (82C32)

Related Items (7)

Analyticity of the energy in an Ising spin glass with correlated disorder ⋮ Strong replica symmetry in high-dimensional optimal Bayesian inference ⋮ The adaptive interpolation method: a simple scheme to prove replica formulas in Bayesian inference ⋮ TheCommitteeMachine ⋮ Hidden unit specialization in layered neural networks: ReLU vs. sigmoidal activation ⋮ Large scale analysis of generalization error in learning using margin based classification methods ⋮ A Unifying Tutorial on Approximate Message Passing

Uses Software

• GitHub
• Entropy-SGD

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Finding hidden cliques of size \(\sqrt{N/e}\) in nearly linear time
• Notes on computational-to-statistical gaps: predictions using statistical physics
• Broken replica symmetry bounds in the mean field spin glass model
• The adaptive interpolation method: a simple scheme to prove replica formulas in Bayesian inference
• Universality in polytope phase transitions and message passing algorithms
• Statistical Mechanics of Learning
• Accurate Prediction of Phase Transitions in Compressed Sensing via a Connection to Minimax Denoising
• Graphical Models, Exponential Families, and Variational Inference
• Information, Physics, and Computation
• Generalization properties of multilayered neural networks
• Learning a rule in a multilayer neural network
• Performance Limits for Noisy Multimeasurement Vector Problems
• 10.1162/153244303321897690
• Decoding from Pooled Data: Sharp Information-Theoretic Bounds
• The adaptive interpolation method for proving replica formulas. Applications to the Curie–Weiss and Wigner spike models
• State evolution for general approximate message passing algorithms, with applications to spatial coupling
• Optimal errors and phase transitions in high-dimensional generalized linear models
• The Dynamics of Message Passing on Dense Graphs, with Applications to Compressed Sensing
• Approximate Message-Passing Decoder and Capacity Achieving Sparse Superposition Codes

This page was built for publication: The committee machine: computational to statistical gaps in learning a two-layers neural network