Generalisation error in learning with random features and the hidden manifold model*

DOI10.1088/1742-5468/ac3ae6OpenAlexW3008127699MaRDI QIDQ5020057

Bruno Loureiro, Federica Gerace, Lenka Zdeborová, Marc Mézard, Florent Krzakala

Publication date: 3 January 2022

Published in: Journal of Statistical Mechanics: Theory and Experiment (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2002.09339

zbMATH Keywords

learning theorymachine learningdeep learningcavity and replica method

Mathematics Subject Classification ID

Statistical mechanics, structure of matter (82-XX)

Related Items (max. 100)

Surprises in high-dimensional ridgeless least squares interpolation ⋮ The inverse variance–flatness relation in stochastic gradient descent is critical for finding flat minima ⋮ Optimal regularizations for data generation with probabilistic graphical models ⋮ Free dynamics of feature learning processes ⋮ Universality of regularized regression estimators in high dimensions

Uses Software

• Scikit
• FALCONN

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• A random matrix approach to neural networks
• The spectral norm of random inner-product kernel matrices
• The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression
• Optimal rates for the regularized least-squares algorithm
• Deterministic equivalents for certain functionals of large random matrices
• The Parisi formula
• Statistical Mechanics of Learning
• Divide and Conquer Kernel Ridge Regression: A Distributed Algorithm with Minimax Optimal Rates
• THE SPECTRUM OF RANDOM INNER-PRODUCT KERNEL MATRICES
• Statistical mechanics of complex neural systems and high dimensional data
• 10.1162/153244303321897690
• Reconciling modern machine-learning practice and the classical bias–variance trade-off
• Optimal errors and phase transitions in high-dimensional generalized linear models
• Understanding Machine Learning
• Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition
• A jamming transition from under- to over-parametrization affects generalization in deep learning

This page was built for publication: Generalisation error in learning with random features and the hidden manifold model*