Learning curves of generic features maps for realistic datasets with a teacher-student model*

DOI10.1088/1742-5468/ac9825OpenAlexW3171395320MaRDI QIDQ5055409

Sebastian Goldt, Hugo Cui, Florent Krzakala, Bruno Loureiro, Marc Mézard, Lenka Zdeborová, Cédric Gerbelot

Publication date: 13 December 2022

Published in: Journal of Statistical Mechanics: Theory and Experiment (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2102.08127

zbMATH Keywords

learning theorymachine learningdeep learning

Mathematics Subject Classification ID

Statistical mechanics, structure of matter (82-XX)

Related Items

Free dynamics of feature learning processes, Debiasing convex regularized estimators and interval estimation in linear models, An introduction to machine learning: a perspective from statistical physics, Universality of regularized regression estimators in high dimensions

Uses Software

• Adam
• Kymatio

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• High dimensional robust M-estimation: asymptotic variance via approximate message passing
• Cox's regression model for counting processes: A large sample study
• Eigenvectors of some large sample covariance matrix ensembles
• The spectrum of kernel random matrices
• Concentration of measure and spectra of random matrices: applications to correlation matrices, elliptical distributions and beyond
• Some inequalities for Gaussian processes and applications
• The spectral norm of random inner-product kernel matrices
• High-dimensional asymptotics of prediction: ridge regression and classification
• Surprises in high-dimensional ridgeless least squares interpolation
• The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression
• Optimal rates for the regularized least-squares algorithm
• On the convergence of the extremal eigenvalues of empirical covariance matrices with dependence
• Deterministic equivalents for certain functionals of large random matrices
• Statistical Mechanics of Learning
• THE SPECTRUM OF RANDOM INNER-PRODUCT KERNEL MATRICES
• On robust regression with high-dimensional predictors
• Information, Physics, and Computation
• An Introduction to Random Matrices
• Probability
• High-Dimensional Probability
• Precise Error Analysis of Regularized <inline-formula> <tex-math notation="LaTeX">$M$ </tex-math> </inline-formula>-Estimators in High Dimensions
• When do neural networks outperform kernel methods?*
• Two Models of Double Descent for Weak Features
• Benign overfitting in linear regression
• Reconciling modern machine-learning practice and the classical bias–variance trade-off
• Large scale analysis of generalization error in learning using margin based classification methods
• A jamming transition from under- to over-parametrization affects generalization in deep learning
• Convex analysis and monotone operator theory in Hilbert spaces
• Support vector machines learning noisy polynomial rules