Linearized two-layers neural networks in high dimension

DOI10.1214/20-AOS1990zbMath1473.62134arXiv1904.12191MaRDI QIDQ2039801

Yanyan Li

Publication date: 5 July 2021

Published in: The Annals of Statistics (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1904.12191

zbMATH Keywords

kernel ridge regressionrandom featuresapproximation boundsneural tangent class

Mathematics Subject Classification ID

Computational methods for problems pertaining to statistics (62-08) Nonparametric regression and quantile regression (62G08) Ridge regression; shrinkage estimators (Lasso) (62J07) Neural nets and related approaches to inference from stochastic processes (62M45)

Related Items (11)

Deep learning: a statistical viewpoint ⋮ Neural network approximation ⋮ Surprises in high-dimensional ridgeless least squares interpolation ⋮ Generalization error of random feature and kernel methods: hypercontractivity and kernel matrix concentration ⋮ Overparameterization and Generalization Error: Weighted Trigonometric Interpolation ⋮ Training Neural Networks as Learning Data-adaptive Kernels: Provable Representation and Approximation Benefits ⋮ Weighted neural tangent kernel: a generalized and improved network-induced kernel ⋮ Landscape and training regimes in deep learning ⋮ Unnamed Item ⋮ Unnamed Item ⋮ The interpolation phase transition in neural networks: memorization and generalization under lazy training

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• On information plus noise kernel random matrices
• The spectrum of kernel random matrices
• Projection-based approximation and a duality with kernel methods
• On best approximation by ridge functions
• Optimal nonlinear approximation
• A distribution-free theory of nonparametric regression
• Bayesian learning for neural networks
• On the near optimality of the stochastic approximation of smooth functions by neural networks
• The landscape of empirical risk for nonconvex losses
• Gradient descent optimizes over-parameterized deep ReLU networks
• Just interpolate: kernel ``ridgeless regression can generalize
• Mean field analysis of neural networks: a central limit theorem
• Optimal rates for the regularized least-squares algorithm
• Approximation by Ridge Functions and Neural Networks
• Universal approximation bounds for superpositions of a sigmoidal function
• Dimension-independent bounds on the degree of approximation by neural networks
• Neural Network Learning
• A mean field view of the landscape of two-layer neural networks
• Reconciling modern machine-learning practice and the classical bias–variance trade-off
• Breaking the Curse of Dimensionality with Convex Neural Networks
• On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions
• Spherical Harmonics in p Dimensions
• Disentangling feature and lazy training in deep neural networks
• Wide neural networks of any depth evolve as linear models under gradient descent ^*
• Introduction to nonparametric estimation
• Approximation by superpositions of a sigmoidal function

This page was built for publication: Linearized two-layers neural networks in high dimension