High-dimensional asymptotics of prediction: ridge regression and classification

DOI10.1214/17-AOS1549zbMath1428.62307arXiv1507.03003MaRDI QIDQ1747738

Stefan Wager, Edgar Dobriban

Publication date: 27 April 2018

Published in: The Annals of Statistics (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1507.03003

zbMATH Keywords

prediction errorridge regressionrandom matrix theoryhigh-dimensional asymptoticsregularized discriminant analysis

Mathematics Subject Classification ID

Multivariate analysis (62H99) Ridge regression; shrinkage estimators (Lasso) (62J07) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Linear regression; mixed models (62J05)

Related Items (35)

On Genetic Correlation Estimation With Summary Statistics From Genome-Wide Association Studies ⋮ Surprises in high-dimensional ridgeless least squares interpolation ⋮ Limiting Spectral Distribution for Large Sample Covariance Matrices with Graph-Dependent Elements ⋮ WONDER: Weighted one-shot distributed ridge regression in high dimensions ⋮ Learning curves of generic features maps for realistic datasets with a teacher-student model* ⋮ Generalization error rates in kernel regression: the crossover from the noiseless to noisy regime* ⋮ Ridge regression revisited: debiasing, thresholding and bootstrap ⋮ Fast Cross-validation for Multi-penalty High-dimensional Ridge Regression ⋮ Unnamed Item ⋮ High-Dimensional Factor Regression for Heterogeneous Subpopulations ⋮ Rates of Bootstrap Approximation for Eigenvalues in High-Dimensional PCA ⋮ On polygenic risk scores for complex traits prediction ⋮ Cross-Trait Prediction Accuracy of Summary Statistics in Genome-Wide Association Studies ⋮ Free dynamics of feature learning processes ⋮ A Universal Trade-off Between the Model Size, Test Loss, and Training Loss of Linear Predictors ⋮ Smoothly varying regularization ⋮ Unnamed Item ⋮ High-Dimensional Analysis of Double Descent for Linear Regression with Random Projections ⋮ Universality of regularized regression estimators in high dimensions ⋮ Unnamed Item ⋮ Most powerful test against a sequence of high dimensional local alternatives ⋮ Unnamed Item ⋮ Which bridge estimator is the best for variable selection? ⋮ Permutation methods for factor analysis and PCA ⋮ From Fixed-X to Random-X Regression: Bias-Variance Decompositions, Covariance Penalties, and Prediction Error Estimation ⋮ High-dimensional asymptotics of prediction: ridge regression and classification ⋮ On the dimension effect of regularized linear discriminant analysis ⋮ Prediction risk for the horseshoe regression ⋮ High-dimensional linear models: a random matrix perspective ⋮ Unnamed Item ⋮ Exact minimax risk for linear least squares, and the lower tail of sample covariance matrices ⋮ Unnamed Item ⋮ Unnamed Item ⋮ Large dimensional analysis of general margin based classification methods ⋮ A random matrix analysis of random Fourier features: beyond the Gaussian kernel, a precise phase transition, and the corresponding double descent*

Uses Software

• Stanford Tagger

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Optimal equivariant prediction for high-dimensional linear models with arbitrary predictor covariance
• Random design analysis of ridge regression
• Eigenvectors of some large sample covariance matrix ensembles
• On the impact of predictor geometry on the performance on high-dimensional ridge-regularized generalized robust regression estimators
• Asymptotic expansion of the misclassification probabilities of D- and A- criteria for discrimination from two high dimensional populations using the theory of large dimensional random matrices
• Spectral analysis of large dimensional random matrices
• Results in statistical discriminant analysis: A review of the former Soviet Union literature.
• High-dimensional asymptotics of prediction: ridge regression and classification
• Some theory for Fisher's linear discriminant function, `naive Bayes', and some alternatives when there are many more variables than observations
• Strong convergence of the empirical distribution of eigenvalues of large dimensional random matrices
• Asymptotic expansion and estimation of EPMC for linear classification rules in high dimension
• Sharp detection in PCA under correlations: all eigenvalues matter
• Deterministic equivalents for certain functionals of large random matrices
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• Random Matrix Methods for Wireless Communications
• A Regularized Hotelling’sT²Test for Pathway Analysis in Proteomic Studies
• A New Approach for Mutual Information Analysis of Large Dimensional Multi-Antenna Channels
• The Efficiency of Logistic Regression Compared to Normal Discriminant Analysis
• Asymptotic aproximations for EPMC’s of the linear and the quadratic discriminant functions when the sample sizes and the dimension are large
• Analytic Study of Performance of Error Estimators for Linear Discriminant Analysis
• Finite-Sample Linear Filter Optimization in Wireless Communications and Financial Systems
• Generalized Consistent Error Estimator of Linear Discriminant Analysis
• 10.1162/153244303321897690
• Large Sample Covariance Matrices and High-Dimensional Data Analysis
• The LASSO Risk for Gaussian Matrices
• On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities
• Multivariate Statistics
• Random Matrix Theory and Wireless Communications
• Ridge regression and asymptotic minimax estimation over spheres of growing dimension

This page was built for publication: High-dimensional asymptotics of prediction: ridge regression and classification