From Fixed-X to Random-X Regression: Bias-Variance Decompositions, Covariance Penalties, and Prediction Error Estimation

DOI10.1080/01621459.2018.1424632zbMath1437.62351arXiv1704.08160OpenAlexW2625919377MaRDI QIDQ3304841

Saharon Rosset, Ryan J. Tibshirani

Publication date: 3 August 2020

Published in: Journal of the American Statistical Association (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1704.08160

zbMATH Keywords

bias-variance decompositioncovariance penaltiesrandom-X regression

Mathematics Subject Classification ID

Inference from stochastic processes and prediction (62M20) Ridge regression; shrinkage estimators (Lasso) (62J07) Learning and adaptive systems in artificial intelligence (68T05) Analysis of variance and covariance (ANOVA) (62J10)

Related Items (13)

From Fixed-X to Random-X Regression: Bias-Variance Decompositions, Covariance Penalties, and Prediction Error Estimation: Correction ⋮ Semi-supervised empirical risk minimization: using unlabeled data to improve prediction ⋮ A Relabeling Approach to Handling the Class Imbalance Problem for Logistic Regression ⋮ Active Operator Inference for Learning Low-Dimensional Dynamical-System Models from Noisy Data ⋮ Empirical Bayes Mean Estimation With Nonparametric Errors Via Order Statistic Regression on Replicated Data ⋮ Inadmissibility of the corrected Akaike information criterion ⋮ Correcting the corrected AIC ⋮ Prediction of the Nash through penalized mixture of logistic regression models ⋮ Inference for a two-stage enrichment design ⋮ Homeostasis phenomenon in conformal prediction and predictive distribution functions ⋮ Optimal selection of sample-size dependent common subsets of covariates for multi-task regression prediction ⋮ Average effects based on regressions with a logarithmic link function: a new approach with stochastic covariates ⋮ Comment on: ``Models as approximations

Uses Software

• ElemStatLearn

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Optimal equivariant prediction for high-dimensional linear models with arbitrary predictor covariance
• Random design analysis of ridge regression
• A survey of cross-validation procedures for model selection
• Evaluation and selection of models for out-of-sample prediction when the sample size is small relative to the complexity of the data-generating process
• Concentration of measure and spectra of random matrices: applications to correlation matrices, elliptical distributions and beyond
• Estimation of the mean of a multivariate normal distribution
• Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation
• High-dimensional asymptotics of prediction: ridge regression and classification
• Persistene in high-dimensional linear predictor-selection and the virtue of overparametrization
• A distribution-free theory of nonparametric regression
• On the ``degrees of freedom of the lasso
• How Many Variables Should be Entered in a Regression Equation?
• How Biased is the Apparent Error Rate of a Prediction Rule?
• A comparative study of ordinary cross-validation, v-fold cross-validation and the repeated learning-testing methods
• Selection of Variables in Multiple Regression: Part II. Chosen Procedures, Computations and Examples
• Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter
• A Biometrics Invited Paper. The Analysis and Selection of Variables in Linear Regression
• Selection of Variables in Multiple Regression: Part I. A Review and Evaluation
• Some Comments on C P
• The Estimation of Prediction Error

This page was built for publication: From Fixed-X to Random-X Regression: Bias-Variance Decompositions, Covariance Penalties, and Prediction Error Estimation