Ridge Regression: Structure, Cross-Validation, and Sketching

arXiv1910.02373MaRDI QIDQ6326610

Edgar Dobriban, Sifan Liu

Publication date: 6 October 2019

Abstract: We study the following three fundamental problems about ridge regression: (1) what is the structure of the estimator? (2) how to correctly use cross-validation to choose the regularization parameter? and (3) how to accelerate computation without losing too much accuracy? We consider the three problems in a unified large-data linear model. We give a precise representation of ridge regression as a covariance matrix-dependent linear combination of the true parameter and the noise. We study the bias of $K$-fold cross-validation for choosing the regularization parameter, and propose a simple bias-correction. We analyze the accuracy of primal and dual sketching for ridge regression, showing they are surprisingly accurate. Our results are illustrated by simulations and by analyzing empirical data.

Has companion code repository: https://github.com/liusf15/Sketching-lr

This page was built for publication: Ridge Regression: Structure, Cross-Validation, and Sketching

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6326610)