Cross-validation Confidence Intervals for Test Error

arXiv2007.12671MaRDI QIDQ6345790

Author name not available (Why is that?)

Publication date: 24 July 2020

Abstract: This work develops central limit theorems for cross-validation and consistent estimators of its asymptotic variance under weak stability conditions on the learning algorithm. Together, these results provide practical, asymptotically-exact confidence intervals for $k$-fold test error and valid, powerful hypothesis tests of whether one learning algorithm has smaller $k$-fold test error than another. These results are also the first of their kind for the popular choice of leave-one-out cross-validation. In our real-data experiments with diverse learning algorithms, the resulting intervals and tests outperform the most popular alternative methods from the literature.

Has companion code repository: https://github.com/alexandre-bayle/cvci

This page was built for publication: Cross-validation Confidence Intervals for Test Error

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6345790)