Guaranteed Validity for Empirical Approaches to Adaptive Data Analysis
From MaRDI portal
Publication:6320862
arXiv1906.09231MaRDI QIDQ6320862
Author name not available (Why is that?)
Publication date: 21 June 2019
Abstract: We design a general framework for answering adaptive statistical queries that focuses on providing explicit confidence intervals along with point estimates. Prior work in this area has either focused on providing tight confidence intervals for specific analyses, or providing general worst-case bounds for point estimates. Unfortunately, as we observe, these worst-case bounds are loose in many settings --- often not even beating simple baselines like sample splitting. Our main contribution is to design a framework for providing valid, instance-specific confidence intervals for point estimates that can be generated by heuristics. When paired with good heuristics, this method gives guarantees that are orders of magnitude better than the best worst-case bounds. We provide a Python library implementing our method.
Has companion code repository: https://github.com/omthkkr/empirical_adaptive_data_analysis
This page was built for publication: Guaranteed Validity for Empirical Approaches to Adaptive Data Analysis
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6320862)
