Degrees of freedom for piecewise Lipschitz estimators (Q1650119)

The multivariate Gaussian model \(Y\sim N(\mu,\sigma^2I)\) is considered on \(\mathbb{R}^n\) with \(\mu\) the unknown parameter. The degrees of freedom (DF) of an estimator \(\hat{\mu}\) is \(df(\hat{\mu}):=\sum_{i=1}^n \frac{\mathrm{cov}(Y_i,\hat{\mu}(Y)_i)}{\sigma^2}\). If \(\hat{\mu}\) is differentiable in Lebesgue almost all points, Steinś DF of \(\mu\) is \(df_S(\hat{\mu}):=E(\operatorname{div}(\hat{\mu})(Y)).\) If \(\hat{\mu}\) is \textit{almost differentiable} then \(df(\hat{\mu})= df_S(\hat{\mu})\) due to \textit{C. M. Stein} [Ann. Stat. 9, 1135--1151 (1981; Zbl 0476.62035)]. The main result of the paper under review provides a representation of \(df(\hat{\mu})-df_S(\hat{\mu})\) for a range of discontinuous estimators containing a number of regression estimators that include variable selection. For lasso -- Ordinary Least Squares (OLS) this yields an estimate of DF, which in turn enables to estimate the quadratic risk of lasso-OLS. Simulations demonstrate that the risk estimate leads to reliable model selection and that the risk estimate itself has smaller mean squared error than cross-validation estimate. For best subset selection a representation of \(df(\hat{\mu})-df_S(\hat{\mu})\) holds as well, but the situation is more complicated compared with lasso-OLS. However, it is possible to derive an approximation, which is exact for orthogonal designs.