Least square regression with indefinite kernels and coefficient regularization (Q617706)

Let \((y_i,x_i)_{i=1}^m\) be i.i.d. observations with \(y_i\in\mathbb R\), \(x_i\in X\), \(X\) being some compact metric space. The authors consider estimates \(f_z\) for the regression function \(f_\rho(x)=E(y_i|x_i)\), where \(f_z=f_{\alpha^z}\), \(f_\alpha(x)=\sum_{i=1}^m \alpha_i K(x,x_i)\), \[ \alpha^z=\arg\min_{\alpha\in R^m} {1\over m}\,\sum_{i=1}^m (y_i - f_\alpha(x_i))^2+\lambda m\sum_{i=1}^m\alpha_i^2, \] \(K:X\times X\to\mathbb R\) is a continuous bounded function (kernel), \(\lambda\) is a regularization parameter. Consistency of \(f_z\) is demonstrated under the assumptions that \(\lambda=\lambda(m)\to 0\), \(\lambda^{3/2}\sqrt{m}\to\infty\) and the true regression function belongs to the closure of \(\{f_\alpha\}\) in some suitable reproducing kernel Hilbert space. To analyze the rates of convergence the authors make assumptions of the form \( E\|L^{-r}f_\rho(x_i)\|^2<\infty\) for some \(r>0\), where \(L f(x)=E \tilde K(x,x_i)f(x_i)\), \(\tilde K(x,t)=E_u K(x,u)K(t,u) \). E.g. if \(r>1\) then choosing \(\lambda=m^{1/5}\) they get \(\|f_z-f_\rho\|_{L^2}=O(m^{-1/5})\). Results of simulations are presented for \(X=[0,1]\) and the Gaussian kernel \(K\).