Finite-dimensional regularization with nonidentity smoothing matrices (Q1115102)

Let x be the solution of the least-squares problem \(\| Kx-y\| \to \min.\), where K is a rectangular matrix assumed to have full rank. By \(x_{\alpha}\) we denote the regularized approximation, i.e., the unique minimizer of \(\| Kx-y\|^ 2+\alpha \| Lx\|^ 2\) for a certain matrix L. Now assume that y is perturbed by random noise \(\epsilon\) with \(E(\epsilon)=0\), \(Cov(\epsilon)=S\). The author investigates the behaviour of \(C(\alpha):=E(\| x-x_{\alpha}\|^ 2).\) He shows that if \(K=I\) or \(L=I\), C(\(\alpha)\) always has a minimum for \(\alpha >0\). On the other hand, for \(S=\sigma^ 2I\), \(C'(0)<0\) always hold. Numerical studies with different matrices L show that the shape of C(\(\alpha)\) may be quite complicated.