An identity for kernel ridge regression (Q1939267)

Ridge regression is a powerful technique of machine learning. This paper proves an identity connecting the least square losses of ridge regression used on the same data in batch and on-line fashions. Although the problem and some conclusions are not entirely new, two proofs for the kernel case are proposed, one obtains the terms of the identity through calculating the same likelihood in a Gaussian processes model by three different methods, the other is based on the analysis of a Bayesian-type algorithm for prediction with expert advice. Moreover, the identity is used to derive several inequalities providing upper bounds for cumulative loss of ridge regression applied in the on-line fashion.