Kernel-based linear system identification: when does the representer theorem hold? (Q6537302)

Kernel-based identification is an increasingly popular paradigm in learning dynamic systems from data. It is centred on estimation of an unknown function (an impulse response) from a finite set of linearly related noisy measurements. Mathematically, this function is assumed to belong to a reproducing kernel Hilbert space (RKHS), a particular space of functions induced by positive definite kernels. The corresponding estimate minimizes an objective functional which trades-off measurements fit and the RKHS norm which acts as regularizer.\N\NIn this setting, the representer theorem (RT) constitutes a main result in kernel-based linear system identification, which states that regularized estimates of impulse responses are sums of a finite number of basis functions obtained by convolving the kernel with the system input. For its applicability, however, the structure of the kernel-based estimator should satisfy some technical conditions regarding the continuity of the functionals which map the impulse responses into the system output.\N\NThe key result of the paper in question regards the family of stable RKHSs induced by discrete-time kernels or continuous-time Mercer (continuous) kernels. The authors show that this family coincides with the class of RKHSs where all the convolutions induced by physical (bounded) inputs are continuous functionals. In turn, this proves a fundamental connection between RT and BIBO stability, the latter meaning that the output of a dynamic system fed with a bounded input remains bounded. Specifically, RT is valid given any physical (bounded) input to the system if and only if the kernel is stable. In other words, RKHS stability is the necessary and sufficient condition to make kernel based linear system identification well-posed. As a by-product, a new stability test is obtained and a the relationship between uniform continuity of convolutions and stable RKHSs is derived. Overall, the paper is an extremely good and valuable piece of work on a subject that attracts considerable attention in statistics, system identification and machine learning.