On the generalization problem (Q5956967)

The generalization problem consists of finding the type of networks of which the output \(y\) will approximate the output \(s(x)\) of a given system for any value of the system input \(x\). Three cases are considered: (a) \((0,1)\) output; (b) accurate real output and (c) noisy real output. It is shown that it is necessary to have prior knowledge about the output of the system, to solve the problem accurately(?). In the case (c), the least squares approach works badly, and a regularization method is proposed to circumvent this defect. The cases (a) and (b) lead to constrained nonlinear optimization problems.