Geometrical aspects of discrimination by multilayer perceptrons (Q1283923)

The problem of the discrimination between two \(m\)-dimensional finite populations \(\Omega_1\) and \(\Omega_2\) by means of discriminant functions \(f_p(x)= \sum^p_{k=1} c_k\tau (a_k'x)\) is investigated, \(\tau (\cdot)\) being a sigmoidal function and \(a_1,a_2, \dots, a_p\in \mathbb{R}^m\), \(c_1, \dots, c_p\in \mathbb{R}\) being parameters. This function is realized by a one hidden layer perceptron with \(m\) units in the input layer, \(p\) units in the hidden layer, and one unit in the output layer. In the context of discriminant analysis, the function \(f_p\) performs the following two transformations: (1) a nonlinear transformation from \(\mathbb{R}^m\) to \(\mathbb{R}^p\) given by the sigmoidal function \(\tau\), and (2) a linear transformation from \(\mathbb{R}^p\) to \(\mathbb{R}\) according to \(c_1, \dots, c_p\). In fact, \(p\) measures the ``complexity'' of the target discriminant function to be estimated by \(f_p\). The role of the sigmoidal transformations in separating two nonlinearly separable populations is analyzed. For finite populations the discriminating power of \(f_p\) as \(p\) increases is investigated, and it is proved that there exists a finite \(p\) leading to the separation between the above populations by means of a function of kind \(f_p(x)\). Some results concerning geometrical aspects of the sigmoidal functions are given, and they are applied to the analysis of the discriminating power of the function \(f_p\). It is shown that the class of the populations that can be distinguished by \(f_p\) is monotonically increasing in \(p\), and a minimal sufficient \(p\) leading to a complete separation between the populations is given.