Geometrical aspects of discrimination by multilayer perceptrons
type: Item
instance of: scholarly article
title: Geometrical aspects of discrimination by multilayer perceptrons
zbMATH Open document ID: 0951.62047
author: Salvatore Ingrassia
published in: Journal of Multivariate Analysis
publication date: 2000-12-27T00:00:00Z
review text: The problem of the discrimination between two \(m\)-dimensional finite populations \(\Omega_1\) and \(\Omega_2\) by means of discriminant functions \(f_p(x)= \sum^p_{k=1} c_k\tau (a_k'x)\) is investigated, \(\tau (\cdot)\) being a sigmoidal function and \(a_1,a_2, \dots, a_p\in \mathbb{R}^m\), \(c_1, \dots, c_p\in \mathbb{R}\) being parameters. This function is realized by a one hidden layer perceptron with \(m\) units in the input layer, \(p\) units in the hidden layer, and one unit in the output layer. In the context of discriminant analysis, the function \(f_p\) performs the following two transformations: (1) a nonlinear transformation from \(\mathbb{R}^m\) to \(\mathbb{R}^p\) given by the sigmoidal function \(\tau\), and (2) a linear transformation from \(\mathbb{R}^p\) to \(\mathbb{R}\) according to \(c_1, \dots, c_p\). In fact, \(p\) measures the ``complexity'' of the target discriminant function to be estimated by \(f_p\). The role of the sigmoidal transformations in separating two nonlinearly separable populations is analyzed. For finite populations the discriminating power of \(f_p\) as \(p\) increases is investigated, and it is proved that there exists a finite \(p\) leading to the separation between the above populations by means of a function of kind \(f_p(x)\). Some results concerning geometrical aspects of the sigmoidal functions are given, and they are applied to the analysis of the discriminating power of the function \(f_p\). It is shown that the class of the populations that can be distinguished by \(f_p\) is monotonically increasing in \(p\), and a minimal sufficient \(p\) leading to a complete separation between the populations is given.
Mathematics Subject Classification ID: 62H30
Mathematics Subject Classification ID: 68T05
zbMATH DE Number: 1271312
zbMATH Keywords: multilayer perceptron
zbMATH Keywords: discrimination
zbMATH Keywords: sigmoidal function
reviewed by: V. Yu. Urbakh
MaRDI profile type: Publication
full work available at URL: jmva.1998.1786
OpenAlex ID: W2055555749
cites work: Q3661894
cites work: Universal approximation bounds for superpositions of a sigmoidal function
cites work: Neural networks: A review from a statistical perspective. With comments and a rejoinder by the authors
cites work: Q5516857
cites work: Q4298917
cites work: Q5526190
DOI: 10.1006/JMVA.1998.1786
Recommended article: Q5386558
Recommended article: Feed-forward neural networks: a geometrical perspective
Recommended article: Geometry of learning: Visualizing the performance of neural network supervised training methods
Recommended article: Some geometrical and topological properties of DNNs' decision boundaries
Recommended article: A geometric approach of gradient descent algorithms in linear neural networks
Recommended article: Geometry of Linear Convolutional Networks
Recommended article: Q4810100
Recommended article: Q4810099
description: scientific article; zbMATH DE number 1271312