Side Effects of Learning from Low-dimensional Data Embedded in a Euclidean Space
From MaRDI portal
Publication:6392496
DOI10.1007/S40687-023-00378-YarXiv2203.00614MaRDI QIDQ6392496
Richard Tsai, Juncai He, Rachel Ward
Publication date: 1 March 2022
Abstract: The low-dimensional manifold hypothesis posits that the data found in many applications, such as those involving natural images, lie (approximately) on low-dimensional manifolds embedded in a high-dimensional Euclidean space. In this setting, a typical neural network defines a function that takes a finite number of vectors in the embedding space as input. However, one often needs to consider evaluating the optimized network at points outside the training distribution. This paper considers the case in which the training data is distributed in a linear subspace of . We derive estimates on the variation of the learning function, defined by a neural network, in the direction transversal to the subspace. We study the potential regularization effects associated with the network's depth and noise in the codimension of the data manifold. We also present additional side effects in training due to the presence of noise.
This page was built for publication: Side Effects of Learning from Low-dimensional Data Embedded in a Euclidean Space
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6392496)
