Why Shallow Networks Struggle with Approximating and Learning High Frequency: A Numerical Study
From MaRDI portal
Publication:6442044
arXiv2306.17301MaRDI QIDQ6442044
Author name not available (Why is that?)
Publication date: 29 June 2023
Abstract: In this work, a comprehensive numerical study involving analysis and experiments shows why a two-layer neural network has difficulties handling high frequencies in approximation and learning when machine precision and computation cost are important factors in real practice. In particular, the following fundamental computational issues are investigated: (1) the best accuracy one can achieve given a finite machine precision, (2) the computation cost to achieve a given accuracy, and (3) stability with respect to perturbations. The key to the study is the spectral analysis of the corresponding Gram matrix of the activation functions which also shows how the properties of the activation function play a role in the picture.
Has companion code repository: https://github.com/shijunzhangmath/mmnn
This page was built for publication: Why Shallow Networks Struggle with Approximating and Learning High Frequency: A Numerical Study
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6442044)
