Full error analysis for the training of deep neural networks
From MaRDI portal
Publication:5083408
DOI10.1142/S021902572150020XzbMath1492.65132arXiv1910.00121OpenAlexW2977465327MaRDI QIDQ5083408
Christian Beck, Arnulf Jentzen, Benno Kuckuck
Publication date: 20 June 2022
Published in: Infinite Dimensional Analysis, Quantum Probability and Related Topics (Search for Journal in Brave)
Full work available at URL: https://arxiv.org/abs/1910.00121
convergence analysisartificial neural networksmachine learningapproximation errorgeneralization erroroptimization errorfull error analysis
Related Items
A proof of convergence for gradient descent in the training of artificial neural networks for constant target functions ⋮ A proof of convergence for stochastic gradient descent in the training of artificial neural networks with ReLU activation for constant target functions ⋮ Overall error analysis for the training of deep neural networks via stochastic gradient descent with random initialisation ⋮ Lower bounds for artificial neural network approximations: a proof that shallow neural networks fail to overcome the curse of dimensionality ⋮ Learning the random variables in Monte Carlo simulations with stochastic gradient descent: Machine learning for parametric PDEs and financial derivative pricing
Cites Work
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Tractability of multivariate problems. Volume I: Linear information
- Tractability of multivariate problems. Volume II: Standard information for functionals.
- Approximation and estimation bounds for artificial neural networks
- Multilayer feedforward networks are universal approximators
- General multilevel adaptations for stochastic approximation algorithms of Robbins-Monro and Polyak-Ruppert type
- Provable approximation properties for deep neural networks
- A distribution-free theory of nonparametric regression
- Degree of approximation by neural and translation networks with a single hidden layer
- Approximation of functions and their derivatives: A neural network implementation with applications
- Exponential convergence of the deep neural network approximation for analytic functions
- Topological properties of the set of functions generated by neural networks of fixed size
- Solving the Kolmogorov PDE by means of deep learning
- Proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients
- DNN expression rate analysis of high-dimensional PDEs: application to option pricing
- A theoretical analysis of deep neural networks and parametric PDEs
- Approximation spaces of deep neural networks
- Universal approximations of invariant maps by neural networks
- A proof of convergence for gradient descent in the training of artificial neural networks for constant target functions
- On the approximation by single hidden layer feedforward neural networks with fixed weights
- Optimal approximation of piecewise smooth functions using deep ReLU neural networks
- Gradient descent optimizes over-parameterized deep ReLU networks
- Nonlinear approximation via compositions
- A comparative analysis of optimization and generalization properties of two-layer neural network and random feature models under gradient descent dynamics
- A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations
- Error bounds for approximations with deep ReLU networks
- Lower error bounds for the stochastic gradient descent optimization algorithm: sharp convergence rates for slowly and fast decaying learning rates
- Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations
- Local Rademacher complexities
- On the mathematical foundations of learning
- Deep vs. shallow networks: An approximation theory perspective
- Sets of Finite Perimeter and Geometric Variational Problems
- Universal approximation bounds for superpositions of a sigmoidal function
- Neural Networks for Localized Approximation
- Deep learning in high dimension: Neural network expression rates for generalized polynomial chaos expansions in UQ
- Strong error analysis for stochastic gradient descent optimization algorithms
- Deep Neural Network Approximation Theory
- Optimal Approximation with Sparsely Connected Deep Neural Networks
- Analysis of the Generalization Error: Empirical Risk Minimization over Deep Artificial Neural Networks Overcomes the Curse of Dimensionality in the Numerical Approximation of Black--Scholes Partial Differential Equations
- Error bounds for approximations with deep ReLU neural networks in Ws,p norms
- Rectified deep neural networks overcome the curse of dimensionality for nonsmooth value functions in zero-sum games of nonlinear stiff systems
- Deep Neural Networks Algorithms for Stochastic Control Problems on Finite Horizon: Convergence Analysis
- Deep Network Approximation Characterized by Number of Neurons
- On Stochastic Gradient Langevin Dynamics with Dependent Data Streams: The Fully Nonconvex Case
- Equivalence of approximation by convolutional neural networks and fully-connected networks
- Probability Inequalities for Sums of Bounded Random Variables
- Breaking the Curse of Dimensionality with Convex Neural Networks
- Understanding Machine Learning
- Approximation by superpositions of a sigmoidal function
- Error bounds for approximation with neural networks
