Better Approximations of High Dimensional Smooth Functions by Deep Neural Networks with Rectified Power Units

DOI10.4208/cicp.OA-2019-0168zbMath1474.65031arXiv1903.05858MaRDI QIDQ5162006

Hai-jun Yu, Bo Li, Shanshan Tang

Publication date: 1 November 2021

Published in: Communications in Computational Physics (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1903.05858

zbMATH Keywords

sparse gridsdeep neural networkhigh dimensional approximationrectified linear unitrectified power unitrectified quadratic unit

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Stability and convergence of numerical methods for initial value and initial-boundary value problems involving PDEs (65M12) Error bounds for initial value and initial-boundary value problems involving PDEs (65M15) Algorithms for approximation of functions (65D15)

Related Items (16)

High Order Deep Neural Network for Solving High Frequency Partial Differential Equations ⋮ Convergence Rate Analysis for Deep Ritz Method ⋮ Discovering phase field models from image data with the pseudo-spectral physics informed neural networks ⋮ Sparse approximation of triangular transports. I: The finite-dimensional case ⋮ A convergent deep learning algorithm for approximation of polynomials ⋮ The robust physics-informed neural networks for a typical fourth-order phase field model ⋮ Optimal approximation of infinite-dimensional holomorphic functions ⋮ Two-layer networks with the \(\text{ReLU}^k\) activation function: Barron spaces and derivative approximation ⋮ Neural Control of Parametric Solutions for High-Dimensional Evolution PDEs ⋮ A functional equation with polynomial solutions and application to neural networks ⋮ Efficient spectral methods for quasi-equilibrium closure approximations of symmetric problems on unit circle and sphere ⋮ Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks ⋮ Approximation rates for neural networks with encodable weights in smoothness spaces ⋮ Constructive deep ReLU neural network approximation ⋮ Solving parametric partial differential equations with deep rectified quadratic unit neural networks ⋮ Exponential ReLU DNN expression of holomorphic maps in high dimension

Uses Software

• darch
• ImageNet
• OCVdM

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Optimally scaled and optimally conditioned vandermonde and Vandermonde-like matrices
• A dynamic-solver-consistent minimum action method: with an application to 2D Navier-Stokes equations
• The hyperbolic cross space approximation of electronic wavefunctions
• When are quasi-Monte Carlo algorithms efficient for high dimensional integrals?
• Numerical integration using sparse grids
• Sparse finite elements for stochastic elliptic problems --- higher order moments
• The Deep Ritz Method: a deep learning-based numerical algorithm for solving variational problems
• Optimal nonlinear approximation
• Approximation properties of a multilayered feedforward artificial neural network
• High dimensional polynomial interpolation on sparse grids
• Exponential convergence of the deep neural network approximation for analytic functions
• Optimal approximation of piecewise smooth functions using deep ReLU neural networks
• Error bounds for approximations with deep ReLU networks
• Sparse grid discontinuous Galerkin methods for high-dimensional elliptic equations
• Approximations by orthonormal mapped Chebyshev functions for higher-dimensional problems in unbounded domains
• An Adaptive Sparse Grid Algorithm for Elliptic PDEs with Lognormal Diffusion Coefficient
• Efficient Spectral-Element Methods for the Electronic Schrödinger Equation
• Efficient Spectral Sparse Grid Methods and Applications to High-Dimensional Elliptic Equations II. Unbounded Domains
• Spectral Methods
• On the Expressive Power of Deep Architectures
• Relu Deep Neural Networks and Linear Finite Elements
• A Sparse Grid Stochastic Collocation Method for Partial Differential Equations with Random Input Data
• Approximation by Ridge Functions and Neural Networks
• Neural Networks for Localized Approximation
• Solving high-dimensional partial differential equations using deep learning
• New Error Bounds for Deep ReLU Networks Using Sparse Grids
• PowerNet: Efficient Representations of Polynomials and Smooth Functions by Deep Neural Networks with Rectified Power Units
• Deep ReLU networks and high-order finite element methods
• Deep Potential: A General Representation of a Many-Body Potential Energy Surface
• Data-Driven, Physics-Based Feature Extraction from Fluid Flow Fields using Convolutional Neural Networks
• Model Reduction with Memory and the Machine Learning of Dynamical Systems
• Finding the Origin of Noise Transients in LIGO Data with Machine Learning
• Efficient Spectral Sparse Grid Methods and Applications to High-Dimensional Elliptic Problems
• Error Analysis of the Dynamically Orthogonal Approximation of Time Dependent Random PDEs
• A Nonhomogeneous Kinetic Model of Liquid Crystal Polymers and Its Thermodynamic Closure Approximation
• Sparse Spectral Approximations of High-Dimensional Problems Based on Hyperbolic Cross
• Sparse grids for the Schrödinger equation
• A Fast Learning Algorithm for Deep Belief Nets
• Why Are High-Dimensional Finance Problems Often of Low Effective Dimension?
• A logical calculus of the ideas immanent in nervous activity
• Approximation by superpositions of a sigmoidal function
• A sparse finite element method with high accuracy. I

This page was built for publication: Better Approximations of High Dimensional Smooth Functions by Deep Neural Networks with Rectified Power Units