On the eigenvector bias of Fourier feature networks: from regression to solving multi-scale PDEs with physics-informed neural networks

DOI10.1016/j.cma.2021.113938zbMath1506.35130arXiv2012.10047OpenAlexW3116268267WikidataQ114196890 ScholiaQ114196890MaRDI QIDQ2237440

Paris Perdikaris, Hanwen Wang, Sifan Wang

Publication date: 27 October 2021

Published in: Computer Methods in Applied Mechanics and Engineering (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2012.10047

zbMATH Keywords

partial differential equationsdeep learningscientific machine learningneural tangent kernelspectral bias

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Spectral, collocation and related methods for boundary value problems involving PDEs (65N35) General topics in linear spectral theory for PDEs (35P05) Fourier and Fourier-Stieltjes transforms and other transforms of Fourier type (42B10)

Related Items

SPINN: sparse, physics-based, and partially interpretable neural networks for PDEs, Solving and learning nonlinear PDEs with Gaussian processes, Enforcing exact physics in scientific machine learning: a data-driven exterior calculus on graphs, Multi-variance replica exchange SGMCMC for inverse and forward problems via Bayesian PINN, CAN-PINN: a fast physics-informed neural network based on coupled-automatic-numerical differentiation method, Physics-informed neural network simulation of multiphase poroelasticity using stress-split sequential training, Improved architectures and training algorithms for deep operator networks, Active Neuron Least Squares: A Training Method for Multivariate Rectified Neural Networks, Scalable uncertainty quantification for deep operator networks using randomized priors, Improved deep neural networks with domain decomposition in solving partial differential equations, A unified scalable framework for causal sweeping strategies for physics-informed neural networks (PINNs) and their temporal decompositions, Deep learning methods for partial differential equations and related parameter identification problems, Enhanced physics‐informed neural networks for hyperelasticity, MOD-Net: A Machine Learning Approach via Model-Operator-Data Network for Solving PDEs, BINN: a deep learning approach for computational mechanics problems based on boundary integral equations, Physics-informed radial basis network (PIRBN): a local approximating neural network for solving nonlinear partial differential equations, Pre-training strategy for solving evolution equations based on physics-informed neural networks, Physics-informed neural networks combined with polynomial interpolation to solve nonlinear partial differential equations, Long-time integration of parametric evolution equations with physics-informed DeepONets, HomPINNs: homotopy physics-informed neural networks for solving the inverse problems of nonlinear differential equations with multiple solutions, Respecting causality for training physics-informed neural networks, Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios, Efficient physics-informed neural networks using hash encoding, Bayesian Deep Learning Framework for Uncertainty Quantification in Stochastic Partial Differential Equations, Subspace decomposition based DNN algorithm for elliptic type multi-scale PDEs, Exact Dirichlet boundary physics-informed neural network EPINN for solid mechanics, Finite basis physics-informed neural networks (FBPINNs): a scalable domain decomposition approach for solving differential equations, A Neural Network Approach for Homogenization of Multiscale Problems, On the spectral bias of coupled frequency predictor-corrector triangular DNN: the convergence analysis, opPINN: physics-informed neural network with operator learning to approximate solutions to the Fokker-Planck-Landau equation, Feature-adjacent multi-fidelity physics-informed machine learning for partial differential equations, Gradient-enhanced physics-informed neural networks based on transfer learning for inverse problems of the variable coefficient differential equations, Is the neural tangent kernel of PINNs deep learning general partial differential equations always convergent?, Physics-informed ConvNet: learning physical field from a shallow neural network, Physical informed neural networks with soft and hard boundary constraints for solving advection-diffusion equations using Fourier expansions, Residual-based attention in physics-informed neural networks, Optimization of physics-informed neural networks for solving the nolinear Schrödinger equation, An unsupervised latent/output physics-informed convolutional-LSTM network for solving partial differential equations using peridynamic differential operator, Deep energy method in topology optimization applications, An overview on deep learning-based approximation methods for partial differential equations, Exact imposition of boundary conditions with distance functions in physics-informed deep neural networks, CENN: conservative energy method based on neural networks with subdomains for solving variational problems involving heterogeneous and complex geometries, NH-PINN: neural homogenization-based physics-informed neural network for multiscale problems, Unified regression model in fitting potential energy surfaces for quantum dynamics, Theory-guided physics-informed neural networks for boundary layer problems with singular perturbation, A deep domain decomposition method based on Fourier features

Uses Software

• Chebfun
• NumPy
• Matplotlib
• DiffSharp
• Adam
• GitHub
• JAX
• PhyGeoNet
• DeepONet
• PINNsNTK
• MscaleDNN
• NSFnets

Cites Work

• Unnamed Item
• Unnamed Item
• Exponential time differencing for stiff systems
• Machine learning in cardiovascular flows modeling: predicting arterial blood pressure from non-invasive 4D flow MRI data using physics-informed neural networks
• Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data
• Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data
• Adversarial uncertainty quantification in physics-informed neural networks
• Simulator-free solution of high-dimensional stochastic elliptic partial differential equations using deep neural networks
• Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
• Spectral Methods for Time-Dependent Problems
• On the Eigenspectrum of the Gram Matrix and the Generalization Error of Kernel-PCA
• Nonlinear analysis of hydrodynamic instability in laminar flames—I. Derivation of basic equations
• Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
• Solving high-dimensional partial differential equations using deep learning
• Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations
• Bayesian differential programming for robust systems identification under uncertainty
• On the Convergence of Physics Informed Neural Networks for Linear Second-Order Elliptic and Parabolic Type PDEs
• Wide neural networks of any depth evolve as linear models under gradient descent ^*