Deep neural networks motivated by partial differential equations

DOI10.1007/s10851-019-00903-1zbMath1434.68522arXiv1804.04272OpenAlexW2974916071WikidataQ127229601 ScholiaQ127229601MaRDI QIDQ1988348

Lars Ruthotto, Eldad Haber

Publication date: 23 April 2020

Published in: Journal of Mathematical Imaging and Vision (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1804.04272

zbMATH Keywords

partial differential equationsmachine learningimage classificationPDE-constrained optimizationdeep neural networks

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Computing methodologies for image processing (68U10) PDEs in connection with computer science (35Q68)

Related Items

Translating numerical concepts for PDEs into neural architectures ⋮ Using neural networks to accelerate the solution of the Boltzmann equation ⋮ A New Artificial Neural Network Method for Solving Schrödinger Equations on Unbounded Domains ⋮ Deep microlocal reconstruction for limited-angle tomography ⋮ On quadrature rules for solving partial differential equations using neural networks ⋮ Mean-field and kinetic descriptions of neural differential equations ⋮ Wasserstein-Based Projections with Applications to Inverse Problems ⋮ Modelling spatiotemporal dynamics from Earth observation data with neural differential equations ⋮ Designing rotationally invariant neural networks from PDEs and variational methods ⋮ slimTrain---A Stochastic Approximation Method for Training Separable Deep Neural Networks ⋮ Structure preservation for the deep neural network multigrid solver ⋮ Deep Neural Networks and PIDE Discretizations ⋮ Strong stationarity for optimal control problems with non-smooth integral equation constraints: application to a continuous DNN ⋮ Deep learning methods for partial differential equations and related parameter identification problems ⋮ Deep limits of residual neural networks ⋮ Learning high-dimensional parametric maps via reduced basis adaptive residual networks ⋮ Model discovery of compartmental models with graph-supported neural networks ⋮ Accuracy and architecture studies of residual neural network method for ordinary differential equations ⋮ Pseudo-Hamiltonian neural networks for learning partial differential equations ⋮ A deep Fourier residual method for solving PDEs using neural networks ⋮ Optimal control by deep learning techniques and its applications on epidemic models ⋮ An ODE-based neural network with Bayesian optimization ⋮ DRIP: deep regularizers for inverse problems ⋮ A functional approach to interpreting the role of the adjoint equation in machine learning ⋮ Estimating a Potential Without the Agony of the Partition Function ⋮ Connections between numerical algorithms for PDEs and neural networks ⋮ PDE-based group equivariant convolutional neural networks ⋮ Discrete processes and their continuous limits ⋮ Bounded nonlinear forecasts of partially observed geophysical systems with physics-constrained deep learning ⋮ Deep learning via dynamical systems: an approximation perspective ⋮ Learning Hamiltonian systems with mono-implicit Runge-Kutta methods ⋮ Dynamical Systems–Based Neural Networks ⋮ Nonparametric inference of stochastic differential equations based on the relative entropy rate ⋮ Convex and concave envelopes of artificial neural network activation functions for deterministic global optimization ⋮ Discovering first principle of behavioural change in disease transmission dynamics by deep learning ⋮ On mathematical modeling in image reconstruction and beyond ⋮ A review on deep learning in medical image reconstruction ⋮ Convolutional Neural Networks in Phase Space and Inverse Problems ⋮ Deep learning methods for the computation of vibrational wavefunctions ⋮ PNKH-B: A Projected Newton--Krylov Method for Large-Scale Bound-Constrained Optimization ⋮ Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection ⋮ Scale-covariant and scale-invariant Gaussian derivative networks ⋮ An Efficient Parallel-in-Time Method for Optimization with Parabolic PDEs ⋮ Model reduction and neural networks for parametric PDEs ⋮ The Random Feature Model for Input-Output Maps between Banach Spaces ⋮ Numerical solving of nonlinear differential equations using a hybrid method on a semi-infinite interval ⋮ Some open questions on morphological operators and representations in the deep learning era. A personal vision ⋮ Quantized convolutional neural networks through the lens of partial differential equations ⋮ Fully hyperbolic convolutional neural networks ⋮ Bilevel optimization, deep learning and fractional Laplacian regularization with applications in tomography ⋮ Structure-preserving deep learning

Uses Software

• PESC
• CIFAR
• ImageNet
• FAIR.m
• torchdiffeq
• AlexNet
• STL-10 dataset
• DeepXplore

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Nonlinear total variation based noise removal algorithms
• Variational methods in imaging
• DeepXplore
• Deep neural network structures solving variational inequalities
• Deep relaxation: partial differential equations for optimizing deep neural networks
• A proposal on machine learning via dynamical systems
• Determining optical flow
• Optimal approximations by piecewise smooth functions and associated variational problems
• Algorithms for PDE-constrained optimization
• Computational Optimization of Systems Governed by Partial Differential Equations
• Approximation of functional depending on jumps by elliptic functional via t-convergence
• Learning Deep Architectures for AI
• Deblurring Images
• FAIR
• Active contours without edges
• Stable architectures for deep neural networks
• Real-Time PDE-Constrained Optimization