Regularisation of neural networks by enforcing Lipschitz continuity

DOI10.1007/s10994-020-05929-wOpenAlexW3113374047MaRDI QIDQ2051250

Michael J. Cree, Henry Gouk, Eibe Frank, Bernhard Pfahringer

Publication date: 24 November 2021

Published in: Machine Learning (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1804.04368

zbMATH Keywords

neural networksLipschitz continuityregularisation

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Related Items

CLIP: cheap Lipschitz training of neural networks ⋮ Regularization via Mass Transportation ⋮ System identification through Lipschitz regularized deep neural networks ⋮ On quadrature rules for solving partial differential equations using neural networks ⋮ Lipschitzness is all you need to tame off-policy generative adversarial imitation learning ⋮ Stable parameterization of continuous and piecewise-linear functions ⋮ Principled deep neural network training through linear programming ⋮ Lipschitz stability analysis of fractional-order impulsive delayed reaction-diffusion neural network models ⋮ An Unrolled Implicit Regularization Network for Joint Image and Sensitivity Estimation in Parallel MR Imaging with Convergence Guarantee ⋮ Feature importance in neural networks as a means of interpretation for data-driven turbulence models ⋮ Approximation of Lipschitz Functions Using Deep Spline Neural Networks ⋮ Deep-plug-and-play proximal Gauss-Newton method with applications to nonlinear, ill-posed inverse problems ⋮ Diametrical risk minimization: theory and computations ⋮ Connections between numerical algorithms for PDEs and neural networks ⋮ Learning‐based adaptive‐scenario‐tree model predictive control with improved probabilistic safety using robust Bayesian neural networks ⋮ Data-consistent neural networks for solving nonlinear inverse problems ⋮ Residual networks as flows of diffeomorphisms

Uses Software

• CIFAR
• Hyperopt
• Adam
• Fashion-MNIST
• Wasserstein GAN

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Tikhonov, Ivanov and Morozov regularization for support vector machine learning
• Robustness and generalization
• The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network
• Lipschitz properties for deep convolutional networks
• Size-independent sample complexity of neural networks
• On Lipschitz Bounds of General Convolutional Neural Networks
• Understanding Machine Learning