Stabilize deep ResNet with a sharp scaling factor \(\tau\)

DOI10.1007/S10994-022-06192-XOpenAlexW4289345622MaRDI QIDQ2102389

Tie-Yan Liu, Mingyang Yi, Da Yu, Wei Chen, Huishuai Zhang

Publication date: 28 November 2022

Published in: Machine Learning (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1007/s10994-022-06192-x

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Uses Software

• CIFAR
• Tensor2Tensor
• BERT

Cites Work

• Adaptive estimation of a quadratic functional by model selection.
• Gradient descent optimizes over-parameterized deep ReLU networks
• Smoothed analysis of algorithms
• Stable architectures for deep neural networks
• A mean field view of the landscape of two-layer neural networks
• Convex Formulation of Overparameterized Deep Neural Networks

This page was built for publication: Stabilize deep ResNet with a sharp scaling factor \(\tau\)