Why Does Large Batch Training Result in Poor Generalization? A Comprehensive Explanation and a Better Strategy from the Viewpoint of Stochastic Optimization

DOI10.1162/neco_a_01089zbMath1472.68159OpenAlexW2797559419WikidataQ52322412 ScholiaQ52322412MaRDI QIDQ5157215

Tomoumi Takase, Satoshi Oyama, Masahito Kurihara

Publication date: 12 October 2021

Published in: Neural Computation (Search for Journal in Brave)

Full work available at URL: http://hdl.handle.net/2115/71558

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Nonconvex programming, global optimization (90C26) Learning and adaptive systems in artificial intelligence (68T05) Stochastic programming (90C15)

Uses Software

• MNIST
• RMSprop
• Adam
• ADADELTA

Cites Work

• Optimization by Simulated Annealing
• Thermodynamical approach to the travelling salesman problem: An efficient simulation algorithm
• Future paths for integer programming and links to artificial intelligence
• Convergence of an annealing algorithm
• Unnamed Item
• Unnamed Item
• Unnamed Item

This page was built for publication: Why Does Large Batch Training Result in Poor Generalization? A Comprehensive Explanation and a Better Strategy from the Viewpoint of Stochastic Optimization