Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be (Q6434495)
From MaRDI portal
 | This is the item page for this Wikibase entity, intended for internal use and editing purposes. Please use this page instead for the normal view: Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be |
preprint article from arXiv
| Language | Label | Description | Also known as |
|---|---|---|---|
| English | Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be |
preprint article from arXiv |
Statements
27 April 2023
0 references
cs.LG
0 references
math.OC
0 references
Frederik Kunstner
0 references
Jacques Chen
0 references
Jonathan Wilder Lavington
0 references
Mark Schmidt
0 references
