Neural collapse under cross-entropy loss

DOI10.1016/J.ACHA.2021.12.011OpenAlexW4205698766MaRDI QIDQ2134115

Stefan Steinerberger, Jian-feng Lu

Publication date: 6 May 2022

Published in: Applied and Computational Harmonic Analysis (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2012.08465

zbMATH Keywords

neural networksframe potentialcross-entropy lossneural collapse

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Nonconvex programming, global optimization (90C26) General harmonic expansions, frames (42C15)

Uses Software

• word2vec

Cites Work

• Optimal asymptotic bounds for spherical designs
• Finite normalized tight frames
• Energy on spheres and discreteness of minimizing measures
• The spectral function of an elliptic operator
• Positive definite functions on spheres
• Prevalence of neural collapse during the terminal phase of deep learning training

This page was built for publication: Neural collapse under cross-entropy loss