Gradient methods for optimizing metaparameters in the knowledge distillation problem

DOI10.1134/S00051179220100071zbMath1506.68092OpenAlexW4312589547MaRDI QIDQ2689575

M. Gorpinich, O. Yu. Bakhteev, Vadim V. Strijov

Publication date: 13 March 2023

Published in: Automation and Remote Control (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1134/s00051179220100071

zbMATH Keywords

machine learninggradient optimizationknowledge distillationmetaparameter assignmentmetaparameter optimization

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Applications of mathematical programming (90C90) Learning and adaptive systems in artificial intelligence (68T05)

Uses Software

• CIFAR
• GitHub
• DeepSpeed

Cites Work

• GitHub
• Comprehensive analysis of gradient-based hyperparameter optimization algorithms
• Unnamed Item
• Unnamed Item

This page was built for publication: Gradient methods for optimizing metaparameters in the knowledge distillation problem