Optimal re-materialization strategies for heterogeneous chains: how to train deep neural networks with limited memory

DOI10.1145/3648633MaRDI QIDQ6604162

Alena Shilova, Lionel Eyraud-Dubois, Alexis Joly, Julien Herrmann, Olivier Beaumont

Publication date: 12 September 2024

Published in: ACM Transactions on Mathematical Software (Search for Journal in Brave)

zbMATH Keywords

dynamic programmingcheckpointingmemoryconvolutional neural networksre-materialization

Mathematics Subject Classification ID

Numerical analysis (65-XX)

Cites Work

• DAG reversal is NP-complete
• Optimal multistage algorithm for adjoint computation
• Evaluating Derivatives
• Algorithm 799: revolve
• Divide-and-conquer checkpointing for arbitrary programs with no user annotation
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)

This page was built for publication: Optimal re-materialization strategies for heterogeneous chains: how to train deep neural networks with limited memory

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6604162)