Optimization Dynamics of Equivariant and Augmented Neural Networks

Author name not available (Why is that?)

Publication date: 23 March 2023

Abstract: We investigate the optimization of multilayer perceptrons on symmetric data. We compare the strategy of constraining the architecture to be equivariant to that of using augmentation. We show that, under natural assumptions on the loss and non-linearities, the sets of equivariant stationary points are identical for the two strategies, and that the set of equivariant layers is invariant under the gradient flow for augmented models. Finally, we show that stationary points may be unstable for augmented training although they are stable for the equivariant models

Has companion code repository: https://github.com/usinedepain/eq_aug_dyn

Mathematics Subject Classification ID

No records found.

This page was built for publication: Optimization Dynamics of Equivariant and Augmented Neural Networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6430637)