Born-Infeld (BI) for AI: Energy-Conserving Descent (ECD) for Optimization

Author name not available (Why is that?)

Publication date: 26 January 2022

Abstract: We introduce a novel framework for optimization based on energy-conserving Hamiltonian dynamics in a strongly mixing (chaotic) regime and establish its key properties analytically and numerically. The prototype is a discretization of Born-Infeld dynamics, with a squared relativistic speed limit depending on the objective function. This class of frictionless, energy-conserving optimizers proceeds unobstructed until slowing naturally near the minimal loss, which dominates the phase space volume of the system. Building from studies of chaotic systems such as dynamical billiards, we formulate a specific algorithm with good performance on machine learning and PDE-solving tasks, including generalization. It cannot stop at a high local minimum, an advantage in non-convex loss functions, and proceeds faster than GD+momentum in shallow valleys.

Has companion code repository: https://github.com/gbdl/bbi

This page was built for publication: Born-Infeld (BI) for AI: Energy-Conserving Descent (ECD) for Optimization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6389288)