Accelerated information gradient flow
type: Item
instance of: scholarly article
title: Accelerated information gradient flow
zbMATH Open document ID: 1484.90059
author: Wuchen Li
author: Yifei Wang
published in: Journal of Scientific Computing
publication date: 2021-11-29T00:00:00Z
full work available at URL: https://arxiv.org/abs/1909.02102
review text: Nesterov's accelerated gradient method for classical optimization problems in the Euclidean space is a wide-applied optimization method and it accelerates a gradient descent method. The continuous-time limit of this method is known as the accelerated gradient flow problem. In this paper, the authors present a unified framework of the accelerated gradient flow in the probability space embedded with information metrics, named accelerated information gradient (AIG). They formulate an AIG flow, under the Fisher-Rao metric, Wasserstein metric, Kalman-Wasserstein metric and Stein metric and theoretically prove the convergence rate of the AIG flow. The authors present the discrete-time algorithm for the Wasserstein AIG flow, including the Brownian motions method and the adaptive restart technique. Numerical experiments, including a Bayesian logistic regression and a Bayesian neural network, show the strength of the proposed methods compared with state-of-the-art algorithms. In supplementary materials, the authors provide discrete-time algorithms for both Kalman-Wasserstein AIG and Stein AIG flows.
Mathematics Subject Classification ID: 90C15
zbMATH DE Number: 7435304
zbMATH Keywords: Nesterov's accelerated gradient method
zbMATH Keywords: Bayesian inverse problem
zbMATH Keywords: optimal transport
zbMATH Keywords: information geometry
reviewed by: I. M. Stancu-Minasian
describes a project that uses: Adam
MaRDI profile type: Publication
OpenAlex ID: W2971906246
arXiv ID: 1909.02102
cites work: Information geometry and its applications
cites work: Q3995194
cites work: Convergence to equilibrium in Wasserstein distance for damped Euler equations with interaction forces
cites work: A blob method for diffusion
cites work: Wasserstein Hamiltonian flows
cites work: Q4877337
cites work: Interacting Langevin Diffusions: Gradient Structure and Ensemble Kalman Sampler
cites work: The Density Manifold and Configuration Space Quantization
cites work: Is there an analog of Nesterov acceleration for gradient-based MCMC?
cites work: Wasserstein Riemannian geometry of Gaussian densities
cites work: Geometry of matrix decompositions seen through optimal transport and information geometry
cites work: Q3320132
cites work: THE GEOMETRY OF DISSIPATIVE EVOLUTION EQUATIONS: THE POROUS MEDIUM EQUATION
cites work: Adaptive restart for accelerated gradient schemes
cites work: Improvement on some known nonparametric uniformly consistent estimators of derivatives of a density
cites work: Functional and shape data analysis
cites work: Inverse problems: a Bayesian perspective
cites work: A Mean-Field Optimal Control Formulation for Global Optimization
cites work: On Wasserstein geometry of the space of Gaussian measures
cites work: Q4805362
cites work: A variational perspective on accelerated methods in optimization
DOI: 10.1007/S10915-021-01709-3
description: scientific article