On the Convergence of Stochastic Gradient Descent for Nonlinear Ill-Posed Problems

DOI10.1137/19M1271798zbMath1487.65061arXiv1907.03132OpenAlexW3028233350MaRDI QIDQ5110563

Jun Zou, Zehui Zhou, Bangti Jin

Publication date: 20 May 2020

Published in: SIAM Journal on Optimization (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1907.03132

zbMATH Keywords

convergence ratesnonlinear inverse problemsstochastic gradient descentregularizing property

Mathematics Subject Classification ID

Nonlinear ill-posed problems (47J06) Numerical solutions of ill-posed problems in abstract spaces; regularization (65J20) Numerical solution to inverse problems in abstract spaces (65J22)

Related Items

Stochastic asymptotical regularization for linear inverse problems, Stochastic gradient descent for linear inverse problems in Hilbert spaces, An extended primal-dual algorithm framework for nonconvex problems: application to image reconstruction in spectral CT, On maximum residual nonlinear Kaczmarz-type algorithms for large nonlinear systems of equations, Stochastic mirror descent method for linear ill-posed problems in Banach spaces, Stochastic linear regularization methods: random discrepancy principle and applications, On the Convergence of Stochastic Gradient Descent for Linear Inverse Problems in Banach Spaces, On pseudoinverse-free block maximum residual nonlinear Kaczmarz method for solving large-scale nonlinear system of equations, On sampling Kaczmarz-Motzkin methods for solving large-scale nonlinear systems, On the discrepancy principle for stochastic gradient descent, Nonlinear greedy relaxed randomized Kaczmarz method, On stochastic Kaczmarz type methods for solving large scale systems of ill-posed equations, An analysis of stochastic variance reduced gradient for linear inverse problems ^*, Reconstructing the Thermal Phonon Transmission Coefficient at Solid Interfaces in the Phonon Transport Equation

Uses Software

• Adam

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Nonparametric stochastic approximation with large step-sizes
• Regularization methods in Banach spaces.
• Iterative regularization methods for nonlinear ill-posed problems
• Variational methods in imaging
• A randomized Kaczmarz algorithm with exponential convergence
• Online gradient descent learning algorithms
• A uniform approach to gradient methods for linear operator equations
• A convergence analysis of the Landweber iteration for nonlinear ill-posed problems
• Bouligand-Landweber iteration for a non-smooth ill-posed problem
• Inverse Problems
• Online Learning as Stochastic Approximation of Regularization Paths: Optimality and Almost-Sure Convergence
• A new approach to nonlinear constrained Tikhonov regularization
• Relaxation methods for image reconstruction
• Online learning in optical tomography: a stochastic approach
• Preasymptotic convergence of randomized Kaczmarz method
• Optimal Rates for Multi-pass Stochastic Gradient Methods
• Optimization Methods for Large-Scale Machine Learning
• On the regularizing property of stochastic gradient descent
• Projected Stochastic Gradients for Convex Constrained Problems in Hilbert Spaces
• Phase retrieval via randomized Kaczmarz: theoretical guarantees
• An Iteration Formula for Fredholm Integral Equations of the First Kind
• A Stochastic Approximation Method