Analysis of generalized Bregman surrogate algorithms for nonsmooth nonconvex statistical learning

DOI10.1214/21-AOS2090zbMath1486.62212arXiv2112.09191OpenAlexW4200303272MaRDI QIDQ2073715

Yiyuan She, Jiuwu Jin, Zhifeng Wang

Publication date: 7 February 2022

Published in: The Annals of Statistics (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2112.09191

zbMATH Keywords

nonsmooth optimizationnonconvex optimizationBregman divergencestatistical algorithmic analysisMM algorithmsmomentum-based acceleration

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Ridge regression; shrinkage estimators (Lasso) (62J07) Linear regression; mixed models (62J05) Nonconvex programming, global optimization (90C26) Learning and adaptive systems in artificial intelligence (68T05)

Related Items

Gaining Outlier Resistance With Progressive Quantiles: Fast Algorithms and Theoretical Studies, A framework of regularized low-rank matrix models for regression and classification

Uses Software

• AdaGrad

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Nearly unbiased variable selection under minimax concave penalty
• A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
• Accelerated gradient methods for nonconvex nonlinear and stochastic programming
• Optimal computational and statistical rates of convergence for sparse nonconvex learning problems
• Oracle inequalities in empirical risk minimization and sparse recovery problems. École d'Été de Probabilités de Saint-Flour XXXVIII-2008.
• Oracle inequalities and optimal inference under group sparsity
• Iterative hard thresholding for compressed sensing
• Fast global convergence of gradient methods for high-dimensional statistical recovery
• One-step sparse estimates in nonconcave penalized likelihood models
• Introductory lectures on convex optimization. A basic course.
• The DC (Difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems
• Weak convergence and empirical processes. With applications to statistics
• An iterative algorithm for fitting nonconvex penalized generalized linear models with grouped predictors
• Thresholding-based iterative selection procedures for model selection and shrinkage
• On the conditions used to prove oracle results for the Lasso
• Simultaneous analysis of Lasso and Dantzig selector
• Sparsity oracle inequalities for the Lasso
• MM algorithms for geometric and signomial programming
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• Strong oracle optimality of folded concave penalized estimation
• Variable selection using MM algorithms
• Penalized Bregman divergence for large-dimensional regression and classification
• Convergence Analysis of a Proximal-Like Minimization Algorithm Using Bregman Functions
• Decoding by Linear Programming
• Graphical Models, Exponential Families, and Variational Inference
• Ideal spatial adaptation by wavelet shrinkage
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• Recovering Sparse Signals With a Certain Family of Nonconvex Penalties and DC Programming
• Group Iterative Spectrum Thresholding for Super-Resolution Sparse Spectral Selection
• Learning Topology and Dynamics of Large Recurrent Neural Networks
• A Statistical View of Some Chemometrics Regression Tools
• Learning the parts of objects by non-negative matrix factorization
• Regularized M-estimators with nonconvexity: Statistical and algorithmic theory for local optima
• Convex Analysis
• Robust Statistics
• On the finite-sample analysis of \(\Theta\)-estimators