A precise high-dimensional asymptotic theory for boosting and minimum-\(\ell_1\)-norm interpolated classifiers

DOI10.1214/22-AOS2170zbMath1490.68188arXiv2002.01586OpenAlexW4283031248WikidataQ114060459 ScholiaQ114060459MaRDI QIDQ2148995

Tengyuan Liang, Pragya Sur

Publication date: 24 June 2022

Published in: The Annals of Statistics (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2002.01586

zbMATH Keywords

overparametrizationboostinghigh-dimensional asymptoticsmargin theoryminimum-norm interpolation

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05)

Related Items (7)

Mehler’s Formula, Branching Process, and Compositional Kernels of Deep Neural Networks ⋮ Sharp global convergence guarantees for iterative nonconvex optimization with random data ⋮ Noisy linear inverse problems under convex constraints: exact risk asymptotics in high dimensions ⋮ Universality of regularized regression estimators in high dimensions ⋮ On the robustness of minimum norm interpolators and regularized empirical risk minimizers ⋮ AdaBoost and robust one-bit compressed sensing ⋮ A Unifying Tutorial on Approximate Message Passing

Uses Software

• AdaBoost.MH

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Greedy function approximation: A gradient boosting machine.
• High dimensional robust M-estimation: asymptotic variance via approximate message passing
• Boosting algorithms: regularization, prediction and model fitting
• On the impact of predictor geometry on the performance on high-dimensional ridge-regularized generalized robust regression estimators
• A new perspective on boosting in linear regression via subgradient optimization and relatives
• Some inequalities for Gaussian processes and applications
• Arcing classifiers. (With discussion)
• Boosting the margin: a new explanation for the effectiveness of voting methods
• Additive logistic regression: a statistical view of boosting. (With discussion and a rejoinder by the authors)
• Empirical margin distributions and bounding the generalization error of combined classifiers
• Rigorous solution of the Gardner problem
• Population theory for boosting ensembles.
• Process consistency for AdaBoost.
• On the Bayes-risk consistency of regularized boosting methods.
• Statistical behavior and consistency of classification methods based on convex risk minimization.
• Boosting a weak learning algorithm by majority
• On the equivalence of weak learnability and linear separability: new relaxations and efficient boosting algorithms
• Surprises in high-dimensional ridgeless least squares interpolation
• The asymptotic distribution of the MLE in high-dimensional logistic models: arbitrary covariance
• The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression
• Just interpolate: kernel ``ridgeless regression can generalize
• Which bridge estimator is the best for variable selection?
• The likelihood ratio test in high-dimensional logistic regression is asymptotically a rescaled Chi-square
• Analysis of boosting algorithms using the smooth margin function
• Boosting for high-dimensional linear models
• Complexities of convex combinations and bounding the generalization error in classification
• Boosting with early stopping: convergence and consistency
• The Rate of Convergence of AdaBoost
• On robust regression with high-dimensional predictors
• On the existence of maximum likelihood estimates in logistic regression models
• Recovering Structured Signals in Noise: Least-Squares Meets Compressed Sensing
• A note on A. Albert and J. A. Anderson's conditions for the existence of maximum likelihood estimates in logistic regression models
• Boosting With theL₂Loss
• Precise Error Analysis of Regularized <inline-formula> <tex-math notation="LaTeX">$M$ </tex-math> </inline-formula>-Estimators in High Dimensions
• 10.1162/1532443041424319
• Boosting in the Presence of Outliers: Adaptive Classification With Nonconvex Loss Functions
• Two Models of Double Descent for Weak Features
• The Generalization Error of Random Features Regression: Precise Asymptotics and the Double Descent Curve
• Benign overfitting in linear regression
• Reconciling modern machine-learning practice and the classical bias–variance trade-off
• A modern maximum-likelihood theory for high-dimensional logistic regression
• Breaking the Curse of Dimensionality with Convex Neural Networks
• The space of interactions in neural network models
• Learning Theory
• Enumeration of Seven-Argument Threshold Functions
• A Unifying Tutorial on Approximate Message Passing
• Prediction Games and Arcing Algorithms
• Soft margins for AdaBoost
• On the existence of linear weak learners and applications to boosting
• Training Neural Networks as Learning Data-adaptive Kernels: Provable Representation and Approximation Benefits

This page was built for publication: A precise high-dimensional asymptotic theory for boosting and minimum-\(\ell_1\)-norm interpolated classifiers