Prediction and estimation consistency of sparse multi-class penalized optimal scoring
type: Item
instance of: scholarly article
title: Prediction and estimation consistency of sparse multi-class penalized optimal scoring
zbMATH Open document ID: 1456.62148
DOI: 10.3150/19-BEJ1126
author: Irina Gaynanova
published in: Bernoulli
publication date: 2019-12-05T00:00:00Z
full work available at URL: https://arxiv.org/abs/1809.04669
full work available at URL: https://projecteuclid.org/euclid.bj/1574758829
review text: The results on the consistency of group-Lasso estimators in linear regression cannot be applied to the penalized optimal scoring problem. The key difficulty is that linear model for the multiple response \(Y\) does not hold, and the dependency between the random covariates in \(X\) and the residual terms in \(E=Y-XB.\) The paper under review overcomes these challenges by using sub-exponential concentration bounds, and exploiting the decomposition of marginal covariance matrix. Main contributions are as follows. 1. A coordinate-sparse multi-class optimal scoring problem is formulated as the penalized multiple response linear regression problem. 2. The concentration bound is derived for the maximal row \(l_2\) norm of \(n^{-1}X^\top E\). Here \(X\) and \(E\) could be non-Gaussian and/or correlated. 3. Out-of-sample prediction and estimation bounds are derived for sparse multi-class optimal scoring problem which allow both the number of features and the number of classes to grow with the sample size. Bounds of two types are obtained: (a) slow-rate bounds which make no assumptions on the correlation structure of \(X\) or the sparsity of \(B\), and (b) fast-rate bounds which lead to faster convergence rates, but rely on exact sparsity of \(B\) and restricted eigenvalue condition. 4. It is shown that out-of-sample prediction consistency implies classification consistency, and finite-sample bounds are derived on misclassification error rate of multi-class penalized optimal scoring problem.
Mathematics Subject Classification ID: 62J07
Mathematics Subject Classification ID: 62H30
Mathematics Subject Classification ID: 62G08
Mathematics Subject Classification ID: 62J05
zbMATH DE Number: 7140500
zbMATH Keywords: classification
zbMATH Keywords: high-dimensional regression
zbMATH Keywords: Lasso
zbMATH Keywords: linear discriminant analysis
reviewed by: A. Kukush
describes a project that uses: penalizedLDA
describes a project that uses: GSPPCA
MaRDI profile type: Publication
arXiv ID: 1809.04669
cites work: Simultaneous analysis of Lasso and Dantzig selector
cites work: Prediction error bounds for linear regression with the TREX
cites work: Bayesian variable selection for globally sparse probabilistic PCA
cites work: Statistics for high-dimensional data. Methods, theory and applications.
cites work: Sparsity oracle inequalities for the Lasso
cites work: A Direct Estimation Approach to Sparse Linear Discriminant Analysis
cites work: On the prediction performance of the Lasso
cites work: Optimal variable selection in multi-group sparse discriminant analysis
cites work: Sparse quadratic classification rules via linear dimension reduction
cites work: Persistene in high-dimensional linear predictor-selection and the virtue of overparametrization
cites work: Q5405260
cites work: Penalized discriminant analysis
cites work: Flexible Discriminant Analysis by Optimal Scoring
cites work: Q5251797
cites work: A tail inequality for quadratic forms of subgaussian random vectors
cites work: Optimal Feature Selection in High-Dimensional Discriminant Analysis
cites work: Oracle inequalities for high-dimensional prediction
cites work: L1 least squares for sparse high-dimensional LDA
cites work: Oracle inequalities and optimal inference under group sparsity
cites work: A direct approach to sparse discriminant analysis in ultra-high dimensions
cites work: Q3870155
cites work: Q4827938
cites work: A unified framework for high-dimensional analysis of \(M\)-estimators with decomposable regularizers
cites work: A unified theory of confidence regions and testing for high-dimensional estimating equations
cites work: Support union recovery in high-dimensional multivariate regression
cites work: Covariate-Adjusted Tensor Classification in High-Dimensions
cites work: Restricted eigenvalue properties for correlated Gaussian designs
cites work: Exponential screening and optimal rates of sparse estimation
cites work: Reconstruction From Anisotropic Random Measurements
cites work: Q4864293
cites work: On the conditions used to prove oracle results for the Lasso
cites work: Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$-Constrained Quadratic Programming (Lasso)
cites work: Penalized Classification using Fisher’s Linear Discriminant
cites work: Model Selection and Estimation in Regression with Grouped Variables
cites work: Some sharp performance bounds for least squares regression with \(L_1\) regularization
cites work: Q3174050
OpenAlex ID: W2990661983
Recommended article: Erratum to: ``Prediction and estimation consistency of sparse multi-class penalized optimal scoring''
Recommended article: Robust and sparse multigroup classification by the optimal scoring approach
Recommended article: Proximal Methods for Sparse Optimal Scoring and Discriminant Analysis
Recommended article: Multiclass Classification by Sparse Multinomial Logistic Regression
Recommended article: Sparse classification: a scalable discrete optimization perspective
Recommended article: Q3112592
Recommended article: A penalized optimization method for classification
Recommended article: Binary Credal Classification Under Sparsity Constraints
description: scientific article