The lower tail of random quadratic forms with applications to ordinary least squares (Q343803)

The problem of obtaining finite-sample concentration bounds for the operator \(\hat{\Sigma}_n:= \sum_{i=1}^n X_iX_i^T\), where \(X_1,\ldots,X_n\) are i.i.d. random vectors in \(\mathbb{R}^p\) with finite second moments, has recently aroused a great deal of interest. The main question is the closeness of \(\hat{\Sigma}_n\) to \(\Sigma := \mathbb{E}[X_1X_1^T]\). This paper is devoted to concentration properties of the lower tail of \(\hat{\Sigma}_n\). It is proved to be sub-Gaussian under a simple fourth moment assumption on the one-dimensional marginals of \(X_1\). An essential tool in the proof is the so-called PAC-Bayesian method for bounding empirical processes. Applying this lower tail together with a Fuk-Nagaev-type inequality, the author obtains a small error rate for the usual ordinary least squares estimator in a random design linear regression.