On the distribution, model selection properties and uniqueness of the Lasso estimator in low and high dimensions (Q2180053)

Consider the linear model \(y=X\beta+\varepsilon\), where \(y\) is an \(n\times1\) vector of observations, \(X\) is an \(n\times p\) regressor matrix, \(\beta\in\mathbb{R}^p\) is an unknown parameter vector, and \(\varepsilon\) is a normally distributed error term. In this setting, the weighted Lasso estimator is the solution to the minimization problem \[ \min_{\beta\in\mathbb{R}^p}\left\{\|y-X\beta\|^2+2\sum_{j=1}^p\lambda_{n,j}|\beta_j|\right\}\,, \] where the \(\lambda_{n,j}\geq0\) are specified weights. The authors investigate properties of this estimator in both the low-dimensional (\(p\leq n\)) and high-dimensional (\(p>n\)) settings. In the low-dimensional case, explicit expressions are given for the distribution function of the Lasso estimator and the corresponding density function conditional on knowing which components of the estimator are non-zero. The relationship between the Lasso estimator and the least-squares estimator is also investigated in terms of shrinkage sets: for any \(b\in\mathbb{R}^p\), this is the set \(S(b)\subseteq\mathbb{R}^p\) such that the Lasso estimator is equal to \(b\) if and only if the least-squares estimator is in \(S(b)\). Explicit expressions for these shrinkage sets are given. In the high-dimensional setting, formulas for the distribution of the Lasso estimator are again given, and selection regions are investigated to relate the Lasso estimator to \(X^\prime y\). For \(b\in\mathbb{R}^p\), these are sets \(T(b)\subseteq\mathbb{R}^p\) such that the Lasso estimator is equal to \(b\) if and only if \(X^\prime y\) lies in \(T(b)\). The structural set is also constructed geometrically; this is the set of covariates that may be included in the Lasso solution for at least some values of \(y\). Finally, a necessary and sufficient condition for uniqueness of the Lasso estimator is also given. The authors make very few assumptions on the regressor matrix \(X\), and many of their results continue to hold if the assumption of normally distributed errors is relaxed.