Comparison of regularized discriminant analysis with the standard discrimination methods (Q1297877)

The objective of the classical discriminant analysis problem is to classify a \(p\)-variate observation \(x\) as coming from one of \(K\) groups or classes. It is usually assumed that the measurements from each group follow a multi-normal distribution. Let us denote the population mean and covariance matrix of group \(i\) by \(\mu_{i}\) and \(\Sigma_{i}\), respectively, \(i=1,\ldots,K\). These parameters are estimated by the sample mean \(\bar x_{i}\) and covariance matrix \(S_{i}\), using a training sample of size \(n_{i}\). Let us denote \(S_{p}=N^{-1}\sum_{i=1}^{K}n_{i}S_{i}\), \(N=\sum_{i=1}^{K}n_{i}\). The sample regularized discriminant function (SRDF) rules allocate an object to group \(k\) as follows: \[ \min_{i}\{(x-\bar x_{i})' \hat\Sigma_{i}^{-1}(\lambda,\gamma)(x-\bar x_{i})+ {\text{ln}}|\hat\Sigma_{i}(\lambda,\gamma)|-2{\text{ln}}\pi_{i}\}, \] where \(\hat\Sigma_{i}(\lambda,\gamma)\), the estimate of \(\Sigma_{i}\), is given by \[ \hat\Sigma_{i}(\lambda,\gamma)=(1-\gamma)\hat\Sigma_{i}(\lambda)+\gamma p^{-1} \text{Tr}\{\hat\Sigma_{i}(\lambda)\}I, \] where \(I\) is the identity matrix and \[ \hat\Sigma_{i}(\lambda)=[(1-\lambda)(n_{i}-1)S_{i}+\lambda S_{p}]/ [(1-\lambda)(n_{i}-1)+\lambda(N-k)],\quad 0\leq \lambda\leq 1. \] The article deals with a critical comparison of the standard approaches with SRDF and its implementation difficulties.