Outlier removal for prediction of covariance matrices with an application to portfolio optimization (Q2732666)

The author considers the problem of composing a portfolio out of a set of stocks \(s_1,\dots,s_n\) with the weights \(w_i,\;\sum_{i=1}^nw_i=1\). The classical approach to analysis of such a portfolio was proposed by \textit{H.M. Markowitz} [J. Finance 7, 77-91 (1952)] who indicated the relation between the return and the variance of the portfolio. The portfolio return \(R_p\) is the weighted sum of the individual stock returns \(r_1,\dots,r_n\) with the expected values NEWLINE\[NEWLINE\mu_1,...,\mu_n\;:\;R_p=\sum_{i=1}^nw_ir_i,\;ER_p=\sum_{i=1}^nw_i\mu_i.NEWLINE\]NEWLINE Using the matrix notation we get the following expression for the variance \(\sigma^2\) of the portfolio: NEWLINE\[NEWLINE\sigma^2=\sum_{i=1}^n\sum_{j=1}^nw_iw_j\sigma_{ij}=w^TCw,NEWLINE\]NEWLINE where \(C\) is the covariance matrix \(C(i,j)=\sigma_{ij}\) and \(w=(w_1,\dots,w_n)^T\), \(R=(\mu_1,\dots,\mu_n)^T\) are the column vectors. Given \(ER_{p}\) and \(\sigma_{p}^{2}\) the following portfolio optimization problem can be formulated: NEWLINE\[NEWLINE\max\limits_{w}\alpha w^{T}R-w^{T}Cw\;\text{ s.t. } w_i\geq 0,\;i=1,\dots,n;\;\sum_{i=1}^nw_i=1.NEWLINE\]NEWLINE (The risk tolerance factor \(\alpha\) expresses the relative importance of expected portfolio return and variance and is often set to \(0.5.\)) This standard problem of modern portfolio theory can be solved using routines of quadratic programming. The optimization requires the vector with expected stock returns \(R\) and the covariance matrix \(C\) to be estimated. The typical procedure is to compute the sample covariances and returns using historical data. The estimated values for \(R\) and \(C\) are then plugged into the optimization routine which computes the weights \((w_1,\dots,w_n)\) that maximize \(\alpha w^{T}R-w^{T}Cw.\)NEWLINENEWLINENEWLINEThis normal ``naive'' method of using historical data to compute sample covariances and returns results in sub-optimal portfolios since the price generating process is time varying and also since the sample covariance is known to be sensitive to outliers in data. A lot of alternative methods for estimation and prediction of covariance matrices have been suggested in the literature. For a survey and examples of common techniques see, for example, \textit{Y. Ma} and \textit{M.G. Genton} [Highly robust estimation of dispersion matrices. Tech. Rep., Dpt. Math., 2-390, Massachusetts Inst. Technol. (1999)], and \textit{O. Ledoit} [Improved estimation of the covariance matrix of stock returns with an application to portfolio selection. Tech. Rep., Anderson Grad. School Management at UCLA, (1999)].NEWLINENEWLINENEWLINEIn this paper, a simple algorithm to improve the naive prediction with systematic removals of outliers in data is presented. The method is a ``leave-one-out'' approach which has been previously used for outlier detection in regression problems. The algorithm computes a penalty measure for each day depending on the increase or decrease in prediction error the inclusion of the day has given for the previous predictions. The penalty measure is updated at each prediction step by additional predictions where each day in the modelling window is removed in sequence. The algorithm gives a significant reduction in the root of the mean squared error (RMSE) for the covariance matrices when tested on data with deliberately planted outliers and also for real stock data. The algorithm gives higher risk-adjusted returns than the naive prediction even if the reduction of RMSE for the covariances is higher than the gain in portfolio optimization.NEWLINENEWLINENEWLINEThis indicates that outliers in stock data do not affect the computation of optimal portfolios to any significant degree, at least not on the daily scale that is used in the presented tests. However, the algorithm shows a clear ability to detect and eliminate the effect of outliers and could be applied to other time series related modelling problems with outliers in data.