[Paper Reading] In-sample and Out-of-sample Sharpe Ratios of Multi-factor Asset Pricing Models
Title: In-sample and Out-of-sample Sharpe Ratios of Multi-factor Asset Pricing Models
Author(s): Raymond Kan*, Xiaolu Wang, Xinghua Zheng
Year: 2019, visited the version revised on 22 Feb 2021
Definitions
- Optimal portfolio weight: For a mean-variance investor who wants to hold a portfolio with a target standard deviation of $\sigma$, then their optimal portfolio has weights of
w^* = \frac{\sigma}{\theta} \Sigma^{-1} \mu,
$$
where $\mu = \mathbb{E}[r_t]$, $\Sigma = Var[r_t]$, and $r_t$ is the return of risky assets in excess of risk-free rate. $\theta$ is the Sharpe ratio.
\hat{\theta} = \sqrt{\hat{\mu}' \hat{\Sigma}^{-1} \hat{\mu}},
$$
- Sharpe Ratio: $\theta = \sqrt{\mu' \Sigma^{-1}\mu}$. In practice, the investor does not know the mean and covariance matrix of the factors; he has to estimate $\theta$ using historical data. Thus, we have the in-sample and out-of-sample Sharpe ratio as follows.
- In-sample Sharpe ratio ($\hat{\theta}$)
\hat{\theta} = \sqrt{\hat{\mu}' \hat{\Sigma}^{-1} \hat{\mu}},
$$
where $\hat{\mu}$ and $\hat{\Sigma}$ are the sample estimators of $\mu$ and $\Sigma$, and they are given by
$$\hat{\mu} = \frac{1}{T} \sum_{t=1}^T r_t,
$$
$$
\hat{\Sigma} = \frac{1}{T} \sum_{t=1}^T (r_t - \hat{\mu}) (r_t - \hat{\mu})'.
$$
The in-sample Sharpe ratio is a post-measure of performance, and it is unattainable by investors.
$$
\hat{w} = \frac{\sigma}{\hat{\theta}} \hat{\Sigma}^{-1} \hat{\mu}.
$$
Thus, the out-of-sample mean and variance of this estimated optimal portfolio are
$$
\hat{w}'\mu = \frac{\sigma}{\hat{\theta}} \hat{\mu}' \hat{\Sigma}^{-1} \hat{\mu},
$$
$$
\hat{w}' \Sigma \hat{w} = \frac{\sigma^2}{\hat{\theta}^2} \hat{\mu}' \hat{\Sigma}^{-1} \Sigma \hat{\Sigma}^{-1} \hat{\mu}.
$$
- Out-of-sample Sharpe ratio ($\tilde{\theta}$)
\hat{w} = \frac{\sigma}{\hat{\theta}} \hat{\Sigma}^{-1} \hat{\mu}.
$$
Thus, the out-of-sample mean and variance of this estimated optimal portfolio are
$$
\hat{w}'\mu = \frac{\sigma}{\hat{\theta}} \hat{\mu}' \hat{\Sigma}^{-1} \hat{\mu},
$$
$$
\hat{w}' \Sigma \hat{w} = \frac{\sigma^2}{\hat{\theta}^2} \hat{\mu}' \hat{\Sigma}^{-1} \Sigma \hat{\Sigma}^{-1} \hat{\mu}.
$$
The out-of-sample Sharpe ratio of the estimated optimal portfolio is then given by
$$\tilde{\theta} = \frac{\hat{w}'\mu}{\sqrt{\hat{w}' \Sigma \hat{w}}} = \frac{ \hat{\mu}' \hat{\Sigma}^{-1} \hat{\mu}}{(\hat{\mu}' \hat{\Sigma}^{-1} \Sigma \hat{\Sigma}^{-1} \hat{\mu})^{\frac{1}{2}}}
$$
Unlike $\hat{\theta}$ or $\theta$, $\tilde{\theta}$ is what an investor can actually obtain out-of-sample by holding the sample optimal portfolio $\hat{w}$.
- Estimation risk: the estimation risk is the difference between the in-sample Sharpe ratio and out-of-sample Sharpe ratio.
Motivation
Many multi-factor asset pricing models proposed in the recent literature have substantially higher sample Sharpe ratios than that of the value-weighted market portfolio. In contrast, such a high sample Sharpe ratio is rarely delivered by professional fund managers. 
Mutual Fund Performance Relative to In-sample Sharpe Ratios of Various Asset Pricing Models
Mutual Fund Performance Relative to Out-of-sample Shape Ratios of Various Asset Pricing Models
|
| Mutual Fund Performance Relative to In-sample Sharpe Ratios of Various Asset Pricing Models |
|
| Mutual Fund Performance Relative to Out-of-sample Shape Ratios of Various Asset Pricing Models |
Why Interesting
- Why is there a discrepancy between the sample Sharpe ratio of multi-factor asset pricing models proposed in academia and the actual performance of professional fund managers? Can't the fund managers just adopt the best-performed models in literature?
- The out-of-sample Sharpe ratio is the fundamental and evaluation metric of the asset pricing model that we all should understand, or we cannot know whether the models work in practice.
- Why is there a discrepancy between the sample Sharpe ratio of multi-factor asset pricing models proposed in academia and the actual performance of professional fund managers? Can't the fund managers just adopt the best-performed models in literature?
- The out-of-sample Sharpe ratio is the fundamental and evaluation metric of the asset pricing model that we all should understand, or we cannot know whether the models work in practice.
Contributions
- Provided the finite sample and asymptotic analyses of the joint distribution of in-sample and out-of-sample Sharpe ratios of multi-factor asset pricing models. This estimation risk can explain why the high sample Sharpe ratios of asset pricing models are difficult to realize in reality.
- Empirically examined that the out-of-sample Sharpe ratio of an asset pricing model is substantially worse than its in-sample Sharpe ratio.
- Their analysis suggested that many newly proposed asset pricing models do not provide superior out-of-sample performance than the value-weighted market portfolio.
- Provided the finite sample and asymptotic analyses of the joint distribution of in-sample and out-of-sample Sharpe ratios of multi-factor asset pricing models. This estimation risk can explain why the high sample Sharpe ratios of asset pricing models are difficult to realize in reality.
- Empirically examined that the out-of-sample Sharpe ratio of an asset pricing model is substantially worse than its in-sample Sharpe ratio.
- Their analysis suggested that many newly proposed asset pricing models do not provide superior out-of-sample performance than the value-weighted market portfolio.
Finite Sample Distribution of Sharpe Ratio
The authors consider a multi-factor asset pricing model with $N$ traded factors and back-tested for $T$ trading periods.
Proposition 1: Stochastic Representation of $(\hat{\theta},\tilde{\theta})$
$$
\hat{\theta} \stackrel{\text{d}}{=} \frac{\sqrt{\tilde{z}^2+\tilde{u}}}{\sqrt{u_1}},
$$
$$
\tilde{\theta} \stackrel{\text{d}}{=} \frac{\theta\tilde{z}}{\sqrt{\tilde{z}^2+\tilde{u}}}
$$
where $\tilde{z}\sim \mathcal{N}(\sqrt{b}\sqrt{T}\theta, 1)$, and $\tilde{u} \sim \chi_{N-1}^2((1-b)T\theta^2)$.
\hat{\theta} \stackrel{\text{d}}{=} \frac{\sqrt{\tilde{z}^2+\tilde{u}}}{\sqrt{u_1}},
$$
$$
\tilde{\theta} \stackrel{\text{d}}{=} \frac{\theta\tilde{z}}{\sqrt{\tilde{z}^2+\tilde{u}}}
$$
where $\tilde{z}\sim \mathcal{N}(\sqrt{b}\sqrt{T}\theta, 1)$, and $\tilde{u} \sim \chi_{N-1}^2((1-b)T\theta^2)$.
The stochastic representation of $(\hat{\theta},\tilde{\theta})$ enables us to do many things.
- Obtain the exact moments and the joint moments of $\hat{\theta}$ and $\tilde{\theta}$ (Lemma 1)
- Prove the important inequalities on $\mathbb{E}[\hat{\theta}]$, $\mathbb{E}[\tilde{\theta}]$ and Cov$[\hat{\theta},\tilde{\theta}]$ (Lemma 2)
\mathbb{E}[\tilde{\theta}] < \theta < \mathbb{E}[\hat{\theta}],
$$
$$
\text{Cov}[\hat{\theta},\tilde{\theta}] > 0.
$$
- The exact joint distribution of $(\hat{\theta},\tilde{\theta})$ and marginal distributions of $\hat{\theta}$ and $\tilde{\theta}$ (Proposition 2)
Figure 1: Ratio of Expected Out-of-sample Sharpe Ratio to Expected In-sample Sharpe Ratio of an Asset Pricing Model
 |
| Ratio of Expected Out-of-sample Sharpe Ratio to Expected In-sample Sharpe Ratio of an Asset Pricing Model |
The figure plots $\mathbb{E}[\tilde{\theta}]/\mathbb{E}[\hat{\theta}]$ as a function of $T$ for various values of $N$ and $\theta$.
- In-sample Sharpe ratio does not give a reliable prediction of what an investor can expect.
- For higher $\theta$ and smaller $N$, the signal-to-noise ratio is higher and hence $\mathbb{E}[\tilde{\theta}]/\mathbb{E}[\hat{\theta}]$ is higher. However, the loss due to estimation risk is still quite significant, especially when the length of time series is short.
Figure 2: Correlation between In-sample and Out-of-sample Sharpe Ratios of an Asset Pricing Model
 |
Correlation between In-sample and Out-of-sample Sharpe Ratios of an Asset Pricing Model |
The figure plots the correlation coefficient between the in-sample and out-of-sample Sharpe ratios as a function of $N$ and $T$. As seen from the figure, $\hat{\theta}$ and $\tilde{\theta}$ are positively correlated, which indicates the measure of past performance ($\hat{\theta}$) can predict future performance ($\tilde{\theta}$).
Figure 7: Expected Conditional Out-of-sample Sharpe Ratio of an Asset Pricing Model
 |
| Expected Conditional Out-of-sample Sharpe Ratio of an Asset Pricing Model ($T=120$) |
The figure plots the expected conditional normalized out-of-sample Sharpe ratio ($\mathbb{E[\tilde{\theta}|\hat{\theta}]/\theta}$) of an asset pricing model as a function of normalized in-sample Sharpe ratio ($\hat{\theta}/\theta$).
- $\mathbb{E[\tilde{\theta}|\hat{\theta}]/\theta}$ is a monotonic increasing function of $\hat{\theta}/\theta$. It suggests that keeping $N$ and $\theta$ constant, one would prefer an asset pricing model that has a higher in-sample Sharpe ratio.
- The relation between $\mathbb{E[\tilde{\theta}|\hat{\theta}]/\theta}$ and $\hat{\theta}/\theta$ depends on $N$ and $\theta$.
Asymptotic Distributions of Sharpe Ratio
In this section, the authors present the asymptotic(/limiting) distributions of $(\hat{\theta},\tilde{\theta})$, and evaluate the accuracy of the limiting distribution using the finite sample results from the last section. Here, they consider two different asymptotic distributions for $(\hat{\theta},\tilde{\theta})$ depending on whether $N$ is fixed or $N \to \infty$ as $T \to \infty$.Proposition 3: Asymptotic Distribution when $N$ is Fixed
Suppose $r_t \stackrel{\text{i.i.d}}{\sim} \mathcal{N}(\mu, \Sigma)$ and $N \leq 2$. When $N$ is fixed and $T \to \infty$, we have
$$\begin{bmatrix} \sqrt{T} (\hat{\theta} - \theta) \\ T(\tilde{\theta} - \theta) \end{bmatrix} \stackrel{\text{d}}{\to} \begin{bmatrix} X \\ Y \end{bmatrix},
$$
where $X \sim \mathcal{N}(0,1+\frac{\theta^2}{2})$, $Y \sim -(1+\theta^2)/(2\theta) \chi_{N-1}^2$, and they are independent of each other.
Figure 9 plots the exact density of $\tilde{\theta}/\theta$ vs. the other two approximations like figure 8. It shows unless $N$ is very small, both asymptotic approximations do not provide reliable approximations of the exact density of $\tilde{\theta}$. Thus, if one wants to draw inference on $\tilde{\theta}$, they would better use the finite sample distribution of $\tilde{\theta}$.
Implications:
- The limiting distribution of $\sqrt{T} (\hat{\theta} - \theta)$ has mean zero, which suggests that $\hat{\theta}$ converges to $\theta$ at a rate of $1/\sqrt{T}$.
- Unlike $\hat{\theta}$ which converges to $\theta$ at a rate of $1/\sqrt{T}$, $\tilde{\theta}$ converges to $\theta$ at a rate of $1/T$.
- The limiting distribution of $T(\tilde{\theta} - \theta)$ is not a normal distribution, but instead it is a negative random variable which is proportional to $\chi_{N-1}^2$. This is consistent with findings in the finite sample distribution such that $\tilde{\theta}$ is always less than $\theta$.
Figure 8: Exact and Approximate Densities of In-sample Sharpe Ratio of an Asset Pricing Model
 |
| Exact and Approximate Densities of In-sample Sharpe Ratio of an Asset Pricing Model ($T=120$) |
This figure plots the exact density of $\hat{\theta}/\theta$ (solid line) vs. its two approximations, one is based on the fixed $N$ asymptotic (dashed line), and the other is based on the $N/T \to \rho$ asymptotic (dotted line). It can be seen that the approximate distribution based on the traditional fixed $N$ asymptotic does not perform well in all cases. This is because $\hat{\theta}$ has a bias, which the fixed $N$ asymptotic distribution of $\hat{\theta}$ ignores. In contrast, the approximation based on the fixed $N/T$ asymptotic works very well in all cases in approximating the exact distribution of $\hat{\theta}$. Therefore, if one would like to use an asymptotic distribution to approximate the exact distribution of $\hat{\theta}$, s/he would better use the fixed $N/T$ asymptotic one, even when $N$ is small and $T$ is large.
Figure 9: Exact and Approximate Densities of Out-of-sample SR of an Asset Pricing Model
 |
| Exact and Approximate Densities of Out-of-sample SR of an Asset Pricing Model ($T = 120$) |
Figure 9 plots the exact density of $\tilde{\theta}/\theta$ vs. the other two approximations like figure 8. It shows unless $N$ is very small, both asymptotic approximations do not provide reliable approximations of the exact density of $\tilde{\theta}$. Thus, if one wants to draw inference on $\tilde{\theta}$, they would better use the finite sample distribution of $\tilde{\theta}$.
Statistical Inference on Sharpe Ratio
This section answers the question that, given an asset pricing model and some historical data on its factor returns, how should an investor forecast the distribution of the out-of-sample Sharpe ratio of the sample optimal portfolio in the following period.
The authors construct a confidence interval for $\mathbb{E}[\tilde{\theta}|\hat{\theta}]$. For example, for $h=240$ months, HMXZ $q^5$ has the highest in-sample Sharpe ratio of 0.469 but the 95% confidence interval for its $\theta$ is $(0.308,0.501)$, which is quite wide.
 |
| Confidence Intervals of Population and Conditional Distribution of Out-of-sample SR |
As a result, the authors do not find strong evidence that the recent multi-factor models can deliver superior out-of-sample performance than value-weighted market portfolio, except for the most recent HMXZ $q^5$ model.
Place of Improvement
The rationales of all the assumptions made when deriving the stochastic representations of $(\hat{\theta},\tilde{\theta})$ are not clear to the readers.
Comments
Post a Comment