How to Dominate the Historical Average: Li, Li, Lyu & Yu (2025)

What this is. The paper’s core results, the theoretical framework (bias-variance trade-off in OOS forecasting, first-order stochastic dominance theorems), and the method (conservative-slope forecast with parameter A): enough to understand what it found and how, without reading all 31 pages. To replicate or extend it, read the full source at the original.

TL;DR

The paper proposes an OOS equity premium forecast: instead of setting the predictive slope to zero (historical average) or estimating it by OLS, use a small signed constant slope $\delta = \pm1/A$. The sign comes from economic restrictions or inference, while confidence bounds can guide the choice of A. The method has zero slope-estimation variance, matching the historical average, but a lower bias when the population slope is nonzero. Under the stated distribution and slope conditions, Theorems 1-4 establish first-order stochastic dominance over the historical average. Empirically, 15 of 23 predictors generate significantly positive OOS $R^2$ at the 90% level. Goyal, Welch, and Zafirov (2024) confirmed the Goyal and Welch (2008) findings using a larger predictor set. The dividend-to-price ratio achieves an OOS $R^2$ of 2.1% (p = .019) at A = 100, versus an insignificant 0.2% for OLS. Clark and West (2006) show finite-sample estimation noise can make predictive-regression OOS $R^2$ negative under the null of no predictability; the proposed method uses a constant slope to avoid slope-estimation variance.

Core results

Magnitudes and significance are as reported; \*/\*\* = 10%/5%. Locators point into the source PDF.

#	Result	Locator	Magnitude
R1	Method OOS $R^2$ for dp predictor (A=50): statistically significant improvement over HM	Table 3, p. 3111	$R^2 = 3.4\%$, p-value = .044; OLS $R^2 = 0.2\%$, p-value = .477
R2	Method OOS $R^2$ for dp predictor (A=100): gains statistical power as A increases	Table 3, p. 3111	$R^2 = 2.1\%$, p-value = .019; OLS $R^2 = 0.2\%$, p = .477; CT++ $R^2 = 1.8\%$, p = .286
R3	Method OOS $R^2$ for dp predictor (A=500, A=1,000): very conservative slopes still beat HM at 99% significance	Table 3, p. 3111	A=500: $R^2 = 0.5\%$, p=.009; A=1,000: $R^2 = 0.2\%$, p=.008
R4	Across 23 predictors, 15 (8) have positive OOS $R^2$ at 90% (95%) significance	§5 / Internet Appendix C, pp. 3090-3091; conclusion p. 3113	15 of 23 at 90%; 8 of 23 at 95%; OLS and CT generate statistically insignificant $R^2$ for most (Goyal and Welch 2008)
R5	Method first-order stochastically dominates HM for dp predictor: empirical CDF of MSE everywhere above HM CDF	Figure 6, p. 3112	A=100 CDF (MSE) > HM CDF for all MSE thresholds in 40-year rolling windows; confirmed with kernel smoothing in Internet Appendix C.5
R6	Simulations confirm method’s OOS $R^2$ distribution is entirely to the right of zero; OLS can be negative	Figure 3, p. 3105	A=50 peaks near 6-7 (broadest), A=100 near 3-4, A=200 at ~2, A=500 at ~1 (narrowest spike); OLS distribution spans $[-10, +10]$ with nontrivial probability of $R^2 < 0$
R7	Previously published confidence bounds (Campbell and Shiller 1988a) add value to OOS forecasts when used as the predictive slope	§5.4, p. 3112-3113	A=209 (95% lower bound from Campbell and Shiller 1988a) produces positive OOS $R^2$ from 1987 onward; demonstrates prior study estimates are not data mining
R8	Simulated predictive-slope RMSE is lower for the method than HM over the reported population-slope range	Figure 2, p. 3103	For $b_r$ from 0.0479 to 0.2306, A=50 has lower bias and RMSE than HM, with zero slope variance for both; OLS slope standard deviation is almost as large as HM bias
R9	Simulated forecast MSE CDFs show first-order stochastic dominance over HM at 1-, 10-, and 40-year horizons	Figure 4, p. 3106	Method CDF is above HM throughout for A=50 to 500 in the 1-, 10-, and 40-year rolling windows
R10	Annual forecast-error differences are positive on average and less variable for the method than OLS	Figure 5, p. 3109	The kernel density of HM squared error minus method squared error is centered in the positive region; higher A reduces noise
R11	The dp forecast improvement corresponds to a 27% proportional increase in expected return for an investor recognizing predictability	text p. 3111	At A=100, $R^2=2.1\%$ and squared market Sharpe ratio is 0.078; $0.021/0.078=27\%$
R12	Cumulative dp forecast performance is statistically better than HM	text p. 3088	Figure 1 cumulative SSE gain has a reported 99.2% confidence level

Overall (paper’s conclusion). A conservative deterministic predictive slope, often guided by a confidence bound near zero, dominates the historical average under the paper’s stated conditions and empirically produces positive OOS predictability for many of the 23 standard predictors from Goyal and Welch (2008). The method offers a candidate benchmark for time-varying expected-return models.

Theory / model

The paper models the equity premium return as a linear predictive relationship (Equation 1, p. 3092):

$$r = \mu + bx + e, \tag{1}$$

where $b$ is the population predictive slope coefficient of the predictor $x$, $e$ is a residual with zero mean uncorrelated with $x$, $x$ has zero mean, and $\mu$ is the unconditional expected return. The historical average sets the slope on $x$ to zero. The unknown $\mu$ is estimated by the historical average $\hat{\mu}$:

$$\mu = \hat{\mu} + \xi, \tag{2}$$

where $\xi$ has zero mean. Substituting gives (Equation 3, p. 3093):

$$r = \hat{\mu} + bx + \epsilon, \quad \epsilon \equiv e + \xi. \tag{3}$$

The method’s forecast for the next return is $\hat{\mu} + \delta x$ where $\delta = 1/A$ if $b > 0$ and $\delta = -1/A$ if $b < 0$ (Equation 5, p. 3093):

$$\delta = \begin{cases} 1/A & \text{if } b > 0 \\ -1/A & \text{if } b < 0. \end{cases} \tag{5}$$

Theorem 1 (p. 3092): Under conditions that $\delta$ is a constant between 0 and $b$ and the pdf of the error vector $\mathbf{e}$ is strictly decreasing in $\|\mathbf{e}\|$, the forecast $\mu + \delta x$ first-order stochastically dominates the forecast $\mu$ (population mean) for predicting return $r$. That is, applying loss $-\text{MSE}$, the forecast $\mu + \delta x$ gives at least as high a probability of achieving any MSE threshold, and strictly higher probability for some.

Theorem 2 (p. 3093): The forecast $\hat{\mu} + \delta x$ first-order stochastically dominates the historical average $\hat{\mu}$ under the same conditions on $\delta$ and the distribution of $\boldsymbol{\epsilon} = e + \xi$. Since $\xi$ can correlate with $x$ (Stambaugh 1999 bias), the monotonicity condition on the pdf of $\boldsymbol{\epsilon}$ is imposed (the $t$ and normal distributions satisfy this).

Corollary 1 (p. 3093): Under Theorem 2’s assumptions, the MSE using $\hat{\mu} + \delta x$ satisfies:

$$\text{MSE}_{\hat{\mu}+\delta x} \overset{d}{=} \text{MSE}_{\hat{\mu}} - \eta, \quad \eta \geq 0. \tag{4}$$

The MSE improvement $\eta$ is a nonnegative random variable, so the method has weakly lower MSE in expectation and first-order stochastically lower MSE.

Theorem 3 (p. 3095) extends stochastic dominance to the case where the sign of $b$ is inferred with error (probability $p$ correct, $q$ wrong). For a given predictor realization $\mathbf{x}$, the forecast $\mu + dx$ (where $d$ is $\delta$, $-\delta$, or 0 according to the sign inference outcome) first-order stochastically dominates the historical average when $p/q > \max(B_1, B_2)$, where $B_1 > (2b+\delta)/(2b-\delta)$ (approximately 1 when $\delta/b$ is near zero, and 3 at the maximum $\delta/b = 1$). Statistical significance at the 95% level gives $p/q \geq 0.95/0.05 = 19$, well above the cutoff of 3.

Theorem 4 (p. 3097) restates Theorem 3 for the estimated-mean setting where $\mu$ is replaced by $\hat{\mu}$, yielding Corollary 2 (p. 3098, Equation 9):

$$\text{MSE}_{\hat{\mu}+dx} \overset{d}{=} \text{MSE}_{\hat{\mu}} - \eta, \quad \eta \geq 0. \tag{9}$$

OOS MSE decomposition. The expected MSE difference between the historical average and the method is (Equation 10, p. 3098):

$$E\!\left[\text{MSE}_{\hat{\mu}} - \text{MSE}_{\hat{\mu}+\hat{b}x}\right] = \left(b^2 - \text{Bias}(\hat{b})^2 - \text{Variance}(\hat{b})\right)x^2 - 2E\!\left[(\hat{\mu}-\mu)\hat{b}\right]x. \tag{10}$$

The historical average’s slope is zero and therefore biased when the predictor is informative, but it uses no predictive information and has no slope-estimation variance. A regression slope can reduce bias but can inflate the variance term to the point where it dominates, yielding a negative OOS $R^2$. With a deterministic slope and an unbiased estimate of $\mu$, the final term in Equation (10) is zero and the variance of $\hat{b}$ is zero, leaving $(b^2 - \text{Bias}(\hat{b})^2)x^2$, which is positive when the fixed slope lies between 0 and $b$. This is the core intuition: a constant nonzero slope reduces bias relative to the historical average while avoiding OLS slope-estimation variance; OLS slope estimates can also be biased in this setting.

Gradient descent interpretation. The method is a one-step gradient descent update of the historical average toward greater predictability, using the sign (but not the magnitude) of $b$ as the gradient signal and step size $1/A$ (Equation 11, p. 3099):

$$a_1 = a_0 - \gamma f'(a_0), \tag{11}$$

where $a_0 = 0$ (the historical average slope), $a_1 = \delta$, and $\gamma = 1/A$.

Method

The implementation has three steps.

Step 1: Obtain the sign of $b$. Sign can come from (a) economic theory, as in Campbell and Thompson (2008), who restrict the OLS slope to have the theoretically expected sign, or (b) statistical inference: use the confidence interval $[\hat{b}_L, \hat{b}_U]$ for population slope $b$. If $0 < \hat{b}_L$ the slope is significantly positive; if $\hat{b}_U < 0$ it is significantly negative. The method builds on time-series-forecasting (predictive regression) but replaces the OLS slope with a constant.

Step 2: Choose A. A 95% confidence interval for the population slope is (Equation 6, p. 3093):

$$\hat{b}_L \leq b \leq \hat{b}_U. \tag{6}$$

When the estimated slope is significantly positive, this gives the following with 95% probability (Equation 7, p. 3093):

$$0 < \hat{b}_L \leq b. \tag{7}$$

Set $1/A$ to the lower confidence bound to choose the conservative slope. For the dividend-to-price ratio, Campbell and Shiller (1988a) Table 4 give a predictive slope of 0.129 (SE = 0.057); the standardized predictor has a standard deviation of 0.277, so the 95% lower confidence bound for the standardized slope is $(0.129 - 1.96 \times 0.057) \times 0.277 = 0.0048$, implying $A = 209$ (p. 3094).

When sign inference can be wrong, let $p$ be the probability of a correct sign and $q$ the probability of an incorrect sign. The probability condition underlying the sign-uncertainty dominance result is (Equation 8, p. 3095):

$$\frac{p}{q} > \frac{F(c,0)-F(c,-1)}{F(c,1)-F(c,0)} \quad \text{for all } c. \tag{8}$$

Step 3: Standardize the predictor. Predictors are standardized to zero mean and unit variance using a 20-year rolling backward-looking window, so $\delta = 1/A$ measures the effect of a one-standard-deviation change in the predictor on the forecast annual return (p. 3094, 3107).

Simulation design. A VAR(1) for log returns $r$, log dividend-to-price ratio $dp$, and log dividend growth $\Delta d$ following Cochrane (2008) provides the data-generating process (Equations 12-13, p. 3101). Parameters are calibrated to the sample (Table 1, p. 3102): population predictive slope $b_r = 0.143$, $\rho = 0.8911$. The historical average (HM) uses a rolling 20-year window. OOS $R^2$ is computed per Equation 14 (p. 3103):

The VAR system is (Equation 12, p. 3101):

$$\begin{aligned} r_{t+1} &= a_r + b_r dp_t + \epsilon^r_{t+1},\\ dp_{t+1}-\overline{dp} &= \rho(dp_t-\overline{dp}) + \epsilon^{dp}_{t+1},\\ \Delta d_{t+1} &= a_d + b_d dp_t + \epsilon^d_{t+1}. \end{aligned} \tag{12}$$

The Campbell-Shiller return identity is (Equation 13, p. 3101):

$$r_{t+1} = \kappa_0 - \kappa_1 dp_{t+1} + dp_t + \Delta d_{t+1}, \tag{13}$$

where $\kappa_0=\log(1+e^{-\overline{dp}})+\kappa_1\overline{dp}$ and $\kappa_1=e^{-\overline{dp}}/(1+e^{-\overline{dp}})$; together the equations imply $b_r=1-\kappa_1\rho+b_d$.

$$R^2 = \frac{\text{MSE}_{\hat{\mu}} - \text{MSE}_{\hat{\mu}+\delta x}}{\text{MSE}_{\hat{\mu}}}. \tag{14}$$

Empirical specifications

Data. Annual value-weighted CRSP market returns (post-1926) and S&P 500 Index returns (pre-1926), 23 predictors from Goyal and Welch (2008) extended through 2017 (Table 2, p. 3105): bm, cape Shiller, cay, corpr, csp, de, dfr, dfy, dp, dy, ep, eqis, ik, infl, ltr, lty, tbl, tms, ntis, svar, and four Robert Shiller series. Predictors are standardized using rolling 20-year windows. Forecasts start 20 years after the sample start year for each predictor.

OOS $R^2$ evaluation (R1-R4, R7). For each predictor, compute annual one-step-ahead OOS forecasts using the method with $A \in \{50, 100, 200, 500, 1000\}$, OLS, CT+, and CT++ (Campbell and Thompson 2008). Compute $R^2$ via Equation (14). Report one-sided p-values using Diebold (2015) heteroscedasticity-adjusted test, cross-checked by Harvey, Leybourne, and Newbold (1997). For dp, the raw predictor sample spans 1872-2017 (146 annual observations); forecast evaluation begins after the initial 20-year estimation window.

The main predictive regression is Equation (1), $r_{t+1}=\mu+b x_t+e_{t+1}$, estimated from annual return and predictor observations. The paper uses the 20-year rolling window for the historical mean, predictor mean and standard deviation, and OLS comparator; the alternative method then fixes the slope at the sign-restricted constant $\delta=\pm1/A$, rather than estimating its magnitude each period. Predictors and returns are annual, while the dp raw predictor sample contains 146 annual observations; the forecast evaluation starts after the initial 20-year estimation window. This is a univariate time-series design with no fixed effects. For Table 3, the one-sided Diebold (2015) p-values use standard errors adjusted for return heteroscedasticity; the Harvey, Leybourne, and Newbold (1997) modified test gives similar results (Table 3 note, p. 3110).

Stochastic dominance evaluation (R5). Estimate the empirical CDF of OOS MSE over 40-year rolling windows of annual successive one-year-ahead forecasts using dp predictor (Figure 6, p. 3112). Compare the method CDF (A=100) to the historical mean CDF. First-order dominance requires the method CDF to lie everywhere above (to the left of) the HM CDF. Kernel smoothing (Internet Appendix C.5) confirms the finding.

Bias-variance simulation (R6). Generate simulation samples of the VAR in Equations (12)-(13) with parameters from Table 1. In each sample compute the OOS $R^2$ using a rolling 20-year window for HM and a rolling 20-year window for $\delta = 1/A$ for the method. Figure 3 (p. 3105) reports the kernel density of the $R^2$ distribution across simulations for $A \in \{50, 100, 200, 500\}$ and OLS.

Datasets used

Dataset	Role in paper	Wiki page
CRSP value-weighted index return	Annual market return post-1926	WRDS / CRSP (licensed)
S&P 500 Index returns (Shiller website)	Annual market return pre-1926	Shiller data
Goyal and Welch (2008) predictor data (Amit Goyal’s website)	19 predictor series for equity premium forecasting, extended to 2017	No page yet
Robert Shiller data (http://www.econ.yale.edu/~shiller/data.htm)	4 additional predictors: cape Shiller, infl Shiller, lty Shiller, Trcape Shiller	Shiller data

Sample: 23 predictors with start years ranging from 1872 to 1947 (Table 2, p. 3105), all ending 2017. Forecasting starts 20 years after the predictor start year. Frequency: annual.

When to read the full paper

Use the original if you are: (a) constructing a competing equity premium forecast and need the formal ex ante dominance proofs for a given distributon assumption; (b) selecting the parameter A for a specific predictor using the confidence-bound rule (Internet Appendix C.1 contains calculations for all 23 predictors); (c) investigating whether previously published coefficient estimates add OOS value (Section 5.4); or (d) comparing to shrinkage estimators such as Ridge and Lasso (Internet Appendix C.6). The key tables are Table 3 (OOS $R^2$ for dp) and Internet Appendix Table C.1 (all predictors); key figures are Figure 1 (cumulative OOS performance), Figure 3 (simulation $R^2$ density), and Figure 6 (stochastic dominance CDF).

Attribution and rights

Source: peer-reviewed, The Review of Financial Studies 38(10). This distillation was updated by an LLM on 2026-10-04 and is not human-verified or independently reproduced. The CC BY-NC-ND 4.0 licence permits non-commercial reproduction with attribution and no derivatives; the verbatim PDF is not hosted here.

Li, Kai, Yingying Li, Changlei Lyu, and Jialin Yu. “How to Dominate the Historical Average.” The Review of Financial Studies 38, no. 10 (2025): 3086-3116. DOI: 10.1093/rfs/hhaf010. Replication code: Harvard Dataverse, https://doi.org/10.7910/DVN/9VNJUN. Licensed under CC BY-NC-ND 4.0. This page is a distilled extract by the Institute for Automated Research.