RegimeFactorZoo, part three: reproducing the estimator is not proving the extension

Abstract

RegimeFactorZoo now reproduces the continuous spike-and-slab estimator from Bryzgalova, Huang and Julliard (2023), using the authors' 51-factor, 60-portfolio replication setup. At prior Sharpe ratio 2, the Bayesian model-averaged risk prices agree with the published output to approximately $10^{-14}$ maximum absolute difference. That is a replication check under a particular code and random-seed convention, not evidence that the proposed volatility-regime extension works. The matched-size permutation comparison for that extension remains pending.

The replication has a satisfying number. The research question still has homework.

1. What changed since the previous post

The earlier results post covered SMB/HML reconstruction, Fama-MacBeth pricing, and a small characteristic panel. Its proposed next step drew on In Search of Sparsity. The current implemented replication is a different estimator: the continuous spike-and-slab approach in Bayesian Solutions for the Factor Zoo.

That distinction matters. Sparse loadings on latent factors, penalized regressions, and Bayesian selection over candidate pricing factors are related questions, not interchangeable labels for the same method.

2. What the numerical agreement means

For a factor $j$, its posterior inclusion probability can be written schematically as

$$\operatorname{PIP}_j =\Pr(\gamma_j=1\mid\mathcal D),$$

where $\gamma_j$ is its inclusion indicator. Bayesian model averaging combines uncertainty over model specifications instead of selecting one model and pretending the selection was certain.

The local replication checks the authors' output at a fixed prior setting, including 52 risk-price entries: the intercept and 51 factors. The maximum difference is approximately machine precision. The exact replication also retains the archive's within-loop random-seed convention; that convention is recorded, not silently replaced.

Matching a published computation establishes that I can reproduce that computation. It does not establish robustness to every prior, an independently validated sampler, or new predictive performance.

3. Why a regime comparison needs a null

The extension labels months using trailing realized market volatility and an expanding-percentile threshold. That avoids using future volatility to decide which regime a historical month belonged to. The estimator is then run separately across the two groups.

But unequal sample sizes alone can change inclusion probabilities. A low correlation between two estimated PIP vectors is therefore not enough. One possible discrepancy statistic is

$$D=1-\operatorname{corr} \left(\widehat{\boldsymbol{\pi}}_{\mathrm{high}}, \widehat{\boldsymbol{\pi}}_{\mathrm{low}}\right).$$

To interpret $D$, I need a comparison preserving relevant sample-size and dependence constraints. A matched-size random-split or permutation experiment is a starting point, with its exchangeability assumptions made explicit. Time dependence may require a more structured resampling design. The current matched-size null has not been run, so I am not presenting the exploratory regime differences as an established finding.

4. Reproducibility and access

The public repository contains the replication note and code. Public code does not imply unrestricted raw data: the earlier CRSP/Compustat construction still requires licensed access, while the Bayesian replication uses the authors' archive. No raw proprietary market-data download is bundled into the repository.

The next milestone is not another model in the comparison table. It is a defensible null experiment, followed by a conclusion that may be smaller than the original idea. That is still progress.

References

1. Bryzgalova, S., Huang, J., and Julliard, C. (2023). Bayesian Solutions for the Factor Zoo: We Just Ran Two Quadrillion Models. The Journal of Finance, 78(1), 487-557.
2. Replication note and implementation details.
3. Current project status and extension limitations.