← All notes

Gradient boosting for volatility: where XGBoost wins and where it does not

Boosted trees are the strongest general-purpose model for tabular data, and volatility forecasting is a tabular problem. That is the case for. The case against is that trees cannot extrapolate, and volatility spikes are extrapolation.

What boosting is doing

Gradient boosting fits an additive ensemble of shallow trees, each new tree fitted to the gradient of the loss with respect to the current prediction. XGBoost's contribution was to use a second-order expansion of the loss and to put explicit regularisation on tree complexity and leaf weights, which is why it behaves so much better than naive boosting on noisy data. LightGBM grows leaf-wise with histogram binning, which is faster on wide feature sets and, with default settings, somewhat more eager to overfit.

The property that matters here: a tree predicts a constant within each leaf. The model's output is therefore bounded by the range of targets observed during training. It cannot produce a number it has never seen.

Where it wins: breadth of information

HAR uses three numbers from one series. Boosting shines when the honest answer to "what else might predict tomorrow's volatility?" is a long list, because it handles many correlated predictors, nonlinearity and interactions without any of them being specified in advance. A feature set worth building:

• Own history. Log RV at the HAR horizons, plus the continuous and jump components separately.
• Asymmetry. Signed return, semivariances, and the leverage interaction between a negative return and current volatility level.
• Forward-looking. Implied volatility, the variance risk premium, the term-structure slope of implied vol. These carry information no backward-looking measure has.
• Cross-sectional. Realized volatility of related assets and indices. Spillover is real and HAR ignores it entirely.
• Microstructure. Spreads, volume, order-flow imbalance, intraday range measures.
• Calendar. Day of week, expiry and roll dates, scheduled macro releases, earnings. Cheap, and trees use them well.

Three modelling choices repeatedly turn out to matter more than hyperparameters:

1. Predict log RV. The raw target is right-skewed and heteroskedastic; squared error on it puts nearly all the gradient on crisis days. Logs fix this. Add the variance correction when inverting.
2. Predict the residual against HAR. Let OLS handle the linear persistence it already captures perfectly, and give boosting only the part that is left. This preserves HAR's extrapolation behaviour while keeping the nonlinear gains — usually the single biggest improvement available.
3. Tune conservatively. Shallow trees (depth three to six), small learning rate with early stopping on a time-ordered validation split, and subsampling. Financial data has a low signal-to-noise ratio; capacity is rarely the binding constraint.

Where it loses

Related weaknesses worth stating plainly:

• Non-stationarity. Learned splits encode the level of the regime they were fitted in. Refit on a rolling window and accept that some of the model's knowledge has an expiry date.
• Sample size. Twenty years of daily data is roughly five thousand rows, with strong autocorrelation reducing the effective count further. That is a small dataset by machine learning standards.
• Attribution is not explanation. SHAP values tell you what the model used. They say nothing about causality, and reading them as economics is a common and expensive error.

The leakage traps

Almost every spectacular volatility backtest is a leakage bug. The usual suspects:

• Random cross-validation. K-fold on time series trains on the future. Use a forward-chaining or purged split with an embargo around the test window.
• Global scaling. Standardising features using the full sample's mean and standard deviation leaks future distribution information. Fit scalers inside the training fold only.
• Overlapping targets. A twenty-two-day-ahead target overlaps across consecutive rows. Without purging, near-identical observations appear in both train and test.
• Revision and publication lags. Use the value that was available at the decision time, not the revised series you downloaded today.
• Tuning on the test set. Selecting hyperparameters by test performance makes the reported number an in-sample number.

An honest summary

With HAR's own inputs, XGBoost roughly ties HAR and costs far more to maintain. With a genuinely wider feature set — implied volatility, cross-asset, order flow — it produces a real and repeatable improvement, typically largest at short horizons and smallest at long ones. Fitted as a residual correction on top of HAR rather than as a replacement, it keeps the linear model's extrapolation behaviour and adds the nonlinear structure. That hybrid is where we have found the reliable gains, and it is the configuration we would defend in front of a sceptical risk committee.

References

• Chen, T. & Guestrin, C. (2016). XGBoost: A Scalable Tree Boosting System. KDD '16, 785–794.
• Ke, G. et al. (2017). LightGBM: A Highly Efficient Gradient Boosting Decision Tree. NeurIPS 30.
• Friedman, J. H. (2001). Greedy Function Approximation: A Gradient Boosting Machine. Annals of Statistics 29(5), 1189–1232.
• Gu, S., Kelly, B. & Xiu, D. (2020). Empirical Asset Pricing via Machine Learning. Review of Financial Studies 33(5), 2223–2273.
• Lundberg, S. & Lee, S. (2017). A Unified Approach to Interpreting Model Predictions. NeurIPS 30.
• Bergmeir, C. & Benitez, J. M. (2012). On the use of cross-validation for time series predictor evaluation. Information Sciences 191, 192–213.