# Online Covariance Forecast Evaluation
This tutorial shows how to evaluate online covariance estimators with
[`online_covariance_forecast_evaluation`](https://skfolio.org/generated/skfolio.model_selection.online_covariance_forecast_evaluation.html.md#skfolio.model_selection.online_covariance_forecast_evaluation).
We compare [`EWCovariance`](https://skfolio.org/generated/skfolio.moments.EWCovariance.html.md#skfolio.moments.EWCovariance), a plain EWMA covariance, against
[`RegimeAdjustedEWCovariance`](https://skfolio.org/generated/skfolio.moments.RegimeAdjustedEWCovariance.html.md#skfolio.moments.RegimeAdjustedEWCovariance), its regime-adjusted counterpart
based on the Short-Term Volatility Update (STVU) [1](#id2).
Both support incremental updates via `partial_fit`, making them suitable for streaming
evaluation. For estimators that do not support `partial_fit`, the batch counterpart
[`covariance_forecast_evaluation`](https://skfolio.org/generated/skfolio.model_selection.covariance_forecast_evaluation.html.md#skfolio.model_selection.covariance_forecast_evaluation) can be used instead.
## Data
We load the S&P 500 [dataset](https://skfolio.org/user_guide/datasets.html.md#datasets) composed of the daily prices of 20
assets from the S&P 500 Index composition starting from 2010-01-04 up to 2022-12-28.
```Python
import numpy as np
from plotly.io import show
from skfolio.datasets import load_sp500_dataset
from skfolio.model_selection import (
CovarianceForecastComparison,
online_covariance_forecast_evaluation,
)
from skfolio.moments import (
EWCovariance,
RegimeAdjustedEWCovariance,
RegimeAdjustmentMethod,
)
from skfolio.preprocessing import prices_to_returns
prices = load_sp500_dataset()
X = prices_to_returns(prices)
X = X["2010":]
```
## Covariance Estimators
We use two covariance estimators:
* [`EWCovariance`](https://skfolio.org/generated/skfolio.moments.EWCovariance.html.md#skfolio.moments.EWCovariance)
* [`RegimeAdjustedEWCovariance`](https://skfolio.org/generated/skfolio.moments.RegimeAdjustedEWCovariance.html.md#skfolio.moments.RegimeAdjustedEWCovariance)
`EWCovariance` can react slowly to volatility shocks.
`RegimeAdjustedEWCovariance` adds a regime adjustment via the Short-Term Volatility
Update (STVU). This applies a scalar multiplier to better align predicted and
realized risk when volatility regimes change faster than a plain EWMA can track.
We set the same variance half-life of 40 trading days for both estimators and
a correlation half-life of 80 trading days for `RegimeAdjustedEWCovariance`. Lower
half-life for variance allows the model to adapt faster to volatility shifts, while
higher half-life for correlation enables more stable estimation of co-movements, which
typically require more data for reliable inference and reduces estimation noise. This
choice also aligns with empirical evidence that volatility tends to mean-revert faster
than correlation.
```Python
ew_cov = EWCovariance(half_life=40)
stvu_cov = RegimeAdjustedEWCovariance(
half_life=40,
corr_half_life=80,
regime_half_life=20,
regime_method=RegimeAdjustmentMethod.RMS,
)
```
## Evaluate Each Estimator
We now evaluate each estimator with
[`online_covariance_forecast_evaluation`](https://skfolio.org/generated/skfolio.model_selection.online_covariance_forecast_evaluation.html.md#skfolio.model_selection.online_covariance_forecast_evaluation).
This function performs a walk-forward evaluation. At each step, it updates the
estimator with `partial_fit` and compares the one-step-ahead forecast with the next
realized return.
Here, `warmup_size=252` reserves the first year for initialization, while
`test_size=1` evaluates the forecast one day at a time.
```Python
ew_evaluation = online_covariance_forecast_evaluation(
ew_cov,
X,
warmup_size=252,
test_size=1,
)
stvu_evaluation = online_covariance_forecast_evaluation(
stvu_cov,
X,
warmup_size=252,
test_size=1,
)
```
## Summary Table
Let’s display the summary of the regime-adjusted covariance forecast
evaluation. The four rows are:
* **Mahalanobis ratio** evaluates whether the full covariance structure
(all eigenvalue directions) is correctly specified. The target is 1.0,
with values above 1.0 indicating underestimated risk and values below
1.0 indicating overestimated risk.
* **Diagonal ratio** evaluates the individual asset variances only, with
the same 1.0 target and interpretation.
* **Portfolio standardized returns** evaluate calibration along one
portfolio direction rather than across all directions. Their `std`
column is the bias statistic, with values near 1.0 meaning
well-calibrated portfolio risk.
* **Portfolio QLIKE** evaluates portfolio variance forecasts along one
portfolio direction by comparing the forecast portfolio variance with
the realized sum of squared portfolio returns over the evaluation
window. Lower values indicate better variance forecasts.
```Python
stvu_evaluation.summary()
```
[plotly figure stripped from llms output]
## Side-by-Side Comparison
We now compare both evaluations with
[`CovarianceForecastComparison`](https://skfolio.org/generated/skfolio.model_selection.CovarianceForecastComparison.html.md#skfolio.model_selection.CovarianceForecastComparison):
```Python
comparison = CovarianceForecastComparison(
[ew_evaluation, stvu_evaluation], names=["EWMA Cov", "STVU Cov"]
)
comparison.summary()
```
[plotly figure stripped from llms output]
## QLIKE Loss
Let’s now plot the QLIKE loss. It compares the forecast portfolio variance
with the realized sum of squared portfolio returns over the evaluation
window, with lower values indicating better portfolio variance forecasts.
Because STVU rescales the forecast toward realized risk, we generally
expect it to achieve a lower QLIKE.
```Python
comparison.plot_qlike_loss()
```
[plotly figure stripped from llms output]
## Exceedance Rates
We can also display the exceedance summary. If the covariance forecast were perfectly
calibrated and returns were Gaussian, the squared Mahalanobis distance would follow a
chi-squared distribution. The exceedance rate measures how often this distance exceeds
the chi-squared threshold at a given significance level.
In practice, daily equity returns are fat-tailed, so in this example both estimators
exceed the nominal levels. This metric is therefore more useful for comparing
estimators than for making an absolute calibration statement.
```Python
comparison.exceedance_summary()
```
[plotly figure stripped from llms output]
The QLIKE plot also includes the P5-P95 bands from the 30 portfolios:
```Python
multi_portfolio_comparison.plot_qlike_loss()
```
[plotly figure stripped from llms output]
## Conclusion
This tutorial showed how to:
1. Define online covariance estimators supporting `partial_fit`.
2. Evaluate them with [`online_covariance_forecast_evaluation`](https://skfolio.org/generated/skfolio.model_selection.online_covariance_forecast_evaluation.html.md#skfolio.model_selection.online_covariance_forecast_evaluation).
3. Inspect calibration diagnostics and QLIKE.
4. Compare multiple estimators with [`CovarianceForecastComparison`](https://skfolio.org/generated/skfolio.model_selection.CovarianceForecastComparison.html.md#skfolio.model_selection.CovarianceForecastComparison).
5. Extend the analysis to multiple portfolio directions.
In the [next tutorial](https://skfolio.org/auto_examples/online_learning/plot_2_online_hyperparameter_tuning.html.md#sphx-glr-auto-examples-online-learning-plot-2-online-hyperparameter-tuning-py),
we show how to tune covariance estimator hyperparameters with online search.
* **[1]** G. Paleologo, “The Elements of Quantitative Investing”, Wiley Finance (2025).
**Total running time of the script:** (0 minutes 23.537 seconds)