Online Learning#

skfolio provides dedicated online utilities for estimators that support partial_fit. By updating a single stateful estimator incrementally rather than refitting from scratch at every split, online evaluation is significantly faster than standard cross-validation. These utilities cover stateful walk-forward evaluation, online covariance forecast diagnostics, and online hyper-parameter tuning.

Examples of supported estimators include EWMu, EWCovariance, RegimeAdjustedEWCovariance, EmpiricalPrior and portfolio optimizers such as MeanRisk when they embed incremental moment estimators through a prior estimator.

How Online Evaluation Works#

The online utilities all follow the same stateful evaluation pattern:

Clone the estimator once, starting from a clean unfitted state.
Initialize it on the first warmup_size observations with partial_fit.
Evaluate on the next test window out-of-sample, with optional purging between the data seen by the estimator and the test window.
Update the same estimator with the newly observed data.
Repeat until the end of the sample.

This differs from standard cross-validation, where each split fits an independent estimator clone. In the online setting, the estimator state is carried forward through time.

Non-Predictor Estimators Versus Portfolio Optimizers#

The online API distinguishes between non-predictor estimators and portfolio optimizers.

Non-predictor estimators such as covariance, expected-return, and prior estimators do not implement predict. Their scores are computed from the fitted estimator and the current test window using callables such as scorer(estimator, X_test). When using make_scorer, the appropriate form is make_scorer(..., response_method=None).
Portfolio optimization estimators such as MeanRisk are evaluated by collecting the out-of-sample predictions into a MultiPeriodPortfolio. Measures are then computed on that aggregate portfolio. In that case, scoring uses BaseMeasure enums directly rather than make_scorer.

Accordingly, online_predict is restricted to portfolio optimizers, while online_score and OnlineGridSearch / OnlineRandomizedSearch accept both categories.

Online Versus Standard Cross-Validation#

The standard cross_val_predict and its online counterpart online_predict are both designed exclusively for portfolio optimization estimators. The key differences are:

Fitting strategy: standard cross-validation clones and refits the estimator from scratch at every fold, while online evaluation maintains a single stateful estimator updated incrementally via partial_fit, which is significantly faster.
Scoring methodology: standard cross-validation scores each test fold independently and averages the results, which can be unreliable when test folds are short (e.g. the Sharpe ratio is undefined on a single observation). Online evaluation instead collects all out-of-sample predictions into a single MultiPeriodPortfolio and computes the metric on the full out-of-sample path, which is generally preferred for short rebalancing horizons.

online_score extends online evaluation to both portfolio optimizers and non-predictor estimators (covariance, expected-return, and prior estimators). For non-predictor estimators, scores are computed per test window and averaged by default.

Because the scoring methodology differs for portfolio optimizers, the online utilities are complementary to the existing cross-validation tools rather than replacements.

Online Covariance Forecast Evaluation#

online_covariance_forecast_evaluation evaluates the quality of covariance forecasts out-of-sample. It is intended for covariance estimators rather than portfolio optimizers, which should instead be evaluated with online_predict or online_score.

At each step, the covariance forecast produced after partial_fit is compared to the realized returns over the next test window. The resulting CovarianceForecastEvaluation provides diagnostics such as:

Mahalanobis calibration ratio for the full covariance structure across all eigenvalue directions.
Diagonal calibration ratio for asset-level variance calibration.
Portfolio standardized returns and the associated bias statistic for calibration along one portfolio direction by default, or multiple portfolio directions when explicit test portfolios are provided.
Portfolio QLIKE for portfolio variance forecast quality along one or more portfolio directions.

When portfolio_weights=None, the portfolio diagnostics use a single dynamic inverse-volatility portfolio direction by default. Passing explicit portfolio weights extends the evaluation to multiple selected traded directions.

See the example Online Covariance Forecast Evaluation for the complete workflow.

Online Hyper-Parameter Tuning#

OnlineGridSearch and OnlineRandomizedSearch extend the online workflow to hyper-parameter selection.

Conceptually, this is the online counterpart of combining GridSearchCV or RandomizedSearchCV with WalkForward using expand_train=True. The key difference is that online search updates each candidate incrementally via partial_fit along one sequential path instead of refitting it from scratch at every split.

Each candidate parameter configuration is evaluated on one full online walk-forward path. When refit=True, best_estimator_ exposes the selected fitted candidate without an additional fit after model selection because it has already been updated through the full sample during evaluation.

For non-predictor estimators, online tuning typically uses callable scorers such as QLIKE or calibration losses, typically wrapped with make_scorer using response_method=None. For multi-metric searches, refit should be set explicitly to the name of the metric used to select the best candidate.

See the example Online Covariance Hyperparameter Tuning for covariance tuning with both OnlineGridSearch and OnlineRandomizedSearch.

Online Evaluation of Portfolio Optimization#

For portfolio optimizers, the main entry points are online_predict and online_score.

online_predict returns a MultiPeriodPortfolio built from the sequence of out-of-sample portfolio predictions.
online_score returns a scalar measure, or a dict of measures, computed on the aggregate online evaluation.

This is useful when a portfolio estimator embeds incremental moment estimators such as EWMu and RegimeAdjustedEWCovariance.

See the example Online Evaluation of Portfolio Optimization for an end-to-end online evaluation of MeanRisk.