skfolio.metrics.make_scorer#

skfolio.metrics.make_scorer(score_func, greater_is_better=None, response_method='predict', **kwargs)[source]#

Make a scorer from a measure, a portfolio score function, or a non-predictor estimator score function.

This function wraps scoring functions for use in model selection:

  • response_method="predict" (default): for portfolio optimization estimators (e.g. MeanRisk). Compatible with GridSearchCV and cross_val_score.

  • response_method=None: for non-predictor estimators (covariance, expected returns, prior) that implement fit but not predict. Compatible with both sklearn cross-validation utilities and skfolio online utilities (OnlineGridSearch, online_score).

Note

For online evaluation of portfolio optimization estimators, pass a measure directly to the scoring parameter instead of using make_scorer. Online evaluation scores the full aggregated MultiPeriodPortfolio rather than averaging per-fold scores.

Parameters:
score_funcMeasure | callable

If score_func is a measure, we return the measure of the predicted Portfolio times 1 or -1 depending on greater_is_better. response_method must be "predict" in this case.

If response_method="predict", score_func must be a score function (or loss function) with signature score_func(pred, **kwargs) where pred is the predicted Portfolio.

If response_method=None, score_func must be a score function (or loss function) with signature score_func(estimator, X_test, **kwargs) where estimator is the fitted non-predictor estimator and X_test the realized returns.

greater_is_betterbool, optional

Whether score_func is a score function (high is good) or a loss function (low is good). In the latter case the scorer sign-flips the outcome so that higher values always indicate a better model. The default (None) is:

response_methodstr or None, default=”predict”

Determines how the scorer obtains predictions. Only "predict" and None are supported:

  • "predict": call estimator.predict(X) and pass the resulting Portfolio to score_func. Use for portfolio optimization estimators (e.g. MeanRisk).

  • None: pass (estimator, X_test) directly to score_func without calling any response method. Use for non-predictor estimators (covariance, expected returns, prior).

**kwargsadditional arguments

Additional parameters to be passed to score_func.

Returns:
scorercallable

Callable object with signature scorer(estimator, X, y=None) that returns a scalar score (higher is better).

Examples

Portfolio scorer from a measure:

>>> from skfolio.measures import RatioMeasure
>>> scorer = make_scorer(RatioMeasure.SHARPE_RATIO)

Portfolio scorer from a custom function:

>>> def custom(pred):
...     return pred.mean - 2 * pred.variance
>>> scorer = make_scorer(custom)

Non-predictor estimator scorer for covariance evaluation:

>>> from skfolio.metrics import portfolio_variance_qlike_loss
>>> import numpy as np
>>> scorer = make_scorer(
...     portfolio_variance_qlike_loss,
...     greater_is_better=False,
...     response_method=None,
...     portfolio_weights=np.ones(20) / 20,
... )