Uncertainty Set Estimator

An uncertainty set estimator builds the region in which a distribution moment is assumed to lie under estimation error. When one is provided to MeanRisk, the objective is evaluated at the least favorable moment within that region instead of at the point estimate. This is called worst-case optimization and is a class of robust optimization. It reduces the instability that arises from the estimation errors of the expected returns and the covariance matrix.

It follows the same API as scikit-learn’s estimator: the fit method takes X as the assets returns and stores the fitted set in its uncertainty_set_ attribute.

X can be any array-like structure (numpy array, pandas DataFrame, etc.)

Norm-ball representation

UncertaintySet represents deviations of a parameter vector \(z\) from its estimate \(\hat{z}\) as \(z - \hat{z} = L u\) with \(\lVert u \rVert_p \leq \kappa\):

\[\mathcal{U}=\left\{\hat{z} + L u\,:\,\lVert u \rVert_p \leq \kappa\right\}\]

The parameter \(z\) is \(\mu\) for expected return uncertainty, and \(\text{vec}(\Sigma)\), the vector obtained by stacking the columns of \(\Sigma\), for covariance uncertainty. The set is defined by three fields:

• radius, the size \(\kappa\) of the normalized ball.

• norm, the norm \(p\) that selects its shape: \(2\) for an ellipsoid, \(\infty\) for a box and \(1\) for a diamond.

• geometry, the linear map \(L\) that scales and mixes the uncertainty directions.

For \(p = 2\) and shape matrix \(S = L L^{T}\), the set is ellipsoidal:

\[U_{\mu}=\left\{\mu\,|\left(\mu-\hat{\mu}\right)S^{-1}\left(\mu-\hat{\mu}\right)^{T}\leq\kappa^{2}\right\}\]

Optimizers use the worst-case deviation over \(\mathcal{U}\), which for a linear exposure vector \(e\) is \(\kappa \lVert L^{T} e \rVert_q\), with \(q\) the dual norm of \(p\). The map \(L\) may be low-rank, which keeps this penalty tractable for covariance uncertainty.

Available estimators

For the expected returns:

• EmpiricalMuUncertaintySet

• BootstrapMuUncertaintySet

• OrthogonalMuUncertaintySet

For the covariance:

• EmpiricalCovarianceUncertaintySet

• BootstrapCovarianceUncertaintySet

• OrthogonalCovarianceUncertaintySet

The size of the set is controlled by confidence_level. The empirical and bootstrap estimators derive the radius from the quantile of a chi-squared distribution at that level, so a higher confidence level widens the set and increases the penalty. OrthogonalCovarianceUncertaintySet is parameterized by radius instead.

The Orthogonal estimators require the optimizer’s prior_estimator to be a factor model, such as CharacteristicsFactorModel. The optimizer passes the fitted return distribution to them. They confine the uncertainty to the subspace orthogonal to the factor-model loading matrix, which penalizes allocations in directions that the factor model prices only through idiosyncratic variance. See Orthogonal Space Regularization.

Covariance estimators may store a CompactCovarianceUncertaintySet instead of a norm ball. It expresses the penalty as a reduced quadratic form that the optimizer adds directly to the variance term, avoiding the lifted semidefinite formulation required by a generic set.

Example:

from skfolio.datasets import load_sp500_dataset
from skfolio.preprocessing import prices_to_returns
from skfolio.uncertainty_set import EmpiricalMuUncertaintySet

prices = load_sp500_dataset()
X = prices_to_returns(prices)

model = EmpiricalMuUncertaintySet()
model.fit(X)
print(model.uncertainty_set_)

Worst-case optimization

Uncertainty set estimators are provided to MeanRisk through mu_uncertainty_set_estimator and covariance_uncertainty_set_estimator. The optimizer fits them and subtracts the resulting penalty from the portfolio expected return:

\[w^{T}\hat{\mu} - \kappa\lVert L^{T}w\rVert_{q}\]

Covariance uncertainty is applied when risk_measure=RiskMeasure.VARIANCE or when max_variance is set.

Example:

from skfolio.datasets import load_sp500_dataset
from skfolio.optimization import MeanRisk, ObjectiveFunction
from skfolio.preprocessing import prices_to_returns
from skfolio.uncertainty_set import (
    BootstrapMuUncertaintySet,
    EmpiricalCovarianceUncertaintySet,
)

prices = load_sp500_dataset()
prices = prices["2020":]
X = prices_to_returns(prices)

model = MeanRisk(
    objective_function=ObjectiveFunction.MAXIMIZE_RATIO,
    mu_uncertainty_set_estimator=BootstrapMuUncertaintySet(confidence_level=0.5),
    covariance_uncertainty_set_estimator=EmpiricalCovarianceUncertaintySet(
        confidence_level=0.5
    ),
)
model.fit(X)
print(model.weights_)