<a id="uncertainty-set-estimator"></a>

<a id="id1"></a>

# Uncertainty Set Estimator

An [uncertainty set estimator](https://skfolio.org/api.html.md#uncertainty-set-ref) builds the region in which a
distribution moment is assumed to lie under estimation error. When one is provided to
[`MeanRisk`](https://skfolio.org/generated/skfolio.optimization.MeanRisk.html.md#skfolio.optimization.MeanRisk), the objective is evaluated at the least
favorable moment within that region instead of at the point estimate. This is called
worst-case optimization and is a class of robust optimization. It reduces the
instability that arises from the estimation errors of the expected returns and the
covariance matrix.

It follows the same API as scikit-learn’s `estimator`: the `fit` method takes `X` as
the assets returns and stores the fitted set in its `uncertainty_set_` attribute.

`X` can be any array-like structure (numpy array, pandas DataFrame, etc.)

<a id="norm-ball-representation"></a>

## Norm-ball representation

[`UncertaintySet`](https://skfolio.org/generated/skfolio.uncertainty_set.UncertaintySet.html.md#skfolio.uncertainty_set.UncertaintySet) represents deviations of a parameter vector $z$ from its
estimate $\hat{z}$ as $z - \hat{z} = L u$ with
$\lVert u \rVert_p \leq \kappa$:

> $$
> \mathcal{U}=\left\{\hat{z} + L u\,:\,\lVert u \rVert_p \leq \kappa\right\}

> $$

The parameter $z$ is $\mu$ for expected return uncertainty, and
$\text{vec}(\Sigma)$, the vector obtained by stacking the columns of
$\Sigma$, for covariance uncertainty. The set is defined by three fields:

* `radius`, the size $\kappa$ of the normalized ball.
* `norm`, the norm $p$ that selects its shape: $2$ for an ellipsoid,
  $\infty$ for a box and $1$ for a diamond.
* `geometry`, the linear map $L$ that scales and mixes the uncertainty
  directions.

For $p = 2$ and shape matrix $S = L L^{T}$, the set is ellipsoidal:

> $$
> U_{\mu}=\left\{\mu\,|\left(\mu-\hat{\mu}\right)S^{-1}\left(\mu-\hat{\mu}\right)^{T}\leq\kappa^{2}\right\}

> $$

Optimizers use the worst-case deviation over $\mathcal{U}$, which for a linear
exposure vector $e$ is $\kappa \lVert L^{T} e \rVert_q$, with $q$ the
dual norm of $p$. The map $L$ may be low-rank, which keeps this penalty
tractable for covariance uncertainty.

<a id="available-estimators"></a>

## Available estimators

For the expected returns:
: * [`EmpiricalMuUncertaintySet`](https://skfolio.org/generated/skfolio.uncertainty_set.EmpiricalMuUncertaintySet.html.md#skfolio.uncertainty_set.EmpiricalMuUncertaintySet)
  * [`BootstrapMuUncertaintySet`](https://skfolio.org/generated/skfolio.uncertainty_set.BootstrapMuUncertaintySet.html.md#skfolio.uncertainty_set.BootstrapMuUncertaintySet)
  * [`OrthogonalMuUncertaintySet`](https://skfolio.org/generated/skfolio.uncertainty_set.OrthogonalMuUncertaintySet.html.md#skfolio.uncertainty_set.OrthogonalMuUncertaintySet)

For the covariance:
: * [`EmpiricalCovarianceUncertaintySet`](https://skfolio.org/generated/skfolio.uncertainty_set.EmpiricalCovarianceUncertaintySet.html.md#skfolio.uncertainty_set.EmpiricalCovarianceUncertaintySet)
  * [`BootstrapCovarianceUncertaintySet`](https://skfolio.org/generated/skfolio.uncertainty_set.BootstrapCovarianceUncertaintySet.html.md#skfolio.uncertainty_set.BootstrapCovarianceUncertaintySet)
  * [`OrthogonalCovarianceUncertaintySet`](https://skfolio.org/generated/skfolio.uncertainty_set.OrthogonalCovarianceUncertaintySet.html.md#skfolio.uncertainty_set.OrthogonalCovarianceUncertaintySet)

The size of the set is controlled by `confidence_level`. The empirical and bootstrap
estimators derive the radius from the quantile of a chi-squared distribution at that
level, so a higher confidence level widens the set and increases the penalty.
[`OrthogonalCovarianceUncertaintySet`](https://skfolio.org/generated/skfolio.uncertainty_set.OrthogonalCovarianceUncertaintySet.html.md#skfolio.uncertainty_set.OrthogonalCovarianceUncertaintySet) is parameterized by `radius` instead.

The `Orthogonal` estimators require the optimizer’s `prior_estimator` to be a factor
model, such as [`CharacteristicsFactorModel`](https://skfolio.org/generated/skfolio.prior.CharacteristicsFactorModel.html.md#skfolio.prior.CharacteristicsFactorModel). The optimizer passes
the fitted return distribution to them. They confine the uncertainty to the subspace
orthogonal to the factor-model loading matrix, which penalizes allocations in directions
that the factor model prices only through idiosyncratic variance. See
[Orthogonal Space Regularization](https://skfolio.org/user_guide/factor_models.html.md#factor-model-orthogonal-space-regularization).

Covariance estimators may store a [`CompactCovarianceUncertaintySet`](https://skfolio.org/generated/skfolio.uncertainty_set.CompactCovarianceUncertaintySet.html.md#skfolio.uncertainty_set.CompactCovarianceUncertaintySet) instead of a
norm ball. It expresses the penalty as a reduced quadratic form that the optimizer adds
directly to the variance term, avoiding the lifted semidefinite formulation required by
a generic set.

**Example:**

```python
from skfolio.datasets import load_sp500_dataset
from skfolio.preprocessing import prices_to_returns
from skfolio.uncertainty_set import EmpiricalMuUncertaintySet

prices = load_sp500_dataset()
X = prices_to_returns(prices)

model = EmpiricalMuUncertaintySet()
model.fit(X)
print(model.uncertainty_set_)
```

<a id="worst-case-optimization"></a>

## Worst-case optimization

Uncertainty set estimators are provided to [`MeanRisk`](https://skfolio.org/generated/skfolio.optimization.MeanRisk.html.md#skfolio.optimization.MeanRisk)
through `mu_uncertainty_set_estimator` and `covariance_uncertainty_set_estimator`. The
optimizer fits them and subtracts the resulting penalty from the portfolio expected
return:

> $$
> w^{T}\hat{\mu} - \kappa\lVert L^{T}w\rVert_{q}

> $$

Covariance uncertainty is applied when `risk_measure=RiskMeasure.VARIANCE` or when
`max_variance` is set.

**Example:**

```python
from skfolio.datasets import load_sp500_dataset
from skfolio.optimization import MeanRisk, ObjectiveFunction
from skfolio.preprocessing import prices_to_returns
from skfolio.uncertainty_set import (
    BootstrapMuUncertaintySet,
    EmpiricalCovarianceUncertaintySet,
)

prices = load_sp500_dataset()
prices = prices["2020":]
X = prices_to_returns(prices)

model = MeanRisk(
    objective_function=ObjectiveFunction.MAXIMIZE_RATIO,
    mu_uncertainty_set_estimator=BootstrapMuUncertaintySet(confidence_level=0.5),
    covariance_uncertainty_set_estimator=EmpiricalCovarianceUncertaintySet(
        confidence_level=0.5
    ),
)
model.fit(X)
print(model.weights_)
```