<a id="skfolio-prior-entropypooling"></a>

# skfolio.prior.EntropyPooling

<a id="skfolio.prior.EntropyPooling"></a>

### *class* skfolio.prior.EntropyPooling(prior_estimator=None, mean_views=None, variance_views=None, correlation_views=None, skew_views=None, kurtosis_views=None, value_at_risk_views=None, cvar_views=None, value_at_risk_beta=0.95, cvar_beta=0.95, groups=None, solver='TNC', solver_params=None)

Entropy Pooling estimator.

Entropy Pooling, introduced by Attilio Meucci in 2008 as a generalization of the
Black-Litterman framework, is a nonparametric method for adjusting a baseline
(“prior”) probability distribution to incorporate user-defined views by finding the
posterior distribution closest to the prior while satisfying those views.

User-defined views can be **elicited** from domain experts or **derived** from
quantitative analyses.

Grounded in information theory, it updates the distribution in the least-informative
way by minimizing the Kullback-Leibler divergence (relative entropy) under the
specified view constraints.

Mathematically, the problem is formulated as:

$$
\begin{aligned}
\min_{\mathbf{q}} \quad & \sum_{i=1}^T q_i \log\left(\frac{q_i}{p_i}\right) \\
\text{subject to} \quad & \sum_{i=1}^T q_i = 1 \quad \text{(normalization constraint)} \\
                        & \mathbb{E}_q[f_j(X)] = v_j \quad(\text{or } \le v_j, \text{ or } \ge v_j), \quad j = 1,\dots,k, \text{(view constraints)} \\
                        & q_i \ge 0, \quad i = 1, \dots, T
\end{aligned}
$$

Where:

- $T$ is the number of observations (number of scenarios).
- $p_i$ is the prior probability of scenario $x_i$.
- $q_i$ is the posterior probability of scenario $x_i$.
- $X$ is the scenario matrix of shape (n_observations, n_assets).
- $f_j$ is the j <sub>th</sub> view function.
- $v_j$ is the target value imposed by the j <sub>th</sub> view.
- $k$ is the total number of views.

The `skfolio` implementation supports the following views:
: * Equalities
  * Inequalities
  * Ranking
  * Linear combinations (e.g. relative views)
  * Views on groups of assets

On the following measures:
: * Mean
  * Variance
  * Skew
  * Kurtosis
  * Correlation
  * Value-at-Risk (VaR)
  * Conditional Value-at-Risk (CVaR)

* **Parameters:**
  **prior_estimator** *BasePrior, optional*
  : Estimator of the asset’s prior distribution, fitted from a
    [prior estimator](https://skfolio.org/user_guide/prior.html.md#prior). The default (`None`) is to use the
    empirical prior [`EmpiricalPrior()`](https://skfolio.org/generated/skfolio.prior.EmpiricalPrior.html.md#skfolio.prior.EmpiricalPrior). To perform Entropy
    Pooling on synthetic data, you can use [`SyntheticData`](https://skfolio.org/generated/skfolio.prior.SyntheticData.html.md#skfolio.prior.SyntheticData)
    by setting `prior_estimator = SyntheticData()`.

  **mean_views** *list[str], optional*
  : Views on asset means.
    The views must match any of following patterns:
    > * `"ref1 >= a"`
    > * `"ref1 == b"`
    > * `"ref1 <= ref1"`
    > * `"ref1 >= a * prior(ref1)"`
    > * `"ref1 == b * prior(ref2)"`
    > * `"a * ref1 + b * ref2 + c <= d * ref3"`
    <br/>
    With `"ref1"`, `"ref2"` … the assets names or the groups names provided
    in the parameter `groups`. Assets names can be referenced without the need of
    `groups` if the input `X` of the `fit` method is a DataFrame with assets names
    in columns. Otherwise, the default asset names `x0, x1, ...` are assigned.
    By using the term `prior(...)`, you can reference the asset prior mean.
    <br/>
    For example:
    > * `"SPX >= 0.0015"` –> The mean of SPX is greater than 0.15% (daily mean if
    >   `X` is daily)
    > * `"SX5E == 0.002"` –> The mean of SX5E equals 0.2%
    > * `"AAPL <= 0.003"` –> The mean of AAPL is less than to 0.3%
    > * `"SPX <= SX5E"` –> Ranking view: the mean of SPX is less than SX5E
    > * `"SPX >= 1.5 * prior(SPX)"` –> The mean of SPX increases by at least 50% (versus its prior)
    > * `"SX5E == 2 * prior(SX5E)"` –> The mean of SX5E doubles (versus its prior)
    > * `"AAPL <= 0.8 * prior(SPX)"` –> The mean of AAPL is less than 0.8 times the SPX prior
    > * `"SX5E + SPX >= 0"` –> The sum of SX5E and SPX mean is greater than zero
    > * `"US == 0.007"` –> The sum of means of US assets equals 0.7%
    > * `"Equity == 3 * Bond"` –> The sum of means of Equity assets equals
    >   three times the  sum of means of Bond assets.
    > * `"2*SPX + 3*Europe <= Bond + 0.05"` –> Mixing assets and group mean views

  **variance_views** *list[str], optional*
  : Views on asset variances.
    It supports the same patterns as `mean_views`.
    <br/>
    For example:
    > * `"SPX >= 0.0009"` –> SPX variance is greater than 0.0009 (daily)
    > * `"SX5E == 1.5 * prior(SX5E)"` –> SX5E variance increases by 150% (versus
    >   its prior)

  **skew_views** *list[str], optional*
  : Views on asset skews.
    It supports the same patterns as `mean_views`.
    <br/>
    For example:
    > * `"SPX >= 2.0"` –> SPX skew is greater than 2.0
    > * `"SX5E == 1.5 * prior(SX5E)"` –> SX5E skew increases by 150% (versus its
    >   prior)

  **kurtosis_views** *list[str], optional*
  : Views on asset kurtosis.
    It supports the same patterns as `mean_views`.
    <br/>
    For example:
    > * `"SPX >= 9.0"` –> SPX kurtosis is greater than 9.0
    > * `"SX5E == 1.5 * prior(SX5E)"` –> SX5E kurtosis increases by 150% (versus
    >   its prior)

  **correlation_views** *list[str], optional*
  : Views on asset correlations.
    The views must match any of following patterns:
    > * `"(asset1, asset2) >= a"`
    > * `"(asset1, asset2) == a"`
    > * `"(asset1, asset2) <= a"`
    > * `"(asset1, asset2) >= a * prior(asset1, asset2)"`
    > * `"(asset1, asset2) == a * prior(asset1, asset2)"`
    > * `"(asset1, asset2) <= a * prior(asset1, asset2)"`
    <br/>
    For example:
    > * `"(SPX, SX5E) >= 0.8"` –> the correlation between SPX and SX5E is greater than 80%
    > * `"(SPX, SX5E) == 1.5 * prior(SPX, SX5E)"` –> the correlation between SPX
    >   and SX5E increases by 150% (versus its prior)

  **value_at_risk_views** *list[str], optional*
  : Views on asset Value-at-Risks (VaR).
    <br/>
    For example:
    > * `"SPX >= 0.03"` –> SPX VaR is greater than 3%
    > * `"SX5E == 1.5 * prior(SX5E)"` –> SX5E VaR increases by 150% (versus its prior)

  **cvar_views** *list[str], optional*
  : Views on asset Conditional Value-at-Risks (CVaR).
    It only supports equalities.
    <br/>
    For example:
    > * `"SPX == 0.05"` –> SPX CVaR equals 5%
    > * `"SX5E == 1.5 * prior(SX5E)"` –> SX5E CVaR increases by 150% (versus its prior)

  **value_at_risk_beta** *float, default=0.95*
  : Confidence level for VaR views, by default 95%.

  **cvar_beta** *float, default=0.95*
  : Confidence level for CVaR views, by default 95%.

  **groups** *dict[str, list[str]] or array-like of strings of shape (n_groups, n_assets), optional*
  : Asset grouping for use in group-based views. If a dict is provided, keys are
    asset names and values are lists of group labels; then `X` must be a DataFrame
    whose columns match those asset names.
    <br/>
    For example:
    > * `groups = {"SX5E": ["Equity", "Europe"], "SPX": ["Equity", "US"], "TLT": ["Bond", "US"]}`
    > * `groups = [["Equity", "Equity", "Bond"], ["Europe", "US", "US"]]`

  **solver** *str, default=”TNC”*
  : The solver to use.
    - “TNC” (default) solves the entropic-pooling dual via SciPy’s Truncated Newton
      Constrained method. By exploiting the smooth Fenchel dual and its
      closed-form gradient, it operates in $\mathbb{R}^k$ (the number of
      constraints) rather than $\mathbb{R}^T$ (the number of scenarios),
      yielding an order-of-magnitude speedup over primal CVXPY interior-point
      solvers.
    - CVXPY solvers (e.g. “CLARABEL”) solve the entropic-pooling problem in its
      primal form using interior-point methods. While they tend to be slower than
      the dual-based approach, they often achieve higher accuracy by enforcing
      stricter primal feasibility and duality-gap tolerances. See the CVXPY
      documentation for supported solvers:
      [https://www.cvxpy.org/tutorial/advanced/index.html#choosing-a-solver](https://www.cvxpy.org/tutorial/advanced/index.html#choosing-a-solver).

  **solver_params** *dict, optional*
  : Additional parameters to pass to the chosen solver.
    - When using **SciPy TNC**, supported options include (but are not limited to)
      `gtol`, `ftol`, `eps`, `maxfun`, `maxCGit`, `stepmx`, `disp`. See the SciPy
      documentation for a full list and descriptions:
      [https://docs.scipy.org/doc/scipy/reference/optimize.minimize-tnc.html](https://docs.scipy.org/doc/scipy/reference/optimize.minimize-tnc.html)
    - When using a **CVXPY** solver (e.g. `"CLARABEL"`), supply any
      solver-specific parameters here. Refer to the CVXPY solver guide for
      details: [https://www.cvxpy.org/tutorial/solvers](https://www.cvxpy.org/tutorial/solvers)
* **Attributes:**
  **return_distribution_** *ReturnDistribution*
  : Fitted [`ReturnDistribution`](https://skfolio.org/generated/skfolio.prior.ReturnDistribution.html.md#skfolio.prior.ReturnDistribution) to be used by the optimization
    estimators, containing the assets distribution, moments estimation and the EP
    posterior probabilities (sample weights).

  **relative_entropy_** *float*
  : The KL-divergence between the posterior and prior distributions.

  **effective_number_of_scenarios_** *float*
  : Effective number of scenarios defined as the perplexity of sample weight
    (exponential of entropy).

  **prior_estimator_** *BasePrior*
  : Fitted `prior_estimator`.

  **n_features_in_** *int*
  : Number of assets seen during `fit`.

  **feature_names_in_** *ndarray of shape (`n_features_in_`,)*
  : Names of features seen during `fit`. Defined only when `X`
    has feature names that are all strings.

### Methods

| [`fit`](#skfolio.prior.EntropyPooling.fit)(X[, y])            | Fit the Entropy Pooling estimator.    |
|-------------------------------------------------------------------------|---------------------------------------|
| [`get_metadata_routing`](#skfolio.prior.EntropyPooling.get_metadata_routing)() | Get metadata routing of this object.  |
| [`get_params`](#skfolio.prior.EntropyPooling.get_params)([deep])     | Get parameters for this estimator.    |
| [`set_params`](#skfolio.prior.EntropyPooling.set_params)(\*\*params) | Set the parameters of this estimator. |

### Notes

Entropy Pooling re-weights the sample probabilities of the prior distribution and is
therefore constrained by the support (completeness) of that distribution. For
example, if the historical distribution contains no returns below -10% for a given
asset, we cannot impose a CVaR view of 15%: no matter how we adjust the sample
probabilities, such tail data do not exist.

Therefore, to impose extreme views on a sparse historical distribution, one must
generate synthetic data. In that case, the EP posterior is only as reliable as the
synthetic scenarios. It is thus essential to use a generator capable of
extrapolating tail dependencies, such as [`VineCopula`](https://skfolio.org/generated/skfolio.distribution.VineCopula.html.md#skfolio.distribution.VineCopula),
to model joint extreme events accurately.

Two methods are available:
: * Dual form: solves the Fenchel dual of the EP problem using Truncated
    Newton Constrained method.
  * Primal form: solves the original relative-entropy projection in
    probability-space via interior-point algorithms.

See the solver parameter’s docstring for full details on available solvers and
options.

To handle nonlinear views, constraints are linearized by fixing the relevant asset
moments (e.g., means or variances) and then solved via **nested entropic tilting**.
At each stage, the KL-divergence is minimized relative to the original prior,
while nesting all previously enforced (linearized) views into the feasible set:

* Stage 1: impose views on asset means, VaR and CVaR.
* Stage 2: carry forward Stage 1 constraints and add variance, fixing the
  mean at its Stage 1 value.
* Stage 3: carry forward Stage 2 constraints and add skewness, kurtosis and
  pairwise correlations, fixing both mean and variance at their Stage 2 values.

Because each entropic projection nests the prior views, every constraint from
earlier stages is preserved as new ones are added, yielding a final distribution
that satisfies all original nonlinear views while staying as close as possible to
the original prior.

Only the necessary moments are fixed. Slack variables with an L1 norm penalty are
introduced to avoid solver infeasibility that may arise from overly tight
constraints.

CVaR view constraints cannot be directly expressed as linear functions of the
posterior probabilities. Therefore, when CVaR views are present, the EP problem is
solved by recursively solving a series of convex programs that approximate the
nonlinear CVaR constraint.

This implementation improves upon Meucci’s algorithm [[1]](#rf4c236d00083-1) by formulating the problem
in continuous space as a function of the dual variables etas (VaR levels), rather
than searching over discrete tail sizes. This formulation not only handles the
CVaR constraint more directly but also supports multiple CVaR views on different
assets.

Although the overall problem is convex in the dual variables etas, it remains
non-smooth due to the presence of the positive-part operator in the CVaR
definition. Consequently, we employ derivative-free optimization methods.
Specifically, for a single CVaR view we use a one-dimensional root-finding
method (Brent’s method), and for the multivariate case (supporting multiple
CVaR views) we use Powell’s method for derivative-free convex descent.

### References

* <a id='rf4c236d00083-1'>**[1]**</a> “Fully Flexible Extreme Views”, Journal of Risk, Meucci, Ardia & Keel (2011)
* <a id='rf4c236d00083-2'>**[2]**</a> “Fully Flexible Views: Theory and Practice”, Risk, Meucci (2013).
* <a id='rf4c236d00083-3'>**[3]**</a> “Effective Number of Scenarios in Fully Flexible Probabilities”, GARP Risk Professional, Meucci (2012)
* <a id='rf4c236d00083-4'>**[4]**</a> “I-Divergence Geometry of Probability Distributions and Minimization Problems”, The Annals of Probability, Csiszar (1975)

### Examples

For a full tutorial on entropy pooling, see [Entropy Pooling](https://skfolio.org/auto_examples/entropy_pooling/plot_1_entropy_pooling.html.md#sphx-glr-auto-examples-entropy-pooling-plot-1-entropy-pooling-py).

```pycon
>>> from skfolio import RiskMeasure
>>> from skfolio.datasets import load_sp500_dataset
>>> from skfolio.preprocessing import prices_to_returns
>>> from skfolio.prior import EntropyPooling
>>> from skfolio.optimization import HierarchicalRiskParity
>>>
>>> prices = load_sp500_dataset()
>>> prices = prices[["AMD", "BAC", "GE", "JNJ", "JPM", "LLY", "PG"]]
>>> X = prices_to_returns(prices)
>>>
>>> groups = {
...     "AMD": ["Technology", "Growth"],
...     "BAC": ["Financials", "Value"],
...     "GE": ["Industrials", "Value"],
...     "JNJ": ["Healthcare", "Defensive"],
...     "JPM": ["Financials", "Income"],
...     "LLY": ["Healthcare", "Defensive"],
...     "PG": ["Consumer", "Defensive"],
... }
>>>
>>> entropy_pooling = EntropyPooling(
...     mean_views=[
...         "JPM == -0.002",
...         "PG >= LLY",
...         "BAC >= prior(BAC) * 1.2",
...         "Financials == 2 * Growth",
...     ],
...     variance_views=[
...         "BAC == prior(BAC) * 4",
...     ],
...     correlation_views=[
...         "(BAC,JPM) == 0.80",
...         "(BAC,JNJ) <= prior(BAC,JNJ) * 0.5",
...     ],
...     skew_views=[
...         "BAC == -0.05",
...     ],
...     cvar_views=[
...         "GE == 0.08",
...     ],
...     cvar_beta=0.90,
...     groups=groups,
... )
>>>
>>> entropy_pooling.fit(X)
EntropyPooling(correlation_views=...
>>>
>>> print(entropy_pooling.relative_entropy_)
0.18...
>>> print(entropy_pooling.effective_number_of_scenarios_)
687...
>>> print(entropy_pooling.return_distribution_.sample_weight)
[0.00010... 0.00009... ... 0.00010... 0.00010...]
>>>
>>> # CVaR Hierarchical Risk Parity optimization on Entropy Pooling
>>> model = HierarchicalRiskParity(
...     risk_measure=RiskMeasure.CVAR,
...     prior_estimator=entropy_pooling
... )
>>> model.fit(X)
HierarchicalRiskParity(prior_estimator=...
>>> print(model.weights_)
[0.073... 0.054... 0.076... 0.20... 0.063... 0.32... 0.20...]
>>>
>>> # Stress Test the Portfolio
>>> entropy_pooling = EntropyPooling(cvar_views=["AMD == 0.10"])
>>> entropy_pooling.fit(X)
EntropyPooling(cvar_views=['AMD == 0.10'])
>>>
>>> stressed_dist = entropy_pooling.return_distribution_
>>>
>>> stressed_ptf = model.predict(stressed_dist)
```

<a id="skfolio.prior.EntropyPooling.fit"></a>

#### fit(X, y=None, \*\*fit_params)

Fit the Entropy Pooling estimator.

* **Parameters:**
  **X** *array-like of shape (n_observations, n_assets)*
  : Price returns of the assets.

  **y** *Ignored*
  : Not used, present for API consistency by convention.

  **\*\*fit_params** *dict*
  : Parameters to pass to the underlying estimators.
    Only available if `enable_metadata_routing=True`, which can be
    set by using `sklearn.set_config(enable_metadata_routing=True)`.
    See [Metadata Routing User Guide](https://skfolio.org/user_guide/metadata_routing.html.md#metadata-routing) for
    more details.
* **Returns:**
  **self** *EntropyPooling*
  : Fitted estimator.

<a id="skfolio.prior.EntropyPooling.get_metadata_routing"></a>

#### get_metadata_routing()

Get metadata routing of this object.

Please check [User Guide](https://skfolio.org/user_guide/metadata_routing.html.md#metadata-routing) on how the routing
mechanism works.

* **Returns:**
  **routing** *MetadataRequest*
  : A `MetadataRequest` encapsulating
    routing information.

<a id="skfolio.prior.EntropyPooling.get_params"></a>

#### get_params(deep=True)

Get parameters for this estimator.

* **Parameters:**
  **deep** *bool, default=True*
  : If True, will return the parameters for this estimator and
    contained subobjects that are estimators.
* **Returns:**
  **params** *dict*
  : Parameter names mapped to their values.

<a id="skfolio.prior.EntropyPooling.set_params"></a>

#### set_params(\*\*params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects
(such as `Pipeline`). The latter have
parameters of the form `<component>__<parameter>` so that it’s
possible to update each component of a nested object.

* **Parameters:**
  **\*\*params** *dict*
  : Estimator parameters.
* **Returns:**
  **self** *estimator instance*
  : Estimator instance.