skfolio.prior.EntropyPooling

class skfolio.prior.EntropyPooling(prior_estimator=None, mean_views=None, variance_views=None, correlation_views=None, skew_views=None, kurtosis_views=None, value_at_risk_views=None, cvar_views=None, value_at_risk_beta=0.95, cvar_beta=0.95, groups=None, solver='TNC', solver_params=None)

Entropy Pooling estimator.

Entropy Pooling, introduced by Attilio Meucci in 2008 as a generalization of the Black-Litterman framework, is a nonparametric method for adjusting a baseline (“prior”) probability distribution to incorporate user-defined views by finding the posterior distribution closest to the prior while satisfying those views.

User-defined views can be elicited from domain experts or derived from quantitative analyses.

Grounded in information theory, it updates the distribution in the least-informative way by minimizing the Kullback-Leibler divergence (relative entropy) under the specified view constraints.

Mathematically, the problem is formulated as:

\[\begin{split}\begin{aligned} \min_{\mathbf{q}} \quad & \sum_{i=1}^T q_i \log\left(\frac{q_i}{p_i}\right) \\ \text{subject to} \quad & \sum_{i=1}^T q_i = 1 \quad \text{(normalization constraint)} \\ & \mathbb{E}_q[f_j(X)] = v_j \quad(\text{or } \le v_j, \text{ or } \ge v_j), \quad j = 1,\dots,k, \text{(view constraints)} \\ & q_i \ge 0, \quad i = 1, \dots, T \end{aligned}\end{split}\]

Where:

• \(T\) is the number of observations (number of scenarios).

• \(p_i\) is the prior probability of scenario \(x_i\).

• \(q_i\) is the posterior probability of scenario \(x_i\).

• \(X\) is the scenario matrix of shape (n_observations, n_assets).

• \(f_j\) is the j ^th view function.

• \(v_j\) is the target value imposed by the j ^th view.

• \(k\) is the total number of views.

The skfolio implementation supports the following views:

• Equalities

• Inequalities

• Ranking

• Linear combinations (e.g. relative views)

• Views on groups of assets

On the following measures:

• Mean

• Variance

• Skew

• Kurtosis

• Correlation

• Value-at-Risk (VaR)

• Conditional Value-at-Risk (CVaR)

Parameters:

prior_estimatorBasePrior, optional

Estimator of the asset’s prior distribution, fitted from a prior estimator. The default (None) is to use the empirical prior EmpiricalPrior(). To perform Entropy Pooling on synthetic data, you can use SyntheticData by setting prior_estimator = SyntheticData().

mean_viewslist[str], optional

Views on asset means. The views must match any of following patterns:

• "ref1 >= a"

• "ref1 == b"

• "ref1 <= ref1"

• "ref1 >= a * prior(ref1)"

• "ref1 == b * prior(ref2)"

• "a * ref1 + b * ref2 + c <= d * ref3"

With "ref1", "ref2" … the assets names or the groups names provided in the parameter groups. Assets names can be referenced without the need of groups if the input X of the fit method is a DataFrame with assets names in columns. Otherwise, the default asset names x0, x1, ... are assigned. By using the term prior(...), you can reference the asset prior mean.

For example:

• "SPX >= 0.0015" –> The mean of SPX is greater than 0.15% (daily mean if X is daily)

• "SX5E == 0.002" –> The mean of SX5E equals 0.2%

• "AAPL <= 0.003" –> The mean of AAPL is less than to 0.3%

• "SPX <= SX5E" –> Ranking view: the mean of SPX is less than SX5E

• "SPX >= 1.5 * prior(SPX)" –> The mean of SPX increases by at least 50% (versus its prior)

• "SX5E == 2 * prior(SX5E)" –> The mean of SX5E doubles (versus its prior)

• "AAPL <= 0.8 * prior(SPX)" –> The mean of AAPL is less than 0.8 times the SPX prior

• "SX5E + SPX >= 0" –> The sum of SX5E and SPX mean is greater than zero

• "US == 0.007" –> The sum of means of US assets equals 0.7%

• "Equity == 3 * Bond" –> The sum of means of Equity assets equals three times the sum of means of Bond assets.

• "2*SPX + 3*Europe <= Bond + 0.05" –> Mixing assets and group mean views

variance_viewslist[str], optional

Views on asset variances. It supports the same patterns as mean_views.

For example:

• "SPX >= 0.0009" –> SPX variance is greater than 0.0009 (daily)

• "SX5E == 1.5 * prior(SX5E)" –> SX5E variance increases by 150% (versus its prior)

skew_viewslist[str], optional

Views on asset skews. It supports the same patterns as mean_views.

For example:

• "SPX >= 2.0" –> SPX skew is greater than 2.0

• "SX5E == 1.5 * prior(SX5E)" –> SX5E skew increases by 150% (versus its prior)

kurtosis_viewslist[str], optional

Views on asset kurtosis. It supports the same patterns as mean_views.

For example:

• "SPX >= 9.0" –> SPX kurtosis is greater than 9.0

• "SX5E == 1.5 * prior(SX5E)" –> SX5E kurtosis increases by 150% (versus its prior)

correlation_viewslist[str], optional

Views on asset correlations. The views must match any of following patterns:

• "(asset1, asset2) >= a"

• "(asset1, asset2) == a"

• "(asset1, asset2) <= a"

• "(asset1, asset2) >= a * prior(asset1, asset2)"

• "(asset1, asset2) == a * prior(asset1, asset2)"

• "(asset1, asset2) <= a * prior(asset1, asset2)"

For example:

• "(SPX, SX5E) >= 0.8" –> the correlation between SPX and SX5E is greater than 80%

• "(SPX, SX5E) == 1.5 * prior(SPX, SX5E)" –> the correlation between SPX and SX5E increases by 150% (versus its prior)

value_at_risk_viewslist[str], optional

Views on asset Value-at-Risks (VaR).

For example:

• "SPX >= 0.03" –> SPX VaR is greater than 3%

• "SX5E == 1.5 * prior(SX5E)" –> SX5E VaR increases by 150% (versus its prior)

cvar_viewslist[str], optional

Views on asset Conditional Value-at-Risks (CVaR). It only supports equalities.

For example:

• "SPX == 0.05" –> SPX CVaR equals 5%

• "SX5E == 1.5 * prior(SX5E)" –> SX5E CVaR increases by 150% (versus its prior)

value_at_risk_betafloat, default=0.95

Confidence level for VaR views, by default 95%.

cvar_betafloat, default=0.95

Confidence level for CVaR views, by default 95%.

groupsdict[str, list[str]] or array-like of strings of shape (n_groups, n_assets), optional

Asset grouping for use in group-based views. If a dict is provided, keys are asset names and values are lists of group labels; then X must be a DataFrame whose columns match those asset names.

For example:

• groups = {"SX5E": ["Equity", "Europe"], "SPX": ["Equity", "US"], "TLT": ["Bond", "US"]}

• groups = [["Equity", "Equity", "Bond"], ["Europe", "US", "US"]]

solverstr, default=”TNC”

The solver to use.

• “TNC” (default) solves the entropic-pooling dual via SciPy’s Truncated Newton Constrained method. By exploiting the smooth Fenchel dual and its closed-form gradient, it operates in \(\mathbb{R}^k\) (the number of constraints) rather than \(\mathbb{R}^T\) (the number of scenarios), yielding an order-of-magnitude speedup over primal CVXPY interior-point solvers.

• CVXPY solvers (e.g. “CLARABEL”) solve the entropic-pooling problem in its primal form using interior-point methods. While they tend to be slower than the dual-based approach, they often achieve higher accuracy by enforcing stricter primal feasibility and duality-gap tolerances. See the CVXPY documentation for supported solvers: https://www.cvxpy.org/tutorial/advanced/index.html#choosing-a-solver.

solver_paramsdict, optional

Additional parameters to pass to the chosen solver.

• When using SciPy TNC, supported options include (but are not limited to) gtol, ftol, eps, maxfun, maxCGit, stepmx, disp. See the SciPy documentation for a full list and descriptions: https://docs.scipy.org/doc/scipy/reference/optimize.minimize-tnc.html

• When using a CVXPY solver (e.g. "CLARABEL"), supply any solver-specific parameters here. Refer to the CVXPY solver guide for details: https://www.cvxpy.org/tutorial/solvers

Attributes:

return_distribution_ReturnDistribution

Fitted ReturnDistribution to be used by the optimization estimators, containing the assets distribution, moments estimation and the EP posterior probabilities (sample weights).

relative_entropy_float

The KL-divergence between the posterior and prior distributions.

effective_number_of_scenarios_float

Effective number of scenarios defined as the perplexity of sample weight (exponential of entropy).

prior_estimator_BasePrior

Fitted prior_estimator.

n_features_in_int

Number of assets seen during fit.

feature_names_in_ndarray of shape (n_features_in_,)

Names of features seen during fit. Defined only when X has feature names that are all strings.

Notes

Entropy Pooling re-weights the sample probabilities of the prior distribution and is therefore constrained by the support (completeness) of that distribution. For example, if the historical distribution contains no returns below -10% for a given asset, we cannot impose a CVaR view of 15%: no matter how we adjust the sample probabilities, such tail data do not exist.

Therefore, to impose extreme views on a sparse historical distribution, one must generate synthetic data. In that case, the EP posterior is only as reliable as the synthetic scenarios. It is thus essential to use a generator capable of extrapolating tail dependencies, such as VineCopula, to model joint extreme events accurately.

Two methods are available:

• Dual form: solves the Fenchel dual of the EP problem using Truncated Newton Constrained method.

• Primal form: solves the original relative-entropy projection in probability-space via interior-point algorithms.

See the solver parameter’s docstring for full details on available solvers and options.

To handle nonlinear views, constraints are linearized by fixing the relevant asset moments (e.g., means or variances) and then solved via nested entropic tilting. At each stage, the KL-divergence is minimized relative to the original prior, while nesting all previously enforced (linearized) views into the feasible set:

• Stage 1: impose views on asset means, VaR and CVaR.

• Stage 2: carry forward Stage 1 constraints and add variance, fixing the mean at its Stage 1 value.

• Stage 3: carry forward Stage 2 constraints and add skewness, kurtosis and pairwise correlations, fixing both mean and variance at their Stage 2 values.

Because each entropic projection nests the prior views, every constraint from earlier stages is preserved as new ones are added, yielding a final distribution that satisfies all original nonlinear views while staying as close as possible to the original prior.

Only the necessary moments are fixed. Slack variables with an L1 norm penalty are introduced to avoid solver infeasibility that may arise from overly tight constraints.

CVaR view constraints cannot be directly expressed as linear functions of the posterior probabilities. Therefore, when CVaR views are present, the EP problem is solved by recursively solving a series of convex programs that approximate the nonlinear CVaR constraint.

This implementation improves upon Meucci’s algorithm [1] by formulating the problem in continuous space as a function of the dual variables etas (VaR levels), rather than searching over discrete tail sizes. This formulation not only handles the CVaR constraint more directly but also supports multiple CVaR views on different assets.

Although the overall problem is convex in the dual variables etas, it remains non-smooth due to the presence of the positive-part operator in the CVaR definition. Consequently, we employ derivative-free optimization methods. Specifically, for a single CVaR view we use a one-dimensional root-finding method (Brent’s method), and for the multivariate case (supporting multiple CVaR views) we use Powell’s method for derivative-free convex descent.

References

[1]

“Fully Flexible Extreme Views”, Journal of Risk, Meucci, Ardia & Keel (2011)

[2]

“Fully Flexible Views: Theory and Practice”, Risk, Meucci (2013).

[3]

“Effective Number of Scenarios in Fully Flexible Probabilities”, GARP Risk Professional, Meucci (2012)

[4]

“I-Divergence Geometry of Probability Distributions and Minimization Problems”, The Annals of Probability, Csiszar (1975)

Examples

For a full tutorial on entropy pooling, see Entropy Pooling.

>>> from skfolio import RiskMeasure
>>> from skfolio.datasets import load_sp500_dataset
>>> from skfolio.preprocessing import prices_to_returns
>>> from skfolio.prior import EntropyPooling
>>> from skfolio.optimization import HierarchicalRiskParity
>>>
>>> prices = load_sp500_dataset()
>>> prices = prices[["AMD", "BAC", "GE", "JNJ", "JPM", "LLY", "PG"]]
>>> X = prices_to_returns(prices)
>>>
>>> groups = {
...     "AMD": ["Technology", "Growth"],
...     "BAC": ["Financials", "Value"],
...     "GE": ["Industrials", "Value"],
...     "JNJ": ["Healthcare", "Defensive"],
...     "JPM": ["Financials", "Income"],
...     "LLY": ["Healthcare", "Defensive"],
...     "PG": ["Consumer", "Defensive"],
... }
>>>
>>> entropy_pooling = EntropyPooling(
...     mean_views=[
...         "JPM == -0.002",
...         "PG >= LLY",
...         "BAC >= prior(BAC) * 1.2",
...         "Financials == 2 * Growth",
...     ],
...     variance_views=[
...         "BAC == prior(BAC) * 4",
...     ],
...     correlation_views=[
...         "(BAC,JPM) == 0.80",
...         "(BAC,JNJ) <= prior(BAC,JNJ) * 0.5",
...     ],
...     skew_views=[
...         "BAC == -0.05",
...     ],
...     cvar_views=[
...         "GE == 0.08",
...     ],
...     cvar_beta=0.90,
...     groups=groups,
... )
>>>
>>> entropy_pooling.fit(X)
EntropyPooling(correlation_views=...
>>>
>>> print(entropy_pooling.relative_entropy_)
0.18...
>>> print(entropy_pooling.effective_number_of_scenarios_)
6876.67...
>>> print(entropy_pooling.return_distribution_.sample_weight)
[0.000103...  0.000093... ... 0.000103...  0.000108...]
>>>
>>> # CVaR Hierarchical Risk Parity optimization on Entropy Pooling
>>> model = HierarchicalRiskParity(
...     risk_measure=RiskMeasure.CVAR,
...     prior_estimator=entropy_pooling
... )
>>> model.fit(X)
HierarchicalRiskParity(prior_estimator=...
>>> print(model.weights_)
[0.073... 0.0541... ... 0.200...]
>>>
>>> # Stress Test the Portfolio
>>> entropy_pooling = EntropyPooling(cvar_views=["AMD == 0.10"])
>>> entropy_pooling.fit(X)
EntropyPooling(cvar_views=['AMD == 0.10'])
>>>
>>> stressed_dist = entropy_pooling.return_distribution_
>>>
>>> stressed_ptf = model.predict(stressed_dist)

Methods

fit(X[, y])	Fit the Entropy Pooling estimator.
get_metadata_routing()	Get metadata routing of this object.
get_params([deep])	Get parameters for this estimator.
set_params(**params)	Set the parameters of this estimator.

fit(X, y=None, **fit_params)

Fit the Entropy Pooling estimator.

Parameters:

Xarray-like of shape (n_observations, n_assets)

Price returns of the assets.

yIgnored

Not used, present for API consistency by convention.

**fit_paramsdict

Parameters to pass to the underlying estimators. Only available if enable_metadata_routing=True, which can be set by using sklearn.set_config(enable_metadata_routing=True). See Metadata Routing User Guide for more details.

Returns:

selfEntropyPooling

Fitted estimator.

get_metadata_routing()

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:

routingMetadataRequest

A MetadataRequest encapsulating routing information.

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

paramsdict

Parameter names mapped to their values.

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**paramsdict

Estimator parameters.

Returns:

selfestimator instance

Estimator instance.