# Hierarchical Risk Parity - CVaR
This tutorial introduces the [`HierarchicalRiskParity`](https://skfolio.org/generated/skfolio.optimization.HierarchicalRiskParity.html.md#skfolio.optimization.HierarchicalRiskParity)
optimization.
Hierarchical Risk Parity (HRP) is a portfolio optimization method developed by Marcos
Lopez de Prado.
This algorithm uses a distance matrix to compute hierarchical clusters using the
Hierarchical Tree Clustering algorithm. It then employs seriation to rearrange the
assets in the dendrogram, minimizing the distance between leaves.
The final step is the recursive bisection where each cluster is split between two
sub-clusters by starting with the topmost cluster and traversing in a top-down
manner. For each sub-cluster, we compute the total cluster risk of an inverse-risk
allocation. A weighting factor is then computed from these two sub-cluster risks,
which is used to update the cluster weight.
#### NOTE
The original paper uses the variance as the risk measure and the single-linkage
method for the Hierarchical Tree Clustering algorithm. Here we generalize it to
multiple risk measures and linkage methods.
The default linkage method is set to the Ward
variance minimization algorithm, which is more stable and has better properties
than the single-linkage method.
In this example, we will use the CVaR risk measure.
## Data
We load the S&P 500 [dataset](https://skfolio.org/user_guide/datasets.html.md#datasets) composed of the daily prices of 20
assets from the SPX Index composition and the Factors dataset composed of the daily
prices of 5 ETFs representing common factors:
```Python
from plotly.io import show
from sklearn.model_selection import train_test_split
from skfolio import Population, RiskMeasure
from skfolio.cluster import HierarchicalClustering, LinkageMethod
from skfolio.datasets import load_factors_dataset, load_sp500_dataset
from skfolio.distance import KendallDistance
from skfolio.optimization import EqualWeighted, HierarchicalRiskParity
from skfolio.preprocessing import prices_to_returns
from skfolio.prior import TimeSeriesFactorModel
prices = load_sp500_dataset()
factor_prices = load_factors_dataset()
prices = prices["2014":]
factor_prices = factor_prices["2014":]
X, factors = prices_to_returns(prices, factor_prices)
X_train, X_test, factors_train, factors_test = train_test_split(
X, factors, test_size=0.33, shuffle=False
)
```
## Model
We create the CVaR Hierarchical Risk Parity model and then fit it on the training set:
```Python
model1 = HierarchicalRiskParity(
risk_measure=RiskMeasure.CVAR, portfolio_params=dict(name="HRP-CVaR-Ward-Pearson")
)
model1.fit(X_train)
model1.weights_
```
```none
array([0.05033705, 0.02773558, 0.05289115, 0.03632272, 0.059202 ,
0.02483767, 0.03790179, 0.07464383, 0.03497807, 0.08622477,
0.06308422, 0.04094166, 0.03144452, 0.08277551, 0.04421773,
0.04807705, 0.02596219, 0.07596741, 0.0393462 , 0.06310889])
```
## Risk Contribution
Let’s analyze the risk contribution of the model on the training set:
```Python
ptf1 = model1.predict(X_train)
ptf1.plot_contribution(measure=RiskMeasure.CVAR)
```
[plotly figure stripped from llms output]
## Dendrogram
To analyze the clusters structure, we plot the dendrogram.
The blue lines represent distinct clusters composed of a single asset.
The remaining colors represent clusters of more than one asset:
```Python
model1.hierarchical_clustering_estimator_.plot_dendrogram(heatmap=False)
```
[plotly figure stripped from llms output]
The horizontal axis represents the assets. The links between clusters are represented
as upside-down U-shaped lines. The height of the U indicates the distance between the
clusters. For example, the link representing the cluster containing assets HD and WMT
has a distance of 0.5 (called cophenetic distance).
When `heatmap` is set to True, the heatmap of the reordered distance matrix is
displayed below the dendrogram and clusters are outlined with yellow squares:
```Python
fig = model1.hierarchical_clustering_estimator_.plot_dendrogram()
show(fig)
```
[plotly figure stripped from llms output]
## Linkage Methods
The clustering can be greatly affected by the choice of the linkage method.
The original HRP is based on the single-linkage (equivalent to the minimum spanning
tree), which suffers from the chaining effect.
In the [`HierarchicalRiskParity`](https://skfolio.org/generated/skfolio.optimization.HierarchicalRiskParity.html.md#skfolio.optimization.HierarchicalRiskParity) estimator, the default
linkage method is set to the Ward variance minimization algorithm, which is more
stable and has better properties than the single-linkage method.
However, since the HRP optimization doesn’t utilize the full cluster structure but
only their orders, the allocation remains relatively stable regardless of the chosen
linkage method.
```Python
# To show this effect, let's create a second model with the single-linkage method:
model2 = HierarchicalRiskParity(
risk_measure=RiskMeasure.CVAR,
hierarchical_clustering_estimator=HierarchicalClustering(
linkage_method=LinkageMethod.SINGLE,
),
portfolio_params=dict(name="HRP-CVaR-Single-Pearson"),
)
model2.fit(X_train)
model2.hierarchical_clustering_estimator_.plot_dendrogram(heatmap=True)
```
[plotly figure stripped from llms output]
We can see that the clustering has been greatly affected by the change of the linkage
method. However, you will see below that the weights remain relatively stable for the
reason explained earlier.
## Distance Estimator
The choice of distance metric also has an important effect on the clustering.
The default is to use the distance from the pearson correlation matrix.
This can be changed using the [distance estimators](https://skfolio.org/user_guide/distance.html.md#distance).
For example, let’s create a third model with a distance computed from the absolute
value of the Kendal correlation matrix:
```Python
model3 = HierarchicalRiskParity(
risk_measure=RiskMeasure.CVAR,
distance_estimator=KendallDistance(absolute=True),
portfolio_params=dict(name="HRP-CVaR-Ward-Kendal"),
)
model3.fit(X_train)
model3.hierarchical_clustering_estimator_.plot_dendrogram(heatmap=True)
```
[plotly figure stripped from llms output]
## Prior Estimator
Finally, HRP like the other portfolio optimization, uses a
[prior estimator](https://skfolio.org/user_guide/prior.html.md#prior) that fits a [`ReturnDistribution`](https://skfolio.org/generated/skfolio.prior.ReturnDistribution.html.md#skfolio.prior.ReturnDistribution)
containing the distribution estimate of asset returns.
The default is the [`EmpiricalPrior`](https://skfolio.org/generated/skfolio.prior.EmpiricalPrior.html.md#skfolio.prior.EmpiricalPrior) estimator.
Let’s create new model with the [`TimeSeriesFactorModel`](https://skfolio.org/generated/skfolio.prior.TimeSeriesFactorModel.html.md#skfolio.prior.TimeSeriesFactorModel) estimator:
```Python
model4 = HierarchicalRiskParity(
risk_measure=RiskMeasure.CVAR,
prior_estimator=TimeSeriesFactorModel(),
portfolio_params=dict(name="HRP-CVaR-Factor-Model"),
)
model4.fit(X_train, factors=factors_train)
model4.hierarchical_clustering_estimator_.plot_dendrogram(heatmap=True)
```
[plotly figure stripped from llms output]
To compare the models, we use an equal weighted benchmark using
the [`EqualWeighted`](https://skfolio.org/generated/skfolio.optimization.EqualWeighted.html.md#skfolio.optimization.EqualWeighted) estimator:
```Python
bench = EqualWeighted()
bench.fit(X_train)
bench.weights_
```
```none
array([0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05,
0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05])
```
## Prediction
We predict the models and the benchmark on the test set:
```Python
population_test = Population([])
for model in [model1, model2, model3, model4, bench]:
population_test.append(model.predict(X_test))
population_test.plot_cumulative_returns()
```
[plotly figure stripped from llms output]
## Composition
From the below composition, we notice that all models are relatively close to each
other, as explained earlier:
```Python
population_test.plot_composition()
```
[plotly figure stripped from llms output]
## Summary
Finally, let’s print the summary statistics:
```Python
summary = population_test.summary()
summary.loc["Annualized Sharpe Ratio"]
```
```none
HRP-CVaR-Ward-Pearson 0.86
HRP-CVaR-Single-Pearson 0.84
HRP-CVaR-Ward-Kendal 0.86
HRP-CVaR-Factor-Model 0.87
EqualWeighted 0.86
Name: Annualized Sharpe Ratio, dtype: str
```
```Python
summary
```
|
HRP-CVaR-Ward-Pearson |
HRP-CVaR-Single-Pearson |
HRP-CVaR-Ward-Kendal |
HRP-CVaR-Factor-Model |
EqualWeighted |
| Mean |
0.081% |
0.079% |
0.079% |
0.081% |
0.084% |
| Annualized Mean |
20.41% |
19.81% |
19.95% |
20.29% |
21.27% |
| Variance |
0.00022 |
0.00022 |
0.00022 |
0.00022 |
0.00024 |
| Annualized Variance |
5.62% |
5.51% |
5.42% |
5.47% |
6.13% |
| Semi-Variance |
0.00011 |
0.00011 |
0.00011 |
0.00011 |
0.00012 |
| Annualized Semi-Variance |
2.89% |
2.82% |
2.78% |
2.78% |
3.11% |
| Standard Deviation |
1.49% |
1.48% |
1.47% |
1.47% |
1.56% |
| Annualized Standard Deviation |
23.71% |
23.48% |
23.29% |
23.38% |
24.75% |
| Semi-Deviation |
1.07% |
1.06% |
1.05% |
1.05% |
1.11% |
| Annualized Semi-Deviation |
16.99% |
16.78% |
16.69% |
16.66% |
17.63% |
| Mean Absolute Deviation |
0.93% |
0.92% |
0.91% |
0.92% |
0.99% |
| CVaR at 95% |
3.51% |
3.44% |
3.43% |
3.42% |
3.65% |
| EVaR at 95% |
6.51% |
6.51% |
6.47% |
6.39% |
6.58% |
| Worst Realization |
10.64% |
10.70% |
10.66% |
10.50% |
10.77% |
| CDaR at 95% |
17.37% |
17.36% |
16.94% |
17.78% |
18.54% |
| MAX Drawdown |
35.36% |
34.19% |
33.56% |
34.03% |
34.70% |
| Average Drawdown |
3.00% |
3.14% |
3.05% |
3.28% |
3.38% |
| EDaR at 95% |
24.28% |
23.72% |
23.32% |
23.86% |
24.47% |
| First Lower Partial Moment |
0.46% |
0.46% |
0.46% |
0.46% |
0.50% |
| Ulcer Index |
0.053 |
0.054 |
0.052 |
0.056 |
0.058 |
| Gini Mean Difference |
1.42% |
1.40% |
1.40% |
1.41% |
1.51% |
| Value at Risk at 95% |
2.02% |
1.93% |
1.97% |
1.97% |
2.15% |
| Drawdown at Risk at 95% |
10.14% |
10.90% |
10.35% |
11.49% |
12.03% |
| Entropic Risk Measure at 95% |
3.00 |
3.00 |
3.00 |
3.00 |
3.00 |
| Fourth Central Moment |
0.000079% |
0.000078% |
0.000073% |
0.000074% |
0.000090% |
| Fourth Lower Partial Moment |
0.000041% |
0.000040% |
0.000039% |
0.000037% |
0.000043% |
| Skew |
-8.53% |
-6.33% |
-11.58% |
-1.42% |
4.61% |
| Kurtosis |
1593.00% |
1634.75% |
1581.88% |
1569.59% |
1530.96% |
| Sharpe Ratio |
0.054 |
0.053 |
0.054 |
0.055 |
0.054 |
| Annualized Sharpe Ratio |
0.86 |
0.84 |
0.86 |
0.87 |
0.86 |
| Sortino Ratio |
0.076 |
0.074 |
0.075 |
0.077 |
0.076 |
| Annualized Sortino Ratio |
1.20 |
1.18 |
1.20 |
1.22 |
1.21 |
| Mean Absolute Deviation Ratio |
0.087 |
0.086 |
0.087 |
0.087 |
0.085 |
| First Lower Partial Moment Ratio |
0.17 |
0.17 |
0.17 |
0.17 |
0.17 |
| Value at Risk Ratio at 95% |
0.040 |
0.041 |
0.040 |
0.041 |
0.039 |
| CVaR Ratio at 95% |
0.023 |
0.023 |
0.023 |
0.024 |
0.023 |
| Entropic Risk Measure Ratio at 95% |
0.00027 |
0.00026 |
0.00026 |
0.00027 |
0.00028 |
| EVaR Ratio at 95% |
0.012 |
0.012 |
0.012 |
0.013 |
0.013 |
| Worst Realization Ratio |
0.0076 |
0.0073 |
0.0074 |
0.0077 |
0.0078 |
| Drawdown at Risk Ratio at 95% |
0.0080 |
0.0072 |
0.0076 |
0.0070 |
0.0070 |
| CDaR Ratio at 95% |
0.0047 |
0.0045 |
0.0047 |
0.0045 |
0.0046 |
| Calmar Ratio |
0.0023 |
0.0023 |
0.0024 |
0.0024 |
0.0024 |
| Average Drawdown Ratio |
0.027 |
0.025 |
0.026 |
0.025 |
0.025 |
| EDaR Ratio at 95% |
0.0033 |
0.0033 |
0.0034 |
0.0034 |
0.0034 |
| Ulcer Index Ratio |
0.015 |
0.015 |
0.015 |
0.014 |
0.015 |
| Gini Mean Difference Ratio |
0.057 |
0.056 |
0.057 |
0.057 |
0.056 |
| Effective Number of Assets |
17.56017723500113 |
17.30105753780178 |
17.533289375090764 |
17.272263586429105 |
19.999999999999993 |
| Assets Number |
20 |
20 |
20 |
20 |
20 |
**Total running time of the script:** (0 minutes 3.067 seconds)