Note
Go to the end to download the full example code. or to run this example in your browser via JupyterLite or Binder
Failure and Fallbacks#
This tutorial introduces the optimization parameters fallback
and raise_on_failure
.
Optimization can sometimes fail during a given rebalancing. For example, a convex mean-variance problem with strict risk or sector constraints may become infeasible on specific dates. Such failures must be handled explicitly depending on the use case (production vs. research).
Fallback#
The fallback
parameter lets you define an estimator, or a list of estimators, to try
in order when the primary optimization raises an error during fit
. Alternatively, you
can use "previous_weights"
to reuse the last valid allocation.
Each attempt is recorded in fallback_chain_
, and the successful estimator is available
through fallback_
.
This mechanism is essential in automated pipelines, ensuring that optimization failures never halt production runs while preserving full reproducibility and traceability. Beyond safeguarding workflows, it can also be used to deliberately relax constraints in a controlled manner when strict convergence cannot be achieved.
Raise on Failure#
In research, cross-validation and hyperparameter tuning (e.g. walk-forward, multiple randomized cross-validation), it’s often useful to let all runs complete while keeping a full record of failures instead of stopping on the first failed rebalancing.
Set
raise_on_failure=True
(default) to fail fast. This is useful in production when the primary optimization or the fallback cascade is expected to succeed.Set
raise_on_failure=False
to continue uninterrupted. This is useful in research and cross-validation. When a failure occurs,predict
returns aFailedPortfolio
(think of it as an augmented NaN) that carries diagnostics such asoptimization_error
andfallback_chain
, while remaining API-compatible with downstream analytics.
Data and Setup#
Load the S&P 500 dataset and split into train/test.
import pandas as pd
from plotly.io import show
from sklearn.model_selection import train_test_split
from sklearn.utils.validation import validate_data
from skfolio.datasets import load_sp500_dataset
from skfolio.model_selection import WalkForward, cross_val_predict
from skfolio.optimization import BaseOptimization, EqualWeighted, MeanRisk
from skfolio.preprocessing import prices_to_returns
from skfolio.typing import Fallback, MultiInput
from skfolio.utils.stats import rand_weights
# Load S&P 500 dataset and split train/test
prices = load_sp500_dataset()
prices = prices["2010":]
X = prices_to_returns(prices)
X_train, X_test = train_test_split(X, test_size=0.33, shuffle=False)
Fallback#
Let’s start with a simple example. The primary model is a minimum-variance optimization made intentionally infeasible (the assets’ minimum weights are set to 10%, which exceeds the feasible upper bound of 1/n_assets = 5%). As a fallback, we provide a feasible minimum-variance model with a 2% minimum weight constraint:
[0.02000143 0.02000011 0.02000017 0.02000038 0.02000046 0.02000042
0.02000085 0.12397314 0.02000024 0.09473377 0.02004185 0.02000133
0.02000057 0.17901455 0.0200018 0.17643479 0.02000026 0.02000095
0.12579215 0.02000078]
Diagnostics#
Let’s retrieve the fitted fallback that produced the final result:
print(model.fallback_)
MeanRisk(min_weights=0.02)
Let’s display the sequence of attempts and their outcomes:
print(model.fallback_chain_)
[('MeanRisk(fallback=MeanRisk(min_weights=0.02), min_weights=0.1)', "Solver 'CLARABEL' failed. Try another solver, or solve with solver_params=dict(verbose=True) for more information"), ('MeanRisk(min_weights=0.02)', 'success')]
The fallback audit trail is also propagated to the predicted portfolio:
portfolio = model.predict(X_test)
assert portfolio.fallback_chain == model.fallback_chain_
Multiple fallbacks#
We can also provide a list of fallbacks to be tried in order, including “previous_weights” as a terminal safety net:
model = MeanRisk(
min_weights=0.1,
previous_weights={
"AAPL": 0.4,
"AMD": 0.2,
"UNH": 0.4,
}, # any missing assets default to 0
fallback=[
MeanRisk(min_weights=0.02),
MeanRisk(min_weights=0.01),
EqualWeighted(),
"previous_weights",
],
)
Chaining#
We can also nest fallbacks.
The chain is evaluated depth-first from the primary estimator to the
first successful fallback, recording each attempt in fallback_chain_
.
This is equivalent to providing an ordered list:
model = MeanRisk(
min_weights=0.1,
fallback=MeanRisk(
min_weights=0.02,
fallback=MeanRisk(
min_weights=0.01,
fallback=EqualWeighted(),
),
),
)
Fallback in cross-validation#
Fallback behavior is fully preserved in cross-validation.
When using cross_val_predict
, all diagnostics (e.g., fallback chains and errors)
are propagated to the resulting portfolios in the MultiPeriodPortfolio
:
Each individual
Portfolio
(orFailedPortfolio
) produced during rebalancing carries its ownfallback_chain
andoptimization_error
.Global counts and statistics (e.g., the number of portfolios that required a fallback) are available through summary attributes such as
n_fallback_portfolios
andn_failed_portfolios
.The
summary()
method consolidates performance and diagnostic information across all rebalances.
model = MeanRisk(min_weights=0.1, fallback=MeanRisk(min_weights=0.02))
# Rebalance semiannually on the third Friday (WOM-3FRI), training on the prior 12 months
walk_forward = WalkForward(test_size=6, train_size=12, freq="WOM-3FRI")
pred = cross_val_predict(model, X, cv=walk_forward)
Let’s retrieve the fallback chain of the first portfolio:
print(pred[0].fallback_chain)
[('MeanRisk(fallback=MeanRisk(min_weights=0.02), min_weights=0.1)', "Solver 'CLARABEL' failed. Try another solver, or solve with solver_params=dict(verbose=True) for more information"), ('MeanRisk(min_weights=0.02)', 'success')]
Let’s print the number of portfolios in
MultiPeriodPortfolio
where a fallback was used:
print(pred.n_fallback_portfolios)
23
Finally, let’s display the last four rows of the MultiPeriodPortfolio
summary,
which contain the fallback statistics:
print(pred.summary().iloc[-4:])
Avg nb of Assets per Portfolio 20.0
Number of Portfolios 23
Number of Failed Portfolios 0
Number of Fallback Portfolios 23
dtype: object
Failure handling#
In this section, we show how to handle optimization failures using the
raise_on_failure
parameter.
As an example, we create a custom optimization that intentionally fails during fit
when the first date of the input window falls on an even day of the month, or when
always_fail=True
.
class CustomOptimization(BaseOptimization):
"""Dummy optimization that intentionally fails during `fit` when the first
date of the input window is an even day-of-month, or when `always_fail=True`."""
def __init__(
self,
always_fail: bool = False,
portfolio_params: dict | None = None,
fallback: Fallback = None,
previous_weights: MultiInput | None = None,
raise_on_failure: bool = True,
):
super().__init__(
portfolio_params=portfolio_params,
fallback=fallback,
raise_on_failure=raise_on_failure,
previous_weights=previous_weights,
)
self.always_fail = always_fail
def fit(self, X: pd.DataFrame, y=None):
validate_data(self, X)
# Fail when first observation date has an even day-of-month, or always.
if self.always_fail:
raise RuntimeError("Forced failure")
first_day = X.index[0].day
if first_day % 2 == 0:
raise RuntimeError("Forced failure (even-start window)")
n_assets = X.shape[1]
self.weights_ = rand_weights(n_assets)
return self
By default, as with all scikit-learn estimators, failures raise an error during fit
:
model = CustomOptimization(always_fail=True)
try:
model.fit(X_train)
except RuntimeError as err:
print(err)
Forced failure
By setting raise_on_failure=False
, a warning is emitted instead of raising an error,
and weights_
are set to None
, with the error message stored in error_
:
model = CustomOptimization(always_fail=True, raise_on_failure=False)
model.fit(X_train)
print(model.weights_)
print(model.error_)
None
Forced failure
In this case, calling predict
will return a FailedPortfolio
carrying the audit
trail in optimization_error
and fallback_chain
(if any fallbacks occurred).
portfolio = model.predict(X_test)
print(portfolio)
print(portfolio.optimization_error)
<FailedPortfolio CustomOptimization>
Forced failure
Setting raise_on_failure=False
is useful for cross-validation and hyperparameter
tuning as it allows all runs to complete without stopping at the first rebalancing
failure. Let’s instantiate our custom optimization and run a walk-forward analysis
where failures occur deterministically on even-start windows:
model = CustomOptimization(raise_on_failure=False)
pred = cross_val_predict(model, X, cv=walk_forward)
cross_val_predict
completed without interruption.
The resulting MultiPeriodPortfolio
is composed of both Portfolio
and
FailedPortfolio
objects:
print(pred.portfolios)
[<Portfolio CustomOptimization>, <FailedPortfolio CustomOptimization>, <Portfolio CustomOptimization>, <Portfolio CustomOptimization>, <FailedPortfolio CustomOptimization>, <FailedPortfolio CustomOptimization>, <FailedPortfolio CustomOptimization>, <Portfolio CustomOptimization>, <Portfolio CustomOptimization>, <FailedPortfolio CustomOptimization>, <FailedPortfolio CustomOptimization>, <Portfolio CustomOptimization>, <Portfolio CustomOptimization>, <Portfolio CustomOptimization>, <FailedPortfolio CustomOptimization>, <Portfolio CustomOptimization>, <Portfolio CustomOptimization>, <FailedPortfolio CustomOptimization>, <FailedPortfolio CustomOptimization>, <Portfolio CustomOptimization>, <Portfolio CustomOptimization>, <Portfolio CustomOptimization>, <Portfolio CustomOptimization>]
Let’s print the number of failed portfolios:
print(pred.n_failed_portfolios)
9
Even though MultiPeriodPortfolio
contains failed portfolios, all statistics and
plots still work properly. This is because FailedPortfolio
is designed to behave
like non-propagating NaNs:
print(pred.summary())
Mean 0.068%
Annualized Mean 17.16%
Variance 0.0074%
Annualized Variance 1.87%
Semi-Variance 0.0040%
Annualized Semi-Variance 1.00%
Standard Deviation 0.86%
Annualized Standard Deviation 13.69%
Semi-Deviation 0.63%
Annualized Semi-Deviation 10.00%
Mean Absolute Deviation 0.62%
CVaR at 95% 2.05%
EVaR at 95% 2.72%
Worst Realization 4.37%
CDaR at 95% 13.27%
MAX Drawdown 24.39%
Average Drawdown 2.42%
EDaR at 95% 16.19%
First Lower Partial Moment 0.31%
Ulcer Index 0.041
Gini Mean Difference 0.92%
Value at Risk at 95% 1.40%
Drawdown at Risk at 95% 8.92%
Entropic Risk Measure at 95% 3.00
Fourth Central Moment 0.000003%
Fourth Lower Partial Moment 0.000002%
Skew -28.75%
Kurtosis 579.85%
Sharpe Ratio 0.079
Annualized Sharpe Ratio 1.25
Sortino Ratio 0.11
Annualized Sortino Ratio 1.72
Mean Absolute Deviation Ratio 0.11
First Lower Partial Moment Ratio 0.22
Value at Risk Ratio at 95% 0.049
CVaR Ratio at 95% 0.033
Entropic Risk Measure Ratio at 95% 0.00023
EVaR Ratio at 95% 0.025
Worst Realization Ratio 0.016
Drawdown at Risk Ratio at 95% 0.0076
CDaR Ratio at 95% 0.0051
Calmar Ratio 0.0028
Average Drawdown Ratio 0.028
EDaR Ratio at 95% 0.0042
Ulcer Index Ratio 0.017
Gini Mean Difference Ratio 0.074
Avg nb of Assets per Portfolio 20.0
Number of Portfolios 23
Number of Failed Portfolios 9
Number of Fallback Portfolios 0
dtype: object
As shown below, MultiPeriodPortfolio
plots gracefully handle FailedPortfolio
instances; for cumulative returns, these appear as gaps corresponding to failed
periods:
fig = pred.plot_cumulative_returns()
show(fig)
Finally, let’s inspect the first failed portfolio:
failed_ptf = pred.failed_portfolios[0]
print(failed_ptf.optimization_error)
Forced failure (even-start window)
To replay the optimization on the failed period, we can run:
# model.fit(failed_ptf.X)
Total running time of the script: (0 minutes 2.682 seconds)