#### NOTE
[Go to the end](#sphx-glr-download-auto-examples-mean-risk-plot-17-failure-and-fallbacks-py)
to download the full example code or to run this example in your browser via JupyterLite.

<a id="sphx-glr-auto-examples-mean-risk-plot-17-failure-and-fallbacks-py"></a>

<a id="failure-and-fallbacks"></a>

# Failure and Fallbacks

This tutorial introduces the optimization parameters `fallback` and `raise_on_failure`.

Optimization can sometimes fail during a given rebalancing. For example, a convex
mean-variance problem with strict risk or sector constraints may become infeasible on
specific dates. Such failures must be handled explicitly depending on the use case
(production vs. research).

<a id="fallback"></a>

## Fallback

The `fallback` parameter lets you define an estimator, or a list of estimators, to try
in order when the primary optimization raises an error during `fit`. Alternatively, you
can use `"previous_weights"` to reuse the last valid allocation.

Each attempt is recorded in `fallback_chain_`, and the successful estimator is available
through `fallback_`.

This mechanism is essential in automated pipelines, ensuring that optimization failures
never halt production runs while preserving full reproducibility and traceability.
Beyond safeguarding workflows, it can also be used to deliberately relax constraints in
a controlled manner when strict convergence cannot be achieved.

If every fallback fails, `raise_on_failure` determines whether the final error is raised
or recorded. See [Fallbacks](https://skfolio.org/user_guide/optimization.html.md#optimization-fallbacks).

<a id="raise-on-failure"></a>

## Raise on Failure

In research, cross-validation and hyperparameter tuning (e.g. walk-forward, multiple
randomized cross-validation), it’s often useful to let all runs complete while keeping
a full record of failures instead of stopping on the first failed rebalancing.

- Set `raise_on_failure=True` (default) to fail fast after the configured
  fallbacks are exhausted. This is useful in production when the primary optimization or
  fallback chain is expected to succeed. If neither succeeds, the final error is raised.
- Set `raise_on_failure=False` to continue research and cross-validation after
  failed fits. When no fallback succeeds, a warning is emitted and `weights_` is set to
  None. `predict` returns a [`FailedPortfolio`](https://skfolio.org/generated/skfolio.portfolio.FailedPortfolio.html.md#skfolio.portfolio.FailedPortfolio) (think of it as
  an augmented NaN) that carries diagnostics such as `optimization_error` and
  `fallback_chain`, while remaining API-compatible with downstream analytics. Failed
  periods remain visible in the evaluation timeline.

See [Failure Handling](https://skfolio.org/user_guide/optimization.html.md#optimization-failure-handling) for diagnostics and multiple-portfolio results.
Online `partial_fit` handles only optimization failures after updating the prior and
other estimators. See [Updates and Failure Handling](https://skfolio.org/user_guide/online_learning.html.md#online-failure-handling).

<a id="data-and-setup"></a>

## Data and Setup

Load the S&P 500 [dataset](https://skfolio.org/user_guide/datasets.html.md#datasets) and split into train/test.

```Python
import pandas as pd
from plotly.io import show
from sklearn.base import clone
from sklearn.model_selection import train_test_split
from sklearn.utils.validation import validate_data

from skfolio import RiskMeasure
from skfolio.datasets import load_sp500_dataset
from skfolio.model_selection import WalkForward, cross_val_predict
from skfolio.optimization import (
    BaseOptimization,
    EqualWeighted,
    MeanRisk,
    RiskBudgeting,
)
from skfolio.preprocessing import prices_to_returns
from skfolio.prior import EntropyPooling
from skfolio.typing import Fallback, MultiInput
from skfolio.utils.stats import rand_weights

# Load S&P 500 dataset and split train/test
prices = load_sp500_dataset()
prices = prices["2010":]
X = prices_to_returns(prices)
X_train, X_test = train_test_split(X, test_size=0.33, shuffle=False)
```

<a id="id1"></a>

## Fallback

Let’s start with a simple example.
For 20 assets, a 10% minimum weight per asset requires a total weight of at least
200%, making a fully invested portfolio infeasible. The fallback uses a feasible
2% minimum weight per asset:

```Python
model = MeanRisk(
    min_weights=0.1,  # intentionally infeasible
    fallback=MeanRisk(min_weights=0.02),  # feasible fallback
)
model.fit(X_train)
print(model.weights_)
```

```none
[0.02000143 0.02000011 0.02000017 0.02000038 0.02000046 0.02000042
 0.02000085 0.12397314 0.02000024 0.09473377 0.02004185 0.02000133
 0.02000057 0.17901455 0.0200018  0.17643479 0.02000026 0.02000095
 0.12579215 0.02000078]
```

<a id="diagnostics"></a>

### Diagnostics

Let’s retrieve the fitted fallback that produced the final result:

```Python
print(model.fallback_)
```

```none
MeanRisk(min_weights=0.02)
```

Let’s display the sequence of attempts and their outcomes:

```Python
print(model.fallback_chain_)
```

```none
[('MeanRisk(fallback=MeanRisk(min_weights=0.02), min_weights=0.1)', "Solver 'CLARABEL' failed. Try another solver, or solve with solver_params=dict(verbose=True) for more information"), ('MeanRisk(min_weights=0.02)', 'success')]
```

The fallback audit trail is also propagated to the predicted portfolio:

```Python
portfolio = model.predict(X_test)
assert portfolio.fallback_chain == model.fallback_chain_
```

<a id="falling-back-to-another-solver"></a>

### Falling back to another solver

A solver can encounter numerical difficulties even when a problem is feasible.
Here, entropy pooling concentrates scenario probabilities, making CVaR risk budgeting
difficult for CLARABEL. We use the full price history and configure a fallback that
refits the same estimator with SCS, using its own tolerances and iteration limit.
Install the optional solver with `pip install scs`.

```Python
X_full = prices_to_returns(load_sp500_dataset())
model = RiskBudgeting(
    risk_measure=RiskMeasure.CVAR,
    prior_estimator=EntropyPooling(
        mean_views=["AMD >= BAC", "JPM <= prior(JPM) * 0.8"],
        cvar_views=["GE == 0.12"],
    ),
)
model.set_params(
    fallback=clone(model).set_params(
        solver="SCS",
        solver_params={"eps_abs": 1e-6, "eps_rel": 1e-6, "max_iters": 100_000},
    )
)
model.fit(X_full)
```

[plotly figure stripped from llms output]<style>html[data-theme="dark"] div.output_subarea:has(.plotly-graph-div){background:#fff;border-radius:0.25rem;padding:0.5rem}@media (prefers-color-scheme: dark){html:not([data-theme="light"]) div.output_subarea:has(.plotly-graph-div){background:#fff;border-radius:0.25rem;padding:0.5rem}}</style><script>if (!window.plotlySphinxGalleryResize) {window.plotlySphinxGalleryResize = true;window.addEventListener("load", function () {document.querySelectorAll(".plotly-graph-div").forEach(function (gd) { Plotly.Plots.resize(gd); });});}</script>

<br/>

Finally, let’s inspect the first failed portfolio:

```Python
failed_ptf = pred.failed_portfolios[0]
print(failed_ptf.optimization_error)
```

```none
Forced failure (even-start window)
```

To replay the optimization on the failed period, we can run:

```Python
# model.fit(failed_ptf.X)
```

**Total running time of the script:** (0 minutes 2.869 seconds)

<a id="sphx-glr-download-auto-examples-mean-risk-plot-17-failure-and-fallbacks-py"></a>
[![Launch JupyterLite](auto_examples/mean_risk/images/jupyterlite_badge_logo.svg)](../../lite/lab/index.html?path=auto_examples/mean_risk/plot_17_failure_and_fallbacks.ipynb)

[`Download Jupyter notebook: plot_17_failure_and_fallbacks.ipynb`](https://skfolio.org/auto_examples/mean_risk/_downloads/ee1b703d250baccfd19f66b808ca8cd0/plot_17_failure_and_fallbacks.ipynb)

[`Download Python source code: plot_17_failure_and_fallbacks.py`](https://skfolio.org/auto_examples/mean_risk/_downloads/59430ab1caa55ff5173a8ebf0b495be0/plot_17_failure_and_fallbacks.py)

[`Download zipped: plot_17_failure_and_fallbacks.zip`](https://skfolio.org/auto_examples/mean_risk/_downloads/3b4e28cfbd9eced01bc75cbd69371254/plot_17_failure_and_fallbacks.zip)

[Gallery generated by Sphinx-Gallery](https://sphinx-gallery.github.io)