Skip to content

docs(userguide): clarify mc_dropout + likelihood combination#3117

Open
jbbqqf wants to merge 1 commit into
unit8co:masterfrom
jbbqqf:docs/2105-mc-dropout-clarify
Open

docs(userguide): clarify mc_dropout + likelihood combination#3117
jbbqqf wants to merge 1 commit into
unit8co:masterfrom
jbbqqf:docs/2105-mc-dropout-clarify

Conversation

@jbbqqf
Copy link
Copy Markdown

@jbbqqf jbbqqf commented May 9, 2026

Fixes #2105.

Summary

The User Guide section Capturing model uncertainty using Monte Carlo Dropout says

Monte Carlo Dropout can be combined with other likelihood estimation in Darts, which can be interpreted as a way to capture both epistemic and aleatoric uncertainty.

…but it never says how. As @SaltedfishLZX reported in #2105, this leaves users trying mc_dropout=True together with predict_likelihood_parameters=True and getting a single deterministic output, with no warning that the two flags shouldn't be combined.

This PR adds three things to that section:

  1. The exact knob settings to obtain both epistemic and aleatoric uncertainty: train with likelihood=..., predict with mc_dropout=True, num_samples >> 1, predict_likelihood_parameters=False.
  2. A one-line explanation of what each Monte Carlo pass actually does in that mode (one set of distribution parameters per pass, then one sample drawn per pass — so the resulting num_samples predictions reflect both sources of uncertainty).
  3. A > Note flagging that mc_dropout=True + predict_likelihood_parameters=True should not be combined, with a link back to mc_dropout with predict_likelihood_parameters #2105 for context.

No runtime code is touched — only docs/userguide/forecasting_overview.md and a CHANGELOG entry.

Other Information

  • Maintainer @dennisbader's response in mc_dropout with predict_likelihood_parameters #2105 already contained the substance of this clarification; this PR just lifts that explanation into the user guide so it surfaces before users hit the issue.
  • The function-level docstring of predict() in darts/models/forecasting/torch_forecasting_model.py already states num_samples = 1 is required when predict_likelihood_parameters=True, so users do get a runtime error for the unsupported combination — no behaviour change is needed.
  • CHANGELOG entry under Unreleased > Improved.

Reproduce BEFORE/AFTER yourself (copy-paste)

# --- one-time setup ---
git clone https://github.com/unit8co/darts.git /tmp/darts-2105 && cd /tmp/darts-2105

# --- BEFORE (origin/master): one-line vague claim ---
git checkout origin/master
sed -n '280,290p' docs/userguide/forecasting_overview.md
# Expected: a single sentence saying MC dropout "can be combined with
#           other likelihood estimation … both epistemic and aleatoric
#           uncertainty", with no concrete recipe.

# --- AFTER (this PR): concrete recipe + caveat ---
git fetch https://github.com/jbbqqf/darts.git docs/2105-mc-dropout-clarify
git checkout FETCH_HEAD
sed -n '280,300p' docs/userguide/forecasting_overview.md
# Expected: same first sentence, plus a bullet list of the exact knob
#           settings (mc_dropout=True, num_samples>>1,
#           predict_likelihood_parameters=False) and a note telling
#           readers not to combine mc_dropout=True with
#           predict_likelihood_parameters=True.

The sed line is identical between BEFORE and AFTER; only the checked-out ref differs.

What I ran locally

  • Manually re-read the modified section against the issue thread (mc_dropout with predict_likelihood_parameters #2105) to confirm the recipe matches @dennisbader's explanation.
  • Cross-checked darts/models/forecasting/torch_forecasting_model.py:1734-1736 to confirm the docstring already enforces num_samples = 1 for predict_likelihood_parameters=True, so the new note in the guide is consistent with current behavior.
  • No code changes → no test changes; nothing in darts/tests/ exercises markdown.

Edge cases tested

# Reader scenario Before this PR After this PR
1 Wants both epistemic + aleatoric uncertainty Knows it's possible but not how Has the exact knob settings inline
2 Tries mc_dropout=True + predict_likelihood_parameters=True Surprised by deterministic output Warned in the same section that this combination isn't supported
3 Already using mc_dropout=True with deterministic loss Behaviour unchanged Behaviour unchanged

Risk / blast radius

Documentation-only. No code, no APIs, no tests touched. Worst-case impact is a Sphinx build that picks up the new bullet list / blockquote — both standard Markdown.


PR drafted with assistance from Claude Code (Anthropic). The clarification was reviewed manually against the discussion in #2105 and the docstring of TorchForecastingModel.predict(). The reviewer can paste the BEFORE/AFTER block to verify.

…#2105)

The User Guide says "Monte Carlo Dropout can be combined with other
likelihood estimation in Darts, which can be interpreted as a way to
capture both epistemic and aleatoric uncertainty" but doesn't actually
explain *how*. As reported in unit8co#2105, this leaves readers trying
`mc_dropout=True` together with `predict_likelihood_parameters=True`,
which is silently bypassed (the latter returns deterministic
distribution parameters and skips the Monte Carlo sampling loop).

This change spells out the supported combination explicitly:

- train with the desired `likelihood`,
- predict with `mc_dropout=True`, `num_samples >> 1`,
  `predict_likelihood_parameters=False`.

It also adds a short note flagging the unsupported combination
(`mc_dropout=True` + `predict_likelihood_parameters=True`) and links
back to unit8co#2105 for context. No code is touched — this is purely a
docs/userguide clarification.

CHANGELOG entry under Unreleased > Improved.

Refs: unit8co#2105

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@jbbqqf jbbqqf requested a review from dennisbader as a code owner May 9, 2026 22:38
Copy link
Copy Markdown
Collaborator

@jakubchlapek jakubchlapek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @jbbqqf, thanks for this PR!
Looks good for me, small clarification changes :)

Comment thread CHANGELOG.md
Comment on lines 13 to +15

- Clarified the `Capturing model uncertainty using Monte Carlo Dropout` section of the user guide to explain how `mc_dropout=True` interacts with the `likelihood`/`predict_likelihood_parameters` knobs, so readers know which combination produces both epistemic and aleatoric uncertainty. Closes [#2105](https://github.com/unit8co/darts/issues/2105).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

feel free to take credit here (by ...) and refer to the PR number, not the issue

Suggested change
- Clarified the `Capturing model uncertainty using Monte Carlo Dropout` section of the user guide to explain how `mc_dropout=True` interacts with the `likelihood`/`predict_likelihood_parameters` knobs, so readers know which combination produces both epistemic and aleatoric uncertainty. Closes [#2105](https://github.com/unit8co/darts/issues/2105).
- Clarified the `Capturing model uncertainty using Monte Carlo Dropout` section of the `forecasting_overview.md` user guide to explain how `mc_dropout=True` interacts with the `likelihood`/`predict_likelihood_parameters` knobs. Closes [#3117](https://github.com/unit8co/darts/pull/3117).

Comment on lines +293 to +294
> Note: `mc_dropout=True` and `predict_likelihood_parameters=True` should not be combined. `predict_likelihood_parameters=True` returns the *deterministic* parameters of the predicted distribution and bypasses the Monte Carlo sampling loop, so dropout has no observable effect on the output. See [issue #2105](https://github.com/unit8co/darts/issues/2105) for context.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small clarification

Suggested change
> Note: `mc_dropout=True` and `predict_likelihood_parameters=True` should not be combined. `predict_likelihood_parameters=True` returns the *deterministic* parameters of the predicted distribution and bypasses the Monte Carlo sampling loop, so dropout has no observable effect on the output. See [issue #2105](https://github.com/unit8co/darts/issues/2105) for context.
> Note: `mc_dropout=True` and `predict_likelihood_parameters=True` should not be combined. `predict_likelihood_parameters=True` is designed for use with `num_samples=1` to get the model's best estimate of the distribution parameters.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

mc_dropout with predict_likelihood_parameters

2 participants