Skip to content

[Repo Assist] fix: return p_value=1 when refuter simulations have zero variance (closes #807)#1454

Open
github-actions[bot] wants to merge 1 commit intomainfrom
repo-assist/fix-issue-807-nan-pvalue-normal-test-b85acf95ce3ec888
Open

[Repo Assist] fix: return p_value=1 when refuter simulations have zero variance (closes #807)#1454
github-actions[bot] wants to merge 1 commit intomainfrom
repo-assist/fix-issue-807-nan-pvalue-normal-test-b85acf95ce3ec888

Conversation

@github-actions
Copy link
Copy Markdown
Contributor

🤖 This is an automated fix from Repo Assist.

Closes #807

Root Cause

perform_normal_distribution_test in dowhy/causal_refuter.py computes a z-score by dividing by the standard deviation of the refuter simulations:

z_score = (estimate.value - mean_refute_values) / std_dev_refute_values
```

When all simulations return the same value (e.g. a random-common-cause refuter has no effect on the estimate), `std_dev_refute_values = 0`.  Division by zero produces `NaN` when `estimate.value == mean` (0 / 0) or `±inf` when they differand `st.norm.cdf(NaN)` or `st.norm.cdf(±inf)` returns `NaN`.

## Fix

Guard against the degenerate zero-variance case before computing the z-score:

- Log a warning that all simulation values are identical.
- Return `p_value = 1.0`, meaning the original estimate is statistically indistinguishable from the simulation distributionthe refutation provides no evidence against the estimate's validity.

This is the interpretation suggested by the original issue reporter.

## Changes

| File | Change |
|------|--------|
| `dowhy/causal_refuter.py` | Add early-return for `std_dev == 0` in `perform_normal_distribution_test` |
| `tests/test_causal_refuter.py` | Add regression test `test_normal_distribution_test_zero_std_returns_one` |

## Test Status

```
tests/test_causal_refuter.py::test_causal_refuter_placeholder_method PASSED
tests/test_causal_refuter.py::test_causal_refuter_bootstrap_test PASSED
tests/test_causal_refuter.py::test_normal_distribution_test_zero_std_returns_one PASSED

3 passed in 3.03s

No new lint errors introduced (pre-existing line-length warnings in the file are unrelated to this change).

Generated by 🌈 Repo Assist, see workflow run. Learn more.

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/repo-assist.md@11c9a2c442e519ff2b427bf58679f5a525353f76

…loses #807)

When all refuter simulations return the same value, std_dev is zero,
causing division by zero and a NaN p-value.  Guard against this
degenerate case: log a warning and return 1.0, meaning the original
estimate is indistinguishable from the simulation distribution.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@emrekiciman emrekiciman marked this pull request as ready for review April 14, 2026 07:51
@emrekiciman
Copy link
Copy Markdown
Member

/repo-assist please fix the linter errors in this commit (see build errors)

@github-actions
Copy link
Copy Markdown
Contributor Author

github-actions bot commented Apr 14, 2026

✗ Repo Assist encountered failed, see workflow run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

p-value NaN in some pathological cases with non-bootstrap method

1 participant