Tests failing with `ValueError: cannot broadcast shape (nan,) to shape (nan,)`

[GaetanLepage]

Describe the issue:

When updating dask to 2025.2.0, the following dask-ml tests start failing:

=========================== short test summary info ============================
FAILED tests/ensemble/test_blockwise.py::TestBlockwiseVotingRegressor::test_fit_array - ValueError: cannot broadcast shape (nan,) to shape (nan,)
FAILED tests/ensemble/test_blockwise.py::TestBlockwiseVotingRegressor::test_fit_frame - ValueError: cannot broadcast shape (nan,) to shape (nan,)
FAILED tests/linear_model/test_glm.py::test_lr_score - ValueError: cannot broadcast shape (nan,) to shape (nan,)
FAILED tests/metrics/test_regression.py::test_ok[r2_score-True] - ValueError: cannot broadcast shape (nan,) to shape (nan,)
FAILED tests/metrics/test_regression.py::test_ok[r2_score-False] - ValueError: cannot broadcast shape (nan,) to shape (nan,)
FAILED tests/preprocessing/test_data.py::TestQuantileTransformer::test_fit_transform_frame - IndexError: boolean index shape (nan,) must match indexed array's (nan,).
FAILED tests/test_incremental.py::test_scoring_string[threads-r2] - ValueError: cannot broadcast shape (nan,) to shape (nan,)
FAILED tests/test_incremental.py::test_scoring_string[distributed-r2] - ValueError: cannot broadcast shape (nan,) to shape (nan,)
FAILED tests/test_parallel_post_fit.py::test_laziness - ValueError: cannot broadcast shape (nan,) to shape (nan,)
= 9 failed, 585 passed, 116 skipped, 1 deselected, 8 xfailed, 6 xpassed, 312 warnings in 204.79s (0:03:24) =

Minimal Complete Verifiable Example:

$ pytest

Anything else we need to know?:

Context: bumping dask in nixpkgs: NixOS/nixpkgs#381896

Environment:

• Dask version: 2025.2.0
• Python version: 3.12.8
• Operating System: NixOS
• Install method (conda, pip, source): nix

[TomAugspurger]

git bisect on dask/dask points to dask/dask#11728 (cc @crusaderky) causing tests/ensemble/test_blockwise.py::TestBlockwiseVotingRegressor::test_fit_array to fail.

I'll look into getting a minimal reproducer later (edit: dask/dask#11753).

[matteociccozzi-ah]

@TomAugspurger

if __name__ == "__main__":
    from dask.distributed import Client, LocalCluster

    cluster = LocalCluster(n_workers=4)
    client = Client(cluster)

    dask_model = lightgbm.DaskLGBMRegressor(client=client)

    x_train = da.random.random((1000, 10), chunks=(500, 10))
    y_train = da.random.random((1000,), chunks=(500,))

    dask_model.fit(x_train, y_train)

    y_pred = dask_model.predict(x_train)

    r2_dask = dask_ml.metrics.r2_score(y_train, y_pred)
    print(f"r2_dask score is: {r2_dask}")

this reproduces the issue on my machine. I've actually been stumped on this for some time, I'm relatively new to dask. What options do I have as a workaround?

EDIT: ignore the above, just saw your commit with the minimal reproducible example :)

[TomAugspurger]

One more fix still tests/preprocessing/test_data.py::TestQuantileTransformer::test_fit_transform_frame

[TomAugspurger]

dask/dask#11943 fixes this last one. Going to make that xfail for dask<=2025.5.1