Unit test fail: `test_aurora_small`

[Aiden0609]

I cloned the repo and ran the test suite without modifying any code. One of the unit tests fails:

======================================= short test summary info ========================================
FAILED tests/test_model.py::test_aurora_small - assert (np.float64(0.05390285048894711) / np.float64(276.2245574525479)) <= 0.0001
======================== 1 failed, 55 passed, 38 warnings in 309.24s (0:05:09) =========================

Environment: Python 3.11.11
Dependencies:

  numpy==2.3.0
  scipy==1.15.3
  timm==1.0.15
  torch==2.5.1
  torchvision==0.20.1
  huggingface-hub==0.33.0

OS: MacOS Sonoma 14.4.1

The computed ratio is ~0.000195, slightly above the test threshold of 0.0001. I am not sure if it's a floating‑point precision issue or a logic error.

[wesselb]

Does the test fail every time for you, or only sometimes?

[Aiden0609]

Does the test fail every time for you, or only sometimes?

Every time, I tried install latest version dependencies, create new environments, but still failed

[wesselb]

Thanks, @Aiden0609. I’m not too worried about this as the tolerance is only exceeded by a small margin. Unfortunately this test is a bit iffy and depends strongly on the numerics, which can behave differently between machines. Feel free to locally increase the tolerance by a little. I think that should be fine.

[Aiden0609]

Thanks, @Aiden0609. I’m not too worried about this as the tolerance is only exceeded by a small margin. Unfortunately this test is a bit iffy and depends strongly on the numerics, which can behave differently between machines. Feel free to locally increase the tolerance by a little. I think that should be fine.

Great, thank you