-
Notifications
You must be signed in to change notification settings - Fork 168
Description
Because of the way pytest collects test fixtures, where we have fixtures set up which are based on calling one of our functions in games.py in the test suite to generate a game, this function is called at the time pytest is run.
The problem with this is that if there is some bug in the game generation functions themselves, then pytest fails in effect at the test collection level (and therefore runs no tests at all), rather than failing partially gracefully (only on the tests affected by the problem with the game-building function).
A suggested improvement would be to revise the tests so that instead of taking a gbt.Game as an argument, they would take a Callable which returns a gbt.Game, and the test itself should load or build the game by invoking the callable.
However, that also seems a bit inelegant, as it then requires all of the test functions first to invoke the callable, which seems like a lot of unnecessary duplication. However, at the moment, some of our functions in games.py do not simply load a saved file, but build games using primitives, so they are nontrivial and therefore could be subject to bugs arising.