Skip to content

Create an end-to-end test#162

Merged
igelstorm merged 10 commits intosimpaths:developfrom
igelstorm:ei-integration-test
Apr 17, 2025
Merged

Create an end-to-end test#162
igelstorm merged 10 commits intosimpaths:developfrom
igelstorm:ei-integration-test

Conversation

@igelstorm
Copy link
Copy Markdown
Contributor

@igelstorm igelstorm commented Mar 31, 2025

What

  • Sets up Maven Failsafe, the standard way to run integration tests in Maven
  • Translates the run-simpaths GitHub Action into an integration test (which does the same thing)
  • Adds assertions about the model output (the Statistics*.csv files)
  • Changes the GitHub Action to run these tests instead of directly running the model

Why

Addresses #138. This will allow major changes to implementation details (e.g. optimisations, design changes) to be done much more safely, since these tests passing gives some assurance that the overall behaviour of the model hasn't changed.

How to use

  • The new integration tests can be run by running mvn verify.
  • Since several steps are time-consuming, I've split them into multiple tests. It's possible to run an individual test by running e.g. mvn verify -D it.test=RunSimPathsIT#testVerifySimulationOutput

Copy link
Copy Markdown
Collaborator

@andrewbaxter439 andrewbaxter439 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks pretty awesome and very neat! Nice to be able to have end-to-end all handled within maven/JUnit.

Have suggested changes but actually not sure they're implementable. If parallelising three types of tests doesn't work then can just delete other tests.

@andrewbaxter439
Copy link
Copy Markdown
Collaborator

andrewbaxter439 commented Apr 4, 2025

One small, additional though - is it worth doing/possible to force the simpaths setup to use only the training data, for speed and consistency when running mvn verify? At the moment it chooses based on the presence or absence of population data outside the training folder in 'Initial Populations':

https://github.com/centreformicrosimulation/SimPaths/blob/e1e3c79b65bc704053898b3061a557f693e669c5/src/main/java/simpaths/experiment/SimPathsStart.java#L207-L213

Think only way of overriding this would be to pass another command-line argument which is in turned passed to runGUIlessSetup to then override the setting of the training flag. If that's a different job though then can leave out of here.

@igelstorm
Copy link
Copy Markdown
Contributor Author

One small, additional though - is it worth doing/possible to force the simpaths setup to use only the training data, for speed and consistency when running mvn verify?

This is a good point - it would definitely be good to make it easier to run locally even if you have other initial population data installed (without having to delete/move files).

Ultimately I think the ideal is if it's always possible to configure which folder is used for input data; then this test could just make use of that feature. I'd be reluctant to add a less thought-out, potentially more awkward fix purely for use in the tests, though - so I suggest we deal with this separately.

Copy link
Copy Markdown
Collaborator

@andrewbaxter439 andrewbaxter439 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks pretty smashing! Hope this 'resolves' my prior review comments

@igelstorm igelstorm merged commit 6bb5102 into simpaths:develop Apr 17, 2025
4 checks passed
@igelstorm igelstorm deleted the ei-integration-test branch April 17, 2025 15:17
@igelstorm igelstorm linked an issue May 9, 2025 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create an end-to-end test

2 participants