Bugfix/257 speed up simpathsmodelpopulatetaxdbreferences process#268
Conversation
fa6ba3e to
840af45
Compare
840af45 to
7878c81
Compare
There was a problem hiding this comment.
Pull Request Overview
This PR improves the performance of the SimplPathsModelPopulateTaxdbReferences process by optimizing how donor tax unit lookups are performed at the start of each run. The changes convert eager fetching to lazy loading and perform sorting in memory instead of in the database.
- Changed from eager to lazy fetching of policies in
DonorTaxUnitPolicy - Modified query to remove LEFT JOIN FETCH and sort in memory by max year policy income
- Added database indexes and batch size optimization for better performance
Reviewed Changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
src/main/java/simpaths/model/taxes/DonorTaxUnitPolicy.java |
Changed fetch strategy from EAGER to LAZY, added BatchSize annotation and database indexes for performance |
src/main/java/simpaths/model/SimPathsModel.java |
Modified donor pool query to remove JOIN FETCH and implemented in-memory sorting by max year policy income |
src/test/java/simpaths/integrationtest/expected/Statistics21.csv |
Updated expected test results with minor numerical differences due to performance optimization changes |
src/test/java/simpaths/integrationtest/expected/HealthStatistics1.csv |
Updated expected test results with minor numerical differences due to performance optimization changes |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
|
To sense-check: this produces miniscule differences in calculated stats, usually beyond the 7th/8th decimal places in only about a 10th of estimates. The only way I think this is possible is due to differing de-duplication procedures in the searching approaches? |
b5c2207 to
73ea1d5
Compare
What
Why