261 slow multirun performance after database setup with simpathstart

[andrewbaxter439]

What

• changes the reloading of persistent data by calling lazily
• Old process: load each persist table -> join households -> join benefitUnits -> join persons -> then filter for year/country/pop
• New process: load persist table, check for year country pop. If present, Then (lazy) load households and return

Why

• speeds things up considerably if a persist table is being recalled
• If no persist table for given year/country/pop, doesn't load households before checking (as they're not needed)
• Closes Slow multirun performance after database setup with SimPathStart #261 by speeding up second run onwards

[Copilot]

Pull Request Overview

Optimizes persistent data loading performance by implementing lazy loading for households and changing the database query strategy to reduce unnecessary data fetching. The changes eliminate loading all related households and nested entities upfront, instead only loading them when a matching persistent table is confirmed to exist.

• Changed households fetch strategy from EAGER to LAZY with SUBSELECT mode
• Split database query into two phases: first check for existence, then lazily load households if found
• Removed unnecessary JOIN operations from initial query to reduce database load

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
SimPathsModel.java	Modified getProcessed method to use two-phase lazy loading approach
Processed.java	Changed households fetch type from EAGER to LAZY with SUBSELECT optimization

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[dav-sonn]

Everything works smoothly on my machine. The branch can be merged into develop.

[Mariia-Var]

Hi Andy,

I tested the model with the different start years - everything works well (no more slow loading of the initial populations).
I agree with David - the new changes can be pushed into the develop branch.