Thread-safe seeding and per-job seeds for drivers

[bennoschoenstein]

Description of the Issue:

If a QUEENS driver contains stochastic code of its own, there is currently no clean way to make it reproducible, because QUEENS does not hand a seed to the driver.

Concretely, consider a Monte Carlo study with seed = 42 and three samples. MonteCarlo.pre_run() calls np.random.seed(42) and then draws the samples. The scheduler forwards (sample, job_id) to Driver.run() - but no seed. A driver that needs randomness (e.g. for pre-processing that generates one stochastic realization per job) has to invent its own derivation. This is error-prone and decouples the driver's randomness from the iterator's sampling randomness.

On top of that, np.random.seed(self.seed) in MonteCarlo.pre_run() and LatinHypercubeSampling.pre_run() mutates the global NumPy RNG. Any other code in the same process that touches np.random (Dask workers, drivers, user code, libraries) shares the same pool, so the order of draws becomes non-deterministic under parallel execution.

Proposed Solution:

Adopt numpy.random.SeedSequence:

• In the iterator, build a local generator instead of seeding globally:

ss = np.random.SeedSequence(self.seed)
rng = np.random.default_rng(ss)
self.samples = parameters.draw_samples(num_samples, rng = rng)

• Spawn one child SeedSequence per sample via ss.spawn(num_samples) and forward it to the driver - either as a new argument to Driver.run() / Scheduler.evaluate(), or attached to the Parameters object so it is available via sample_as_dict(sample) (e.g. sample["__seed__"]).

With this change, a single user-provided super-seed deterministically drives both the sampling and the per-job driver randomness, independently of thread or worker assignment.

Action Items:

No response

Related Issues:

No response

Interested Parties:

@sbrandstaeter

[sbrandstaeter]

Thanks for opening this issue:

Here are two recent examples where it would have been useful:

• In one study, each job required a randomly generated rough surface. Instead of precomputing all surfaces, I wanted to generate them dynamically on the worker during execution.
• @BishrMaradni encountered a similar case, where realisations of a random field needed to be computed at the driver level.
• @bennoschoenstein has a similar scenario where random configurations of fibre-reinforced polymers are needed

More generally, while precomputing these quantities on the iterator level is possible, it can be inefficient. It often requires transferring large amounts of data to workers, which is costly. Additionally, if the generation process is complex (e.g., random-field realisations), doing everything in advance can become a significant computational bottleneck.

An idea to push the proposed feature to the limit is to hand off any realisations of rvs to the driver level, relying entirely on passing a seed to the worker.