feat(config): out-of-process NEST kernel for config-build-time checks (#227)

[Helveg]

Summary

Closes #227 — loading a NestSimulation config no longer mutates NEST's global kernel state in the user's process.

• New bsb.config.BuildContext (ContextVar-backed, attribute-style namespace with LIFO cleanup callbacks). The root build activates it for the whole construction sequence in wrap_root_postnew.
• bsb_nest.get_nest_kernel_proxy() lazily spawns a multiprocessing.managers.BaseManager subprocess that imports NEST, registers the proxy at ctx.bsb_nest.kernel, and shuts down on context exit. The proxy is what keeps config building from touching the in-process kernel.
• NestSynapseSettings.delay is back to a callable required= (_is_delay_required) — same intent as the pre-fix: Loading modules to check NEST models #228 _is_delay_required callable, but routed through the out-of-process proxy.
• import nest stays at module top in the runtime modules (adapter, cell, device, devices). distributions.py keeps a lazy import via _LazyDistributionNames, because it would otherwise import NEST at module-load time just to enumerate distribution names for an attribute validator.

Validation semantics

The checker separates two failure modes:

• Proxy reachable, model unknown → hard ConfigurationError (matches the pre-fix: Loading modules to check NEST models #228 boot-time check — config still rejects bad model names at build time)
• Proxy unreachable (no build context, kernel spawn failed, IPC error) → warn + return False so config loading stays robust; the real NEST error surfaces later at adapter prepare/connect time

The "delay needed?" answer itself is always soft — an unreachable proxy never blocks a valid config.

Note: post-build mutation (cfg.simulations[...] = ...) runs outside the build context, so the hard model-validity check requires wrapping the mutation in with bsb.config.build_context(): if you want strict validation; otherwise the bad model surfaces at sim time.

Status — please review the design

@drodarie — could you sanity-check this against the cerebellar models? Specifically:

1. Does config loading still succeed end-to-end?
2. The synapse-delay requirement is now soft when the proxy can't be reached; the model-name validation stays hard when the proxy IS reachable. For your typical config-mutation flows, do you need help wrapping them in build_context(), or should the proxy spawn even outside the root build?
3. The proxy spawns once per BuildContext and shuts down on exit. With your larger configs, is the per-build subprocess startup cost (a few tenths of a second) an issue, or should we look at reuse?

Happy to iterate on any of the three.

Test plan

• pytest packages/bsb-core/tests/test_build_context.py (9 tests, all pass)
• pytest packages/bsb-nest/tests/test_build_context.py (7 tests, all pass — proxy lifecycle, delay-required/optional, unknown-model hard error, and the no-context / proxy-unreachable fallbacks)
• Full bsb-core suite: 478 pass, 11 skipped, 0 failures
• Updated test_error_gap_junctions_syn (now expects RequirementError inside build_context()), test_unknown_synapse (now expects ConfigurationError inside build_context()), test_unknown_modules, test_unknown_cell — all pass
• @drodarie validation against cerebellar models

🤖 Generated with Claude Code

---

📚 Documentation preview 📚: https://bsb-nest--235.org.readthedocs.build/en/235/

---

📚 Documentation preview 📚: https://bsb--235.org.readthedocs.build/en/235/

---

📚 Documentation preview 📚: https://bsb-core--235.org.readthedocs.build/en/235/

[codecov]

Codecov Report

❌ Patch coverage is 81.84818% with 55 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.42%. Comparing base (80ffbd5) to head (d4bfc4d).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
packages/bsb-nest/bsb_nest/_kernel_server.py	28.30%	37 Missing and 1 partial ⚠️
packages/bsb-nest/bsb_nest/_kernel_proxy.py	92.50%	6 Missing and 3 partials ⚠️
packages/bsb-core/bsb/config/_make.py	90.62%	4 Missing and 2 partials ⚠️
packages/bsb-core/bsb/config/_build_context.py	94.11%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #235      +/-   ##
==========================================
+ Coverage   80.54%   85.42%   +4.87%     
==========================================
  Files         188      103      -85     
  Lines       18179    11813    -6366     
  Branches     2174     1387     -787     
==========================================
- Hits        14642    10091    -4551     
+ Misses       2977     1415    -1562     
+ Partials      560      307     -253     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[drodarie]

@Helveg, It seems that the __boot__ function of the NestSimulation class is not using the context proxy? Same for the other boot functions. Should they not use the proxy instead?
My point is that if I do:

scaffold = from_storage("my_network.hdf5")
import nest 

nest.Models() # contains the module that should be loaded only by the simulation adatper

[Helveg]

@drodarie good catch, you were right. I audited every boot/__boot__ hook in bsb-nest (they all run during from_storage, since _boot_nodes walks the whole config tree, simulations included):

• NestSimulation.__boot__ did nest.Install(...) in-process. That's exactly your repro: loading a network mutated the in-process kernel, so nest.Models() showed the simulation's modules. Removed (3b65aae). Modules are still installed at adapter prepare() time, and into the out-of-process proxy for config-time validation.
• NestCell.__boot__ did an in-process nest.Models(mtype="nodes") lookup. Replaced with a nest_node_model type handler on NestCell.model that queries the out-of-process proxy.
• The device boot() hooks (Multimeter, SinusoidalPoissonGenerator) only do config validation and never touch nest.

While in there I also:

• Made synapse validation symmetric: a nest_synapse_model type handler on NestSynapseSettings.model (4d0fbc0), so _is_delay_required now only answers the delay question.
• Centralized and deduped module loading on the proxy (load_modules / load_simulation_modules); a nonexistent module is now a hard ConfigurationError at build time (04ce8ef).

Net effect: loading a config validates cell + synapse models and installs modules entirely out-of-process, so the in-process nest kernel is no longer touched at config-build time. The module-level import nest statements in the simulation-time components are left as-is, since importing nest is harmless; it is mutating the in-process kernel that we avoid.

test_unknown_cell / test_unknown_modules are updated to assert their errors under build_context(), like test_unknown_synapse.

[drodarie]

Quite a technical solution, I think I understand the idea behind it but there is a major flaw: node attributes initialization order is not guaranteed because they are part of a dict. But here we need the modules attribute of NestSimulation to be set so that it can be used for cell_models or connection_models

[Helveg]

@drodarie thanks, you were right about the ordering not being guaranteed. Fixed it at the source rather than working around it:

Attribute build order is now declaration-order-preserving (get_config_attributes, 92e6bc1 / 3623501). Walking the MRO base-first, each class's declaration order is kept as a subsequence: an overridden attribute keeps its inherited slot, and a subclass's new attributes are spliced in at the position the subclass declares them. So NestSimulation.modules (declared before cell_models / connection_models) is now built before them, which is what the model validators rely on.

A subclass may also reorder inherited attributes by redeclaring them, but only if it redeclares every attribute caught between the ones it moves; otherwise the order is ambiguous and a new AttributeOrderError is raised naming the attributes to redeclare. Covered in test_configuration and documented in an "Attribute ordering" section.

Defensive guard (d7f64b0): load_simulation_modules now stops at the ancestor that owns modules and raises if it is not built yet, instead of walking past it and silently validating models against a kernel missing the modules. If the build order is ever changed, this fails loudly with a message explaining modules must precede cell_models / connection_models.

Also fixed a regression the ordering change introduced: the fallback that recovers dynamically-registered attributes was resurrecting attributes a subclass had unset (Layer.dimensions); now tracked and skipped, with a regression test (6051edf).

Re-requesting review.

[Helveg]

@drodarie this round is mostly CI hardening on top of the proxy + attribute-ordering fixes; it should be ready for another look.

The out-of-process kernel now survives MPI. The bsb-nest suite runs under mpiexec -n 2, and the kernel subprocess was inheriting the launcher's OMPI_*/PMI_*/PMIX_* environment. Its import nest therefore called MPI_Init and blocked forever trying to enroll as a rank of the parent job, which is what was timing CI out at 60 minutes. The subprocess now starts from a stripped environment, so it runs as an independent, uninstrumented NEST process no matter how the parent was launched. (96401d6)

NESTML extension-module coverage. Added a test that builds a minimal NESTML module and loads it through the proxy, asserting the model lands in the out-of-process kernel and never touches the main process. The build itself (CMake compiling and running NEST helper programs) hit the same MPI_Init wall under mpiexec, so it builds once on the main rank with the launcher environment stripped while the other ranks wait on bcast. (d4bfc4d)

NEST 3.8 and CI plumbing. Bumped NEST to 3.8 (added libboost, versioned the cache key) and moved the NEST build ahead of the opentelemetry-instrument-wrapped test step, so the auto-instrumentation no longer corrupts NEST's CMake Python helpers.

Verified locally against a from-source NEST 3.8 build, both single-process and under mpiexec -n 2: the full bsb-nest suite is green (35 passing serially; 14 run plus 21 parallel-skips under MPI, no hang). CI is green across the board.

[drodarie]

So now we have a build context which stores a proxy that interfaces with an isolated server that contains a NEST kernel.

I wonder if we did not get too deep on this one ;)

[Helveg]

Too deep? This is a perfect precedent for dealing with simulators that store global state 👍