fix!: Let writing data to file at flush

[filimarc]

Describe the work done

Add a fix to the simulation checkpoint integration, since it is the expected behavior. Now when results are flushed they are written to file without storing memory. As discussed backward compatibility is preserved, the new behavior is triggered only when run_simulation method is called specifying output_filename argument, this will be the default when CLI is used.

Refer to issue: #50
closes #232

Tasks

---

📚 Documentation preview 📚: https://bsb-nest--213.org.readthedocs.build/en/213/

---

📚 Documentation preview 📚: https://bsb-neuron--213.org.readthedocs.build/en/213/

---

📚 Documentation preview 📚: https://bsb-core--213.org.readthedocs.build/en/213/

---

📚 Documentation preview 📚: https://bsb-arbor--213.org.readthedocs.build/en/213/

[codecov]

Codecov Report

❌ Patch coverage is 87.03704% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.07%. Comparing base (80ffbd5) to head (5ad6d53).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
packages/bsb-core/bsb/simulation/results.py	79.31%	5 Missing and 1 partial ⚠️
packages/bsb-nest/bsb_nest/adapter.py	90.90%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #213      +/-   ##
==========================================
+ Coverage   80.54%   85.07%   +4.53%     
==========================================
  Files         188      123      -65     
  Lines       18179    12590    -5589     
  Branches     2174     1472     -702     
==========================================
- Hits        14642    10711    -3931     
+ Misses       2977     1548    -1429     
+ Partials      560      331     -229     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Helveg]

Looks good! Double check what's going on with multi-sim, and multi-block.

• Add a test with a 1 sim that writes 3 blocks, and verify data goes into correct blocks
• Add a test with 3 sims that each write to 2-3 blocks and also verify integrity
• Add a test that tries to write 10G to disk and check that memory stays flat.

[filimarc]

I've add the tests suggested, for three sim on the same file and to check the flushing with bsb-neuron.

@Helveg @drodarie why is it compiling bsb-otel readthedocs if is not in this branch?

[drodarie]

It is in preparation for the integration of the bsb-otel and related its branch. It is triggered for every PR push so do not worry if it does not pass the tests.

[Helveg]

Good! I'd just like a bit clearer how strong the proof of the memory test is. Ideally we can run it with a debug flag to produce a graph or something so that we can now have some visual evidence and later in case of regressions easily can have a look.

[filimarc]

@Helveg now all the things discussed should be adressed

[Helveg]

API surface rectifications (SimulationResult / streamed checkpoints)

We reworked the streamed-result interface so the in-memory and streamed paths share one clear contract. Summary of what changed and why.

Block identity

Previously both the neo block name and the NIX storage key were set to simulation.name (nix_name=simulation.name). neo's write_block deletes and recreates any block whose nix_name already exists, so re-running a simulation (or running two same-named sims) into one file silently overwrote the earlier block.

• Storage key is now auto-generated and unique per run (we no longer pass nix_name).
• The block label stays simulation.name (human-readable, may repeat across runs).
• Each block stamps run metadata (sim name, timestamp, run index).
• Writing is append-only: re-running into the same file adds a new block, never overwrites.
• flush() addresses its block by the unique key, not by a name match.

Reading

• .block is now a property: returns the in-memory block, and in streamed mode raises with a clear message pointing at the file (instead of returning an inert object).
• Removed the top-level .spiketrains / .analogsignals accessors. In streamed mode they silently returned [], and they only ever exposed segments[0] (the first checkpoint), which is misleading once checkpointing splits a run across segments. Data is read per segment (segment.spiketrains / segment.analogsignals); a proper results-reading utility is left as a follow-up.

Writing

• .write(dest) writes the in-memory block as before; in streamed mode it copies the file (previously it was a silent no-op).

Modes

• Two modes, fixed at construction: in-memory (no output_filename) and streamed (output_filename given). The CLI always streams.

NEST composition

• NEST freezes the kernel time resolution once nodes are created, so setting it per-simulation in set_settings broke composing multiple simulations into one adapter (the second prepare raised "time representation cannot be changed"). Resolution is now set once, up front in simulate() before any neurons are created, and composing simulations with differing resolutions is rejected (NEST has a single kernel resolution). NEST simulations can now be composed into one file.

Bugs fixed along the way

• Same-name rerun overwrite (see Block identity).
• Checkpoint controllers kept their step counter across run_simulation calls, so a reused simulation object did not re-checkpoint. The counter now resets per run.

Tests

• Replaced the misleading triple-sim test (name said "triple", docstring said "two", and it only used distinct names so it never exercised the overwrite path) with an append test and a same-name overwrite regression.
• Added NEST and NEURON composition tests.
• Migrated the remaining results.spiketrains / results.analogsignals callers to result.block.segments[0]....

Open follow-up

• Streamed checkpoint writing is not yet rank-aware, so its behaviour under MPI (rank-0-gated write after a gather, or parallel I/O) needs a dedicated design pass. This matters for HPC runs and is worth a separate issue.

[filimarc]

Good, thanks. Just one the .spiketrains`.analogsignals` interfaces where left to be fully backward compatible. I don't know who is using this things, but maybe we can avoid to flag it as breaking change.

[Helveg]

We can change this to a breaking change, I already have another PR targetting a new v8 branch that is related. The changes required to the simulation results etc in prep of the rest of the LFP will most likely be breaking too.

[filimarc]

Ok, so we can restore the next branch aand merge to it you are ok.