Parent: #82
Scope
Build the foundational infrastructure that all subsequent phases depend on.
Deliverables
-
Module layout — Create the directory structure: model/, analysis/, controller/ with __init__.py files.
-
model/run.py — Refactored spectroscopy_run with:
self.results = {} (flat dict, dot-separated keys)
self.update_status() preserved
- Data loading methods (
load_run_keys, load_run_key_delayed) retained but writing to results dict
-
model/experiment.py — experiment and spectroscopy_experiment classes moved as-is.
-
analysis/registry.py — The step registry:
@register_step(name) decorator
@register_reduction(name) decorator
get_step(name) lookup function
get_reduction(name) lookup function
list_steps() for introspection
-
controller/config_parser.py — YAML parser that:
- Reads a YAML file
- Validates required sections exist (
experiment, data, pipeline, output)
- Returns a structured config object
-
controller/pipeline_runner.py — Step dispatch loop:
- Iterates
pipeline section
- Looks up each step name in registry
- Calls step with
(run, **args)
- Logs execution
-
controller/batch_manager.py — Shot chunking and multiprocessing:
- Breaks runs into shot ranges (existing
break_into_shot_ranges logic)
- Manages
multiprocessing.Pool
- Reconverges batch results into run-level list
-
Pipeline class — The user-facing entry point:
Pipeline.from_yaml(path) class method
.run(cores, batch_size) method
.results attribute for post-run access
Acceptance criteria
Pipeline.from_yaml("test.yaml").run(cores=4, batch_size=1000) executes without error on a minimal YAML with placeholder steps
- Registry correctly discovers and dispatches decorated methods
- Batch manager splits and reconverges shot ranges correctly
Parent: #82
Scope
Build the foundational infrastructure that all subsequent phases depend on.
Deliverables
Module layout — Create the directory structure:
model/,analysis/,controller/with__init__.pyfiles.model/run.py— Refactoredspectroscopy_runwith:self.results = {}(flat dict, dot-separated keys)self.update_status()preservedload_run_keys,load_run_key_delayed) retained but writing toresultsdictmodel/experiment.py—experimentandspectroscopy_experimentclasses moved as-is.analysis/registry.py— The step registry:@register_step(name)decorator@register_reduction(name)decoratorget_step(name)lookup functionget_reduction(name)lookup functionlist_steps()for introspectioncontroller/config_parser.py— YAML parser that:experiment,data,pipeline,output)controller/pipeline_runner.py— Step dispatch loop:pipelinesection(run, **args)controller/batch_manager.py— Shot chunking and multiprocessing:break_into_shot_rangeslogic)multiprocessing.PoolPipelineclass — The user-facing entry point:Pipeline.from_yaml(path)class method.run(cores, batch_size)method.resultsattribute for post-run accessAcceptance criteria
Pipeline.from_yaml("test.yaml").run(cores=4, batch_size=1000)executes without error on a minimal YAML with placeholder steps