Skip to content

[codex] Add observer execution overrides#48

Merged
mvallebr merged 5 commits into
mainfrom
codex/execution-overrides-observers
Jun 20, 2026
Merged

[codex] Add observer execution overrides#48
mvallebr merged 5 commits into
mainfrom
codex/execution-overrides-observers

Conversation

@mvallebr

@mvallebr mvallebr commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Summary

This is the second runtime-DI slice for Issue 45.0.

It extends ExecutionOverrides from materializers to observers, adds the public Scope helper for compiled step keys, and fixes concrete builtin materializers (for example int, list, tuple) so they are accepted as concrete callables instead of misclassified during factory detection.

What changed

  • added ObserverRegistry to ExecutionOverrides
  • added public PIPELINE_SCOPE constant for pipeline-level observer overrides
  • added public Scope helper for sub-pipeline / compiled step keys
  • kept executor logic flat: sync and async executors still operate only on the compiled Dag
  • resolved runtime observer registrations from compiled source metadata instead of mutating the DAG
  • preserved pipeline observer inheritance into step/materialization events when overriding pipeline scope
  • taught registries to accept Scope keys for both materializers and observers
  • made is_factory() defensive so builtin concrete materializers like int are treated as concrete callables
  • added validation for observer override keys and values
  • added sync/async tests for pipeline-scope, step-scope, and sub-pipeline override paths
  • updated README and build-vs-run docs with Scope examples

Why

The previous PR introduced runtime overrides for materializers. The next logical step in Issue 45.0 was observers, because they already have a clear compiled contract in the DAG:

  • dag.pipeline_observers for pipeline lifecycle
  • node.observers for effective step/materialization observers, with source metadata

At the same time, the design document already called out that sub-pipeline override keys should not depend on handwritten magic strings. This PR implements that public helper as Scope, while keeping executors ignorant of sub-pipelines.

Contract boundary

This PR keeps the intended boundary explicit:

  • the DAG still defines topology, dependency resolution, and event semantics
  • runtime override only swaps registrations/callables for existing compiled scopes
  • overriding PIPELINE_SCOPE affects both pipeline lifecycle events and the inherited pipeline portion of step/materialization observers
  • overriding a step key only affects that step's compiled scope
  • Scope is resolved before execution; executors still only consume flat DAG step names

Examples

from synaflow import ExecutionOverrides, Observer, PIPELINE_SCOPE, Scope

sub = Scope("payments")
overrides = ExecutionOverrides.empty(p)

overrides.observers[PIPELINE_SCOPE] = [Observer(noop_metrics)]
overrides.observers[sub.scope("validate")] = [Observer(test_recorder)]
overrides.materializers[sub.scope("normalize")] = list

Impact

Tests can now silence production observers or replace them surgically at runtime without patching globals or rebuilding alternative pipeline definitions, including within included sub-pipelines.

This moves ExecutionOverrides closer to the target design, while still keeping resources out of scope for this PR.

Validation

  • uv run pytest

@mvallebr mvallebr marked this pull request as ready for review June 20, 2026 00:47
@mvallebr mvallebr merged commit 97826b6 into main Jun 20, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant