Skip to content

feat(engine): model-level governance tags inherited from config groups#920

Merged
hugocorreia90 merged 1 commit into
mainfrom
feat/engine/governance-tags
Jun 18, 2026
Merged

feat(engine): model-level governance tags inherited from config groups#920
hugocorreia90 merged 1 commit into
mainfrom
feat/engine/governance-tags

Conversation

@hugocorreia90

Copy link
Copy Markdown
Contributor

What

Adds a model-level [tags] block, inheritable from config groups, so a governance attribute declared once on a group lands on its whole fan-out — completing the engine half of the governed-materialization story (the orchestrator half lands in a follow-up dagster-rocky PR).

# models/groups/finance.toml
schema_template = "mart_{region}"
[tags]
domain = "finance"
tier   = "gold"
# models/fct_orders.toml — inherits domain, overrides tier, adds owner
group = "finance"
[tags]
tier  = "silver"
owner = "data-eng"

rocky compile --output json now emits per-model models_detail[].tags:

agg_revenue (.rocky)  tags={domain: finance, tier: gold}
fct_orders  (.sql)    tags={domain: finance, owner: data-eng, tier: silver}

How

  • GroupConfig.tags + RawModelConfig.tags + ModelConfig.tags (all #[serde(default)]), merged in resolve_model_config — the shared .sql / .rocky resolution chokepoint, so both loaders inherit for free (the same place enforce lives). Precedence mirrors schema/strategy: the model's own [tags] override the group baseline per key (sidecar > group) without dropping the rest of the group's tags.
  • New ModelDetail.tags on the rocky compile JSON output; regenerated schema + Pydantic (rocky-sdk) + TS (vscode) bindings via just codegen.
  • 33 ModelConfig literal sites updated (no Default; mechanical empty-map adds).

Verification

  • cargo test --all-features green across rocky-core / rocky-compiler / rocky-cli / rocky-ai / rocky-server.
  • cargo fmt --check + cargo clippy --all-targets --all-features clean.
  • Dual-path proof (guards the feat(engine): config groups — one definition fans out routing + strategy to N models #917-class "wired for .sql only" gap): unit tests on the .sql path (merge precedence + model-only tags), a salsa-loader regression on the .rocky path, and an end-to-end rocky compile check that a .sql and a .rocky member of the same group both resolve the group's tags.

Follow-up

dagster-rocky translator reads ModelDetail.tags and projects them onto the derived AssetSpec.tags (separate PR — the JSON contract this PR establishes is the seam).

🤖 Generated with Claude Code

Add a `[tags]` block to models and config groups so a governance
attribute (domain, tier, owner, …) declared once on a group lands on its
whole fan-out, with per-model overrides.

- `GroupConfig.tags` + `RawModelConfig.tags` + `ModelConfig.tags`
  (all `#[serde(default)]`), merged in `resolve_model_config` — the
  shared `.sql`/`.rocky` resolution chokepoint — so both loaders inherit
  for free. Precedence mirrors schema/strategy: model `[tags]` override
  the group baseline per key (sidecar > group) without dropping the rest.
- Surfaced on `rocky compile --output json` as `models_detail[].tags`
  (new `ModelDetail.tags`); regenerated schema + Pydantic + TS bindings
  via the codegen cascade. Lets `dagster-rocky` project them onto each
  derived asset's Dagster tags (follow-up PR).
- Dual-path proof: unit tests on the `.sql` path (merge precedence,
  model-only tags) and a salsa-loader regression on the `.rocky` path,
  plus an end-to-end `rocky compile` check that a `.sql` and a `.rocky`
  member of the same group both resolve the group's tags.

Docs: `[tags]` and "Group tags" sections in the model-format reference.
@hugocorreia90 hugocorreia90 merged commit 415c3a4 into main Jun 18, 2026
18 checks passed
@hugocorreia90 hugocorreia90 deleted the feat/engine/governance-tags branch June 18, 2026 00:31
hugocorreia90 added a commit that referenced this pull request Jun 18, 2026
…ster

Engine [Unreleased] gets the [tags] + group-inheritance entry (#920);
sdk + dagster [Unreleased] get the ModelDetail.tags field and the
asset-spec projection (#921).
hugocorreia90 added a commit that referenced this pull request Jun 18, 2026
…#921)

* feat(dagster): project model governance tags onto derived asset specs

Completes the C4 governed-materialization fan-out end-to-end: a model's
resolved `[tags]` (its own, inherited from its config group) now reach the
Dagster asset.

- sdk: add `tags: dict[str, str] | None` to the hand-written
  `rocky_sdk.types.ModelDetail` (the curated model `parse_rocky_output`
  builds and dagster consumes). Populated automatically from the engine's
  `rocky compile` JSON `models_detail[].tags`.
- dagster: `RockyDagsterTranslator.get_model_tags` projects governance tags
  as *first-class* Dagster tags (usable in `tag:domain=finance` selection),
  alongside the synthesized `rocky/*` metadata. Keys are sanitized to the
  Dagster-valid charset; synthesized keys keep their `/`, so a governance
  tag can never clobber Rocky's own metadata. Read via `getattr` so an older
  rocky-sdk degrades to "no governance tags" instead of crashing.

Tests: sdk parse (tags present / absent→None); dagster build_model_specs
projection, no-tags case, and key-sanitization / no-clobber edge cases.

Release ordering: ship rocky-sdk (new field) before the dagster-rocky cut
that floors on it.

* docs(changelog): record model governance tags across engine, sdk, dagster

Engine [Unreleased] gets the [tags] + group-inheritance entry (#920);
sdk + dagster [Unreleased] get the ModelDetail.tags field and the
asset-spec projection (#921).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant