feat(semantic-layer): add reference-data commands (Chart of Accounts in the metastore)

CLAUDE.md

@@ -512,6 +512,10 @@ kbagent semantic-layer import --project P --file PATH [--model M] [--types T,T,.
 kbagent semantic-layer promote --from-project A --to-project B [--from-model M] [--to-model M] [--types T,T,...] [--dry-run] [--yes]
 kbagent semantic-layer build --project P [--model M] --tables T,T,... [--name N] [--dry-run] [--keep-on-failure] [--output PATH]
 kbagent semantic-layer token --encrypt --project P --component-id C
+kbagent semantic-layer reference-data list --project P [--model M]
+kbagent semantic-layer reference-data get --project P (--id ID | --dimension D)
+kbagent semantic-layer reference-data set --project P [--model M] --dimension D --members-file PATH [--dataset-id T] [--description X]
+kbagent semantic-layer reference-data delete --project P --id ID [--yes]
 # Alias: `kbagent sl ...` (hidden) is equivalent to `kbagent semantic-layer ...`.
 
 kbagent http get PATH [--timeout SECONDS]

plugins/kbagent/skills/kbagent/SKILL.md

@@ -60,6 +60,8 @@ description: >
   DIM_METRIC_THRESHOLD, dangling metric FK, orphaned constraint,
   phantom field, AGG on STRING, SUM on PCT, deep validate,
   sl, kbagent sl, semantic layer wizard, sl-build, sl-add, sl-edit,
+  reference data, semantic-layer reference-data, chart of accounts, COA,
+  dimension members, account list, dimension data, member list,
   developer portal, dev-portal, apps-api, register component, vendor app,
   portal property, ui-options, encryption portal, defaultBucket portal,
   app icon, configurationSchema portal, publish component, deprecate component,
@@ -297,6 +299,10 @@ When working inside a git repository or project directory, run `kbagent init` (o
 | Remove a constraint | `kbagent semantic-layer remove constraint --project PROJECT --name NAME` |
 | Remove a relationship. | `kbagent semantic-layer remove relationship --project PROJECT --name NAME` |
 | Remove a glossary term. | `kbagent semantic-layer remove glossary --project PROJECT --term TERM` |
+| List reference-data records (dimension summaries; use ``get`` for members) | `kbagent semantic-layer reference-data list --project PROJECT` |
+| Fetch one record (all members) by ``--id`` or by ``--dimension`` | `kbagent semantic-layer reference-data get --project PROJECT` |
+| Create or replace a reference-data record (keyed by dimension) | `kbagent semantic-layer reference-data set --project PROJECT --dimension DIMENSION --members-file MEMBERS-FILE` |
+| Delete a reference-data record by UUID (server-side soft-delete) | `kbagent semantic-layer reference-data delete --project PROJECT --id ID-` |
 | Encrypt the project's storage token for transformation `user_properties` | `kbagent sl token --project PROJECT --component-id COMPONENT-ID` |
 | Build a semantic-layer model from a list of storage tables (non-interactive) | `kbagent sl build --project PROJECT` |
 | Promote a model from one project to another (NEW + overwrite CHANGED; never deletes) | `kbagent sl promote --from-project FROM-PROJECT --to-project TO-PROJECT` |
@@ -325,6 +331,10 @@ When working inside a git repository or project directory, run `kbagent init` (o
 | Remove a constraint | `kbagent sl remove constraint --project PROJECT --name NAME` |
 | Remove a relationship. | `kbagent sl remove relationship --project PROJECT --name NAME` |
 | Remove a glossary term. | `kbagent sl remove glossary --project PROJECT --term TERM` |
+| List reference-data records (dimension summaries; use ``get`` for members) | `kbagent sl reference-data list --project PROJECT` |
+| Fetch one record (all members) by ``--id`` or by ``--dimension`` | `kbagent sl reference-data get --project PROJECT` |
+| Create or replace a reference-data record (keyed by dimension) | `kbagent sl reference-data set --project PROJECT --dimension DIMENSION --members-file MEMBERS-FILE` |
+| Delete a reference-data record by UUID (server-side soft-delete) | `kbagent sl reference-data delete --project PROJECT --id ID-` |
 | GET an endpoint on the running kbagent serve | `kbagent http get <PATH>` |
 | POST to an endpoint on the running kbagent serve | `kbagent http post <PATH>` |
 | PATCH an endpoint on the running kbagent serve | `kbagent http patch <PATH>` |

plugins/kbagent/skills/kbagent/references/commands-reference.md

@@ -279,6 +279,15 @@ Manage Keboola metastore models -- datasets, metrics, relationships, constraints
 - `semantic-layer build --project P [--model M] --tables T,T,... [--name N] [--dry-run] [--keep-on-failure] [--output PATH]` -- non-interactive heuristic builder. **AI caveat**: the existing `ai_client` has no arbitrary-JSON endpoint, so `build` falls back to a deterministic heuristic synthesising one dataset + one COUNT(*) metric + one glossary entry per table (FQN derived; fields[] role-classified). Response carries `fallback_used: "heuristic"`. The push loop walks all 5 child types in dependency order -- this **fixes** the `sl-build` skill bug where `semantic-constraint` was silently dropped. `--model` omitted creates a new model (default name `kbagent_build_model` or `--name N`). **Rollback on push failure (since v0.41.10)**: every successfully-POSTed child is DELETEd in reverse PUSH_ORDER, and the model itself is DELETEd if we created it during this call. The wrapped `KeboolaApiError` carries `details.rollback={attempted, posted_children, deleted, failed_deletes, model_created_here, model_deleted, model_uuid}` so operators get full diagnostics. Pass `--keep-on-failure` to preserve the partial state for forensic inspection (mirrors `data-app create --keep-on-failure`); the wrapped error then carries `details.rollback.attempted=False, reason='keep_on_failure'`.
 - `semantic-layer token --encrypt --project P --component-id C` -- encrypt the project's storage token for a transformation's `user_properties`. Builds `{"#metastore_token": <token>}` from the project's already-stored Storage API token and delegates to the existing EncryptService. `--encrypt` is currently required; other modes are refused with `USAGE_ERROR` (exit 2). Output (human): the raw envelope ready to paste. JSON: full `{encrypted, component_id, project}`.
 
+### Reference data (dimension members, e.g. a Chart of Accounts) (since 0.55.0)
+
+`semantic-reference-data` is a per-dimension member store: ONE record per dimension holding the full member list in a `members[]` array. The driving use case is a Chart of Accounts (the account list + all attributes) held in the metastore instead of a hardcoded Storage table. It is deliberately kept **outside** `build` / `export` / `diff` / cascade / `PUSH_ORDER` — its members come from `DIM_COA`, not from AI generation — so it has its own self-contained CRUD surface. Member field names mirror the `DIM_COA` columns 1:1 (snake_case: `account_code`, `account_name`, `parent_code`, `is_leaf`, `level_1_code`, `cf_category`, …).
+
+- `semantic-layer reference-data list --project P [--model M]` -- list dimension records (summaries: `id`, `dimension_name`, `model_uuid`, `dataset_id`, `member_count`). `--model` filters to one model. Members are omitted from the summary; use `get` for them.
+- `semantic-layer reference-data get --project P (--id ID | --dimension D)` -- fetch one record with all members. Resolve by record UUID (`--id`) or by `--dimension`. The dimension is **unique per project**, so the lookup is project-wide and needs no model. Passing both `--id` and `--dimension` (or neither) is a usage error (exit 2). Returns `{id, dimension_name, model_uuid, dataset_id, member_count, revision, members[]}`.
+- `semantic-layer reference-data set --project P [--model M] --dimension D --members-file PATH [--dataset-id T] [--description X]` -- create-or-replace. `--members-file` is a JSON array of member objects (`-` reads stdin). **Idempotent** on `dimension`: the lookup is project-wide (the envelope `name` = dimension is unique per project per type), so an existing record is replaced in place via `PUT` (the metastore increments `meta.revision`, preserving history) — distinct from the DELETE+POST used by `edit` — regardless of which `--model` is passed (the resolved model is stored on the record); otherwise a new record is `POST`-ed. Response: `{id, dimension_name, member_count, action: "created"|"updated"}`.
+- `semantic-layer reference-data delete --project P --id ID [--yes]` -- delete by UUID (server-side soft-delete; the record stays in revision history). Non-TTY without `--yes` refuses with exit 2.
+
 ## Self-Call HTTP (inside `kbagent serve` subprocesses; since v0.40.0)
 - `http get PATH [--timeout SECONDS]` -- GET an endpoint on the running `kbagent serve`
 - `http post PATH [--body JSON|@file|-] [--timeout SECONDS]` -- POST with optional JSON body

plugins/kbagent/skills/kbagent/references/gotchas.md

@@ -144,6 +144,43 @@ project's semantic model is populated before kicking off a downstream
 pipeline; the previous workaround (a `keboola-mcp-server` MCP server entry
 in `.mcp.json` solely for these two tools) can be dropped.
 
+## `semantic-layer reference-data` holds a whole dimension as ONE record; `set` is PUT-replace, not append (since v0.55.0)
+
+`semantic-reference-data` stores one record **per dimension** (e.g. a Chart
+of Accounts), with the full member list in a `members[]` array — NOT one
+record per member. Consequences an agent must internalize:
+
+- **`set` replaces the entire members array.** `kbagent sl reference-data
+  set --dimension chart_of_accounts --members-file coa.json` is
+  create-or-replace, idempotent on the `dimension`. To add/remove a
+  single account you must `get` the record, mutate the array client-side,
+  and `set` the whole thing back. There is no per-member endpoint.
+- **It uses the metastore's real `PUT`** (revisioned update, `meta.revision`
+  increments, history preserved) when a record for that dimension already
+  exists — distinct from the DELETE+POST that `edit metric|…` uses.
+  A brand-new dimension is `POST`-ed.
+- **The envelope `name` is the dimension, unique per project per type, so
+  the `set`/`get` lookup is project-wide.** Because the dimension name is the
+  project-unique key, `set` finds and PUT-replaces an existing record
+  regardless of which `--model` you pass (the resolved model is just stored
+  on the record) — it does NOT POST and collide with `ALREADY_EXISTS`. For
+  the same reason `get --dimension` needs no `--model` (one dimension name
+  per project for this type). `get` rejects passing both `--id` and
+  `--dimension` (exit 2).
+- **Member field names mirror the `DIM_COA` columns 1:1 (snake_case):**
+  `account_code` (required key), `account_name`, `parent_code`, `is_leaf`
+  (integer 0/1, not bool), `level_1_code`, `cf_category`, … The metastore
+  schema sets `additionalProperties: true` on members, so unknown columns
+  are stored but not validated.
+- **Deliberately invisible to `build` / `export` / `diff` / cascade /
+  `PUSH_ORDER`.** Deleting a model does NOT cascade-delete its
+  reference-data records (they are not model children in the snapshot
+  sense); `export`/`diff` will not include them. Manage them only through
+  the `reference-data` sub-app.
+- **JSON Schema cannot enforce cross-member referential integrity** (every
+  `parent_code` resolving to some member `account_code`). That stays an
+  app/sync-layer concern.
+
 ## `workspace list` / `workspace detail` now expose loginType + RO + qs_compatible (since v0.42.0, closes #304)
 
 Before v0.42.0 the Storage workspace endpoint already returned