Skip to content

fix: make CatalogStore required on FileSystemUnifiedDataRepository #1157

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
stack72 merged 1 commit into from
Apr 9, 2026
Merged

fix: make CatalogStore required on FileSystemUnifiedDataRepository #1157

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions design/data-query.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -222,9 +222,10 @@ Every mutation in `UnifiedDataRepository` updates the catalog inline:
| `removeLatestMarker()` | Remove row |
| `collectGarbage()` | Update version or remove row |

The `CatalogStore` is an optional constructor parameter on
`UnifiedDataRepository`. When absent (tests, lightweight contexts), the
repository behaves as before.
The `CatalogStore` is a required constructor parameter on
`UnifiedDataRepository`. Every repository instance maintains write-through
catalog consistency. Use `createCatalogStore()` from `repository_factory.ts`
to construct one from a repo directory.

### Population Strategy

Expand Down
78 changes: 65 additions & 13 deletions integration/cel_data_access_test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,11 @@ Deno.test("CEL Data Access: reference hyphenated model name", async () => {
Deno.test("CEL Data Access: access latest resource via model.X.resource.specName", async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const type = ModelType.create("test/model");
const owner = createOwner("test/model:create");
Expand Down Expand Up @@ -256,7 +260,11 @@ Deno.test("CEL Data Access: access latest resource via model.X.resource.specName
Deno.test("CEL Data Access: access specific version via data.version()", async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const type = ModelType.create("test/model");
const owner = createOwner("test/model:version-access");
Expand Down Expand Up @@ -327,7 +335,11 @@ Deno.test("CEL Data Access: access specific version via data.version()", async (
Deno.test("CEL Data Access: access data via data.latest()", async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const type = ModelType.create("test/model");
const owner = createOwner("test/model:latest");
Expand Down Expand Up @@ -385,7 +397,11 @@ Deno.test("CEL Data Access: access data via data.latest()", async () => {
Deno.test("CEL Data Access: list all versions via data.listVersions()", async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const type = ModelType.create("test/model");
const owner = createOwner("test/model:list");
Expand Down Expand Up @@ -435,7 +451,11 @@ Deno.test("CEL Data Access: list all versions via data.listVersions()", async ()
Deno.test("CEL Data Access: reference resource from dependent model", async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const type = ModelType.create("test/model");
const owner = createOwner("test/model:cross-ref");
Expand Down Expand Up @@ -498,7 +518,11 @@ Deno.test("CEL Data Access: reference resource from dependent model", async () =
Deno.test("CEL Data Access: chain data references across multiple models", async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const type = ModelType.create("test/model");
const owner = createOwner("test/model:chain");
Expand Down Expand Up @@ -752,7 +776,11 @@ Deno.test("CEL Data Access: handle missing model gracefully", async () => {
Deno.test("CEL Data Access: handle missing data gracefully", async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const type = ModelType.create("test/model");

Expand Down Expand Up @@ -785,7 +813,11 @@ Deno.test("CEL Data Access: handle missing data gracefully", async () => {
Deno.test("CEL Data Access: multiple resource items from same model", async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const type = ModelType.create("test/model");
const owner = createOwner("test/model:multi-data");
Expand Down Expand Up @@ -845,7 +877,11 @@ Deno.test(
async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const type = ModelType.create("test/model");
const owner = createOwner("test/model:spec-name-test");
Expand Down Expand Up @@ -904,7 +940,11 @@ Deno.test(
async () => {
await withTempDir(async (repoDir) => {
await initializeTestRepo(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const type = ModelType.create("test/model");
const owner = createOwner("test/model:cli-spec-name-test");
Expand Down Expand Up @@ -965,7 +1005,11 @@ Deno.test(
Deno.test("CEL Data Access: cross-type resource with specName tag via ExpressionEvaluationService (#370)", async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const owner = createOwner("user/s3-inventory:create");

Expand Down Expand Up @@ -1032,7 +1076,11 @@ Deno.test("CEL Data Access: cross-type resource with specName tag via Expression
Deno.test("CEL Data Access: resource resolves after model delete and recreate with new UUID (#370)", async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const inventoryType = ModelType.create("@user/s3-inventory");
const reportType = ModelType.create("@user/s3-report");
Expand Down Expand Up @@ -1109,7 +1157,11 @@ Deno.test("CEL Data Access: resource resolves after model delete and recreate wi
Deno.test("CEL Data Access: data.latest() sees data written after buildContext()", async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const type = ModelType.create("test/model");
const owner = createOwner("test/model:fresh-data");
Expand Down
37 changes: 31 additions & 6 deletions integration/cross_model_data_access_test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ import { DataAccessService } from "../src/domain/data/data_access_service.ts";
import { ModelType } from "../src/domain/models/model_type.ts";
import { Definition } from "../src/domain/definitions/definition.ts";
import { FileSystemUnifiedDataRepository } from "../src/infrastructure/persistence/unified_data_repository.ts";
import { CatalogStore } from "../src/infrastructure/persistence/catalog_store.ts";
import { YamlDefinitionRepository } from "../src/infrastructure/persistence/yaml_definition_repository.ts";
import { computeDefinitionHash } from "../src/domain/models/model_output.ts";

Expand Down Expand Up @@ -92,7 +93,11 @@ Deno.test("cross-model data access: read another model's data by name", async ()
await setupRepoDir(repoDir);

const defRepo = new YamlDefinitionRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const sourceType = ModelType.create("test/source");

// Create source model definition
Expand Down Expand Up @@ -128,7 +133,11 @@ Deno.test("cross-model data access: filter by specName", async () => {
await setupRepoDir(repoDir);

const defRepo = new YamlDefinitionRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const sourceType = ModelType.create("test/source");

const sourceDef = Definition.create({
Expand Down Expand Up @@ -179,7 +188,11 @@ Deno.test("cross-model data access: non-existent model returns empty array", asy
await setupRepoDir(repoDir);

const defRepo = new YamlDefinitionRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);

const service = new DataAccessService(defRepo, dataRepo);
const records = await service.readModelData("nonexistent-model");
Expand All @@ -193,7 +206,11 @@ Deno.test("cross-model data access: model with no data returns empty array", asy
await setupRepoDir(repoDir);

const defRepo = new YamlDefinitionRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const sourceType = ModelType.create("test/empty");

// Create model but don't write any data
Expand All @@ -215,7 +232,11 @@ Deno.test("cross-model data access: workflowRunId scoping returns only matching
await setupRepoDir(repoDir);

const defRepo = new YamlDefinitionRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const sourceType = ModelType.create("test/source");

const sourceDef = Definition.create({
Expand Down Expand Up @@ -290,7 +311,11 @@ Deno.test("cross-model data access: orphan recovery reads content from old UUID
await setupRepoDir(repoDir);

const defRepo = new YamlDefinitionRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const sourceType = ModelType.create("test/orphan");

// Step 1: Create original model and write data
Expand Down
54 changes: 45 additions & 9 deletions integration/data_expression_test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,11 @@ function createOwner(ref: string): OwnerDefinition {
Deno.test("Integration: model.X.resource.specName accesses latest version of resource", async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const type = ModelType.create("test/model");

Expand Down Expand Up @@ -148,7 +152,11 @@ Deno.test("Integration: model.X.resource.specName accesses latest version of res
Deno.test("Integration: data.version() retrieves specific version", async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const type = ModelType.create("test/model");

Expand Down Expand Up @@ -213,7 +221,11 @@ Deno.test("Integration: data.version() retrieves specific version", async () =>
Deno.test("Integration: data.version() returns null for missing version", async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const type = ModelType.create("test/model");

Expand Down Expand Up @@ -261,7 +273,11 @@ Deno.test("Integration: data.version() returns null for missing version", async
Deno.test("Integration: data.latest() retrieves latest version", async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const type = ModelType.create("test/model");

Expand Down Expand Up @@ -324,7 +340,11 @@ Deno.test("Integration: data.latest() retrieves latest version", async () => {
Deno.test("Integration: data.listVersions() returns sorted version numbers", async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const type = ModelType.create("test/model");

Expand Down Expand Up @@ -370,7 +390,11 @@ Deno.test("Integration: data.listVersions() returns sorted version numbers", asy
Deno.test("Integration: data.listVersions() returns empty array for missing data", async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const type = ModelType.create("test/model");

Expand Down Expand Up @@ -400,7 +424,11 @@ Deno.test("Integration: data.listVersions() returns empty array for missing data
Deno.test("Integration: data.findByTag() returns matching records", async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const type = ModelType.create("test/model");

Expand Down Expand Up @@ -509,7 +537,11 @@ Deno.test("Integration: data.findByTag() returns matching records", async () =>
Deno.test("Integration: model can have multiple named data items with mixed types", async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const type = ModelType.create("test/model");

Expand Down Expand Up @@ -604,7 +636,11 @@ Deno.test("Integration: model can have multiple named data items with mixed type
Deno.test("Integration: handles hyphenated model names in data expressions", async () => {
await withTempDir(async (repoDir) => {
await setupRepoDir(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(repoDir);
const dataRepo = new FileSystemUnifiedDataRepository(
repoDir,
undefined,
new CatalogStore(join(repoDir, "_catalog.db")),
);
const definitionRepo = new YamlDefinitionRepository(repoDir);
const type = ModelType.create("test/model");

Expand Down
Loading
Loading