Skip to content

Commit fea283d

Browse files
stack72claude
andauthored
feat: validate step inputs against method/workflow required arguments (#1151)
## Summary - `swamp workflow validate` now checks that each step's `inputs:` block provides all required arguments for the target method or workflow - For `model_method` tasks: resolves model type, looks up method's Zod argument schema, reports missing required fields - For `workflow` tasks: looks up nested workflow's JSON Schema inputs, reports missing required inputs - Dynamic CEL references (`${{ }}`) in model/workflow names are gracefully skipped - Models/workflows not found produce pass-with-skip (not hard failure) - Updates swamp-workflow skill docs with new validation checks - Fixes vary integration test to provide required `run` input Closes #40 ## Test Plan - 27 unit tests for `DefaultWorkflowValidationService` (13 new for step input validation covering both task types) - 4 libswamp validate integration tests (updated for async) - Full test suite passes (4249 tests) - Manual verification with compiled binary against a scratch repo: ### Missing required input (`run` omitted from `command/shell` model) ``` $ swamp workflow validate bad-workflow --repo-dir /tmp/swamp-repro-issue-40 Validating: bad-workflow ✓ Schema validation ✓ Unique job names ✓ Unique step names in job 'deploy' ✓ Valid job dependency references ✓ Valid step dependency references in job 'deploy' ✓ No cyclic job dependencies ✓ No cyclic step dependencies in job 'deploy' ✗ Step inputs for 'deploy-step' in job 'deploy' (my-deployer.execute) → Missing required inputs: run Summary: 7/8 validations passed Result: FAILED ``` ### All required inputs present ``` $ swamp workflow validate good-workflow --repo-dir /tmp/swamp-repro-issue-40 Validating: good-workflow ✓ Schema validation ✓ Unique job names ✓ Unique step names in job 'deploy' ✓ Valid job dependency references ✓ Valid step dependency references in job 'deploy' ✓ No cyclic job dependencies ✓ No cyclic step dependencies in job 'deploy' ✓ Step inputs for 'deploy-step' in job 'deploy' (my-deployer.execute) Summary: 8/8 validations passed Result: PASSED ``` ### Nonexistent method on model ``` $ swamp workflow validate bad-method-workflow --repo-dir /tmp/swamp-repro-issue-40 Validating: bad-method-workflow ✓ Schema validation ✓ Unique job names ✓ Unique step names in job 'job1' ✓ Valid job dependency references ✓ Valid step dependency references in job 'job1' ✓ No cyclic job dependencies ✓ No cyclic step dependencies in job 'job1' ✗ Step inputs for 'step1' in job 'job1' (my-deployer.nonexistent_method) → Method 'nonexistent_method' not found on model type 'command/shell' Summary: 7/8 validations passed Result: FAILED ``` 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 85b004c commit fea283d

7 files changed

Lines changed: 757 additions & 45 deletions

File tree

.claude/skills/swamp-workflow/SKILL.md

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -207,7 +207,22 @@ swamp workflow delete my-workflow --json
207207

208208
## Validate Workflows
209209

210-
Validate against schema and check for errors.
210+
Validate against schema, check for structural errors, and verify that step
211+
inputs match required method/workflow arguments.
212+
213+
**Checks performed:**
214+
215+
1. Schema validation (Zod)
216+
2. Unique job names
217+
3. Unique step names within each job
218+
4. Valid job dependency references
219+
5. Valid step dependency references
220+
6. No cyclic job dependencies
221+
7. No cyclic step dependencies within jobs
222+
8. Step inputs match required arguments — for `model_method` tasks, checks that
223+
all required method arguments are provided in the step's `inputs:` block. For
224+
`workflow` tasks, checks that all required workflow inputs are provided.
225+
Dynamic CEL references (`${{ ... }}`) in model/workflow names are skipped.
211226

212227
```bash
213228
swamp workflow validate my-workflow --json
@@ -224,7 +239,11 @@ swamp workflow validate --json # Validate all
224239
{ "name": "Schema validation", "passed": true },
225240
{ "name": "Unique job names", "passed": true },
226241
{ "name": "Valid job dependency references", "passed": true },
227-
{ "name": "No cyclic job dependencies", "passed": true }
242+
{ "name": "No cyclic job dependencies", "passed": true },
243+
{
244+
"name": "Step inputs for 'deploy' in job 'release' (my-app.deploy)",
245+
"passed": true
246+
}
228247
],
229248
"passed": true
230249
}

integration/vary_test.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -582,6 +582,7 @@ Deno.test("CLI: vary schema validates in workflow YAML", async () => {
582582
modelIdOrName: "test-model",
583583
methodName: "execute",
584584
inputs: {
585+
run: "echo hello",
585586
region: "us-east-1",
586587
},
587588
},

src/cli/commands/workflow_validate.ts

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,10 @@ export const workflowValidateCommand = new Command()
4949
});
5050

5151
const ctx = createLibSwampContext({ logger: cliCtx.logger });
52-
const deps = createWorkflowValidateDeps(repoContext.workflowRepo);
52+
const deps = createWorkflowValidateDeps(
53+
repoContext.workflowRepo,
54+
repoContext.definitionRepo,
55+
);
5356

5457
const renderer = createWorkflowValidateRenderer(cliCtx.outputMode);
5558
await consumeStream(

src/domain/workflows/validation_service.ts

Lines changed: 188 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,12 +19,15 @@
1919

2020
import type { Workflow } from "./workflow.ts";
2121
import { WorkflowSchema } from "./workflow.ts";
22+
import type { WorkflowRepository } from "./repositories.ts";
23+
import { createWorkflowId } from "./workflow_id.ts";
2224
import {
2325
CyclicDependencyError,
2426
DuplicateNodeNameError,
2527
type GraphNode,
2628
TopologicalSortService,
2729
} from "./topological_sort_service.ts";
30+
2831
/**
2932
* Value object representing the result of a single validation.
3033
*/
@@ -61,6 +64,28 @@ export class WorkflowValidationResult {
6164
}
6265
}
6366

67+
/**
68+
* Result of resolving a method's required arguments.
69+
*/
70+
export type MethodResolution =
71+
| { status: "resolved"; requiredArgs: string[] }
72+
| { status: "model_not_found" }
73+
| { status: "method_not_found"; modelType: string }
74+
| { status: "type_unresolvable"; modelType: string };
75+
76+
/**
77+
* Port interface for resolving method argument schemas.
78+
*
79+
* Abstracts model type resolution so the validation service can look up
80+
* method argument schemas without depending on infrastructure.
81+
*/
82+
export interface ModelMethodResolver {
83+
resolve(
84+
modelIdOrName: string,
85+
methodName: string,
86+
): Promise<MethodResolution>;
87+
}
88+
6489
/**
6590
* Domain service for workflow validation.
6691
*
@@ -72,6 +97,7 @@ export class WorkflowValidationResult {
7297
* 5. Valid step dependency references
7398
* 6. No cyclic dependencies between jobs
7499
* 7. No cyclic dependencies between steps within jobs
100+
* 8. Step inputs match method/workflow required arguments
75101
*/
76102
export interface WorkflowValidationService {
77103
/**
@@ -80,7 +106,7 @@ export interface WorkflowValidationService {
80106
* @param workflow The workflow to validate
81107
* @returns Array of validation results
82108
*/
83-
validate(workflow: Workflow): WorkflowValidationResult[];
109+
validate(workflow: Workflow): Promise<WorkflowValidationResult[]>;
84110
}
85111

86112
/**
@@ -90,7 +116,12 @@ export class DefaultWorkflowValidationService
90116
implements WorkflowValidationService {
91117
private readonly sortService = new TopologicalSortService();
92118

93-
validate(workflow: Workflow): WorkflowValidationResult[] {
119+
constructor(
120+
private readonly methodResolver?: ModelMethodResolver,
121+
private readonly workflowRepo?: WorkflowRepository,
122+
) {}
123+
124+
async validate(workflow: Workflow): Promise<WorkflowValidationResult[]> {
94125
const results: WorkflowValidationResult[] = [];
95126

96127
// 1. Schema validation
@@ -114,6 +145,11 @@ export class DefaultWorkflowValidationService
114145
// 7. No cyclic step dependencies within jobs
115146
results.push(...this.validateNoStepCycles(workflow));
116147

148+
// 8. Step inputs match required arguments
149+
if (this.methodResolver || this.workflowRepo) {
150+
results.push(...await this.validateStepInputs(workflow));
151+
}
152+
117153
return results;
118154
}
119155

@@ -319,4 +355,154 @@ export class DefaultWorkflowValidationService
319355

320356
return results;
321357
}
358+
359+
private async validateStepInputs(
360+
workflow: Workflow,
361+
): Promise<WorkflowValidationResult[]> {
362+
const results: WorkflowValidationResult[] = [];
363+
364+
for (const job of workflow.jobs) {
365+
for (const step of job.steps) {
366+
const task = step.task;
367+
if (!task) continue;
368+
369+
const taskData = task.data;
370+
if (taskData.type === "model_method" && this.methodResolver) {
371+
results.push(
372+
...await this.validateModelMethodInputs(
373+
job.name,
374+
step.name,
375+
taskData.modelIdOrName,
376+
taskData.methodName,
377+
taskData.inputs,
378+
),
379+
);
380+
} else if (taskData.type === "workflow" && this.workflowRepo) {
381+
results.push(
382+
...await this.validateWorkflowTaskInputs(
383+
job.name,
384+
step.name,
385+
taskData.workflowIdOrName,
386+
taskData.inputs,
387+
),
388+
);
389+
}
390+
}
391+
}
392+
393+
return results;
394+
}
395+
396+
private async validateModelMethodInputs(
397+
jobName: string,
398+
stepName: string,
399+
modelIdOrName: string,
400+
methodName: string,
401+
inputs: Record<string, unknown> | undefined,
402+
): Promise<WorkflowValidationResult[]> {
403+
const checkName =
404+
`Step inputs for '${stepName}' in job '${jobName}' (${modelIdOrName}.${methodName})`;
405+
406+
// Skip dynamic CEL references — cannot resolve statically
407+
if (modelIdOrName.includes("${{")) {
408+
return [WorkflowValidationResult.pass(checkName)];
409+
}
410+
411+
const resolution = await this.methodResolver!.resolve(
412+
modelIdOrName,
413+
methodName,
414+
);
415+
416+
switch (resolution.status) {
417+
case "model_not_found":
418+
return [
419+
WorkflowValidationResult.pass(
420+
checkName +
421+
" (model not found, skipped)",
422+
),
423+
];
424+
case "type_unresolvable":
425+
return [
426+
WorkflowValidationResult.pass(
427+
checkName +
428+
" (model type not resolved, skipped)",
429+
),
430+
];
431+
case "method_not_found":
432+
return [
433+
WorkflowValidationResult.fail(
434+
checkName,
435+
`Method '${methodName}' not found on model type '${resolution.modelType}'`,
436+
),
437+
];
438+
case "resolved": {
439+
const inputKeys = new Set(Object.keys(inputs ?? {}));
440+
const missing = resolution.requiredArgs.filter((arg) =>
441+
!inputKeys.has(arg)
442+
);
443+
if (missing.length > 0) {
444+
return [
445+
WorkflowValidationResult.fail(
446+
checkName,
447+
`Missing required inputs: ${missing.join(", ")}`,
448+
),
449+
];
450+
}
451+
return [WorkflowValidationResult.pass(checkName)];
452+
}
453+
}
454+
}
455+
456+
private async validateWorkflowTaskInputs(
457+
jobName: string,
458+
stepName: string,
459+
workflowIdOrName: string,
460+
inputs: Record<string, unknown> | undefined,
461+
): Promise<WorkflowValidationResult[]> {
462+
const checkName =
463+
`Step inputs for '${stepName}' in job '${jobName}' (workflow: ${workflowIdOrName})`;
464+
465+
// Skip dynamic CEL references
466+
if (workflowIdOrName.includes("${{")) {
467+
return [WorkflowValidationResult.pass(checkName)];
468+
}
469+
470+
// Try to find the nested workflow
471+
let nested: Workflow | null = null;
472+
try {
473+
nested = await this.workflowRepo!.findByName(workflowIdOrName) ??
474+
await this.workflowRepo!.findById(
475+
createWorkflowId(workflowIdOrName),
476+
);
477+
} catch {
478+
// ID may not be a valid UUID — that's fine, just not found
479+
}
480+
481+
if (!nested) {
482+
return [
483+
WorkflowValidationResult.pass(
484+
checkName +
485+
" (workflow not found, skipped)",
486+
),
487+
];
488+
}
489+
490+
const requiredInputs = nested.inputs?.required ?? [];
491+
if (requiredInputs.length === 0) {
492+
return [WorkflowValidationResult.pass(checkName)];
493+
}
494+
495+
const inputKeys = new Set(Object.keys(inputs ?? {}));
496+
const missing = requiredInputs.filter((arg) => !inputKeys.has(arg));
497+
if (missing.length > 0) {
498+
return [
499+
WorkflowValidationResult.fail(
500+
checkName,
501+
`Missing required workflow inputs: ${missing.join(", ")}`,
502+
),
503+
];
504+
}
505+
506+
return [WorkflowValidationResult.pass(checkName)];
507+
}
322508
}

0 commit comments

Comments
 (0)