Skip to content

fix(engine): defer validator fail to inconclusive while the linked task is unmerged#1917

Merged
gsxdsm merged 2 commits into
Runfusion:mainfrom
flexi767:fix/validator-premerge-guard
Jul 6, 2026
Merged

fix(engine): defer validator fail to inconclusive while the linked task is unmerged#1917
gsxdsm merged 2 commits into
Runfusion:mainfrom
flexi767:fix/validator-premerge-guard

Conversation

@flexi767

@flexi767 flexi767 commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Problem

MissionExecutionLoop.runFeatureValidation treats a validator "fail" verdict as authoritative regardless of whether the linked task's code has actually landed. When validation fires while the task is still mid-pipeline — an in-review PR, an external merge train, a deferred base sync — the validator judges a checkout that predates the merge, concludes the feature "is not present", and handleValidationFail mints a Fix feature for work that is already done.

We hit this in production (2026-07-05): a recovery-path validation ran against four features whose implementing tasks were in-review in an external merge pipeline. All four "failed" → four duplicate Fix tasks were created one minute after the real work merged. Worse, the Fix tasks' planned file scopes included hot shared files, so their file-scope leases serialized the entire board until they were manually archived.

Fix

Before dispatching a fail verdict, resolve the linked task's column. If it affirmatively shows the task has not completed (any column other than done/archived), route the outcome to handleValidationInconclusive (R21 — completes the run as blocked, logs verification_inconclusive, notifies autopilot, spawns no Fix feature) with a "code not merged yet — validation deferred" reason. A later validation (post-merge recovery pass) judges the real merged code.

Fails open by design — the guard may only ever defer a fail, never suppress one on missing data. Missing taskId, missing task, unreadable store, or unknown column all fall through to the normal handleValidationFail path:

private async getPremergeTaskColumn(taskId: string | undefined): Promise<string | null> {
  if (!taskId) return null;
  const linkedTask = await this.taskStore.getTask(taskId).catch(() => null);
  const column = linkedTask?.column;
  if (!column || column === "done" || column === "archived") return null;
  return column;
}

The vanilla flow is unaffected: the scheduler triggers validation on toColumn === "done", so by the time a normally-triggered validation runs the task is already done and the guard is a no-op. Only recovery-path / re-validation runs that race an unmerged task are deferred.

Tests

Three new tests in mission-execution-loop.test.ts (premerge guard describe):

  1. fail verdict + linked task in-review → routes to inconclusive: no Fix feature, run completed as blocked, validation:inconclusive emitted (not validation:failed), verification_inconclusive mission event logged
  2. fail verdict + linked task done → normal fail path: Fix feature created, validation:failed emitted
  3. fail verdict + taskStore.getTask rejects → fails open to the normal fail path

npx vitest run src/__tests__/mission-execution-loop.test.ts: 58/58 green. npx tsc --noEmit: clean. Full engine suite: the 46 failures across 24 files present on my branch fail identically on clean 17c4007 (verified by re-running the same files on a detached checkout of upstream main) — all pre-existing/environment-dependent, none related to this change.

Summary by CodeRabbit

  • Bug Fixes
    • Validation failures now account for whether the linked task is actually merged. If the task is still in progress, the result is marked as inconclusive instead of creating a fix flow.
    • Added clearer handling when task details can’t be read, so normal failure behavior still applies.
    • Improved validation status reporting and event logging for merged vs. unmerged task states.

…sk is unmerged

A "fail" verdict from runFeatureValidation is only trustworthy once the
linked task's code has actually landed (column done/archived). When the
task is still mid-pipeline — an in-review PR, an external merge train, a
deferred base sync — the validator judged a checkout that predates the
merge and reports the work as missing, minting a duplicate Fix feature
(and board task) for code that is about to land.

Route that case to handleValidationInconclusive (R21: no Fix feature,
run completed as blocked, distinguishable verification_inconclusive
event) so a later validation judges the merged code instead.

The guard fails open: a missing/unlinked task, an unreadable task store,
or an unknown column all keep the existing handleValidationFail path —
it can only ever defer a fail on affirmative evidence of an unmerged
column, never suppress one on missing data. The vanilla done-triggered
flow (scheduler fires processTaskOutcome on toColumn === "done") is
unaffected; the guard matters for recovery-path validations and for
deployments whose tasks merge through external pipelines.

Incident context: a mission validator racing an external merge train
failed four features while their tasks were in-review, and the four
generated Fix tasks' file-scope leases on hot shared files serialized
the entire board.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 5, 2026 16:54

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 4b4b8832-9479-46b5-b37f-f1c90f9593ec

📥 Commits

Reviewing files that changed from the base of the PR and between 17c4007 and c73c700.

📒 Files selected for processing (2)
  • packages/engine/src/__tests__/mission-execution-loop.test.ts
  • packages/engine/src/mission-execution-loop.ts

📝 Walkthrough

Walkthrough

This change modifies MissionExecutionLoop.runFeatureValidation to check the linked task's board column before handling a validator "fail" verdict. A new getPremergeTaskColumn helper determines whether the fail should be deferred as inconclusive or processed normally. Corresponding tests are added.

Changes

Premerge Guard for Validation Fail Verdict

Layer / File(s) Summary
Premerge column check and fail routing
packages/engine/src/mission-execution-loop.ts
Adds getPremergeTaskColumn(taskId) to resolve a linked task's column, returning it only when the task is not done/archived; the "fail" branch now routes to handleValidationInconclusive when the task is pre-merge, otherwise falls back to handleValidationFail, failing open when the task is missing or unreadable.
Premerge guard test suite
packages/engine/src/__tests__/mission-execution-loop.test.ts
Adds tests priming a fail verdict with a linked task: verifies deferral to inconclusive (blocked status, validation:inconclusive emission, warning event) when task is not done, verifies normal fail flow (validation:failed) when task is done, and verifies fail-open behavior when the linked task cannot be read.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

  • Runfusion/Fusion#1345: Both PRs modify runFeatureValidation in mission-execution-loop.ts, one centralizing the flow and this one adding column-based fail routing.
  • Runfusion/Fusion#1666: Both PRs change how the validator's "fail" outcome is routed to other terminal/inconclusive states within the same file.

Poem

A task not merged, still in review,
the guard says "wait, don't jump the queue!"
Inconclusive whispers, warning logged,
but once it's done, the fail's not dodged.
Fail open wide when data's lost —
a rabbit's caution, worth the cost. 🐇

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main behavior change in the validator flow.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR changes validator failure handling when linked task code may not be merged yet. The main changes are:

  • Added a guard before handling validator fail results.
  • Routes failures for non-completed linked tasks to the inconclusive path.
  • Keeps normal failure handling for completed, missing, or unreadable linked tasks.
  • Added tests for deferred, completed, and task-store failure cases.

Confidence Score: 5/5

This looks safe to merge.

  • No blocking issues found in the changed code.
  • The only concrete concern found during this follow-up matches an existing review thread and is not a separate issue.

Important Files Changed

Filename Overview
packages/engine/src/mission-execution-loop.ts Adds the linked-task column check before converting validator failures into generated Fix features.
packages/engine/src/tests/mission-execution-loop.test.ts Adds focused tests for the new pre-merge validation guard.

Reviews (2): Last reviewed commit: "Merge branch 'main' into fix/validator-p..." | Re-trigger Greptile

Comment thread packages/engine/src/mission-execution-loop.ts
@gsxdsm gsxdsm merged commit 9b7921b into Runfusion:main Jul 6, 2026
7 checks passed
gsxdsm pushed a commit that referenced this pull request Jul 6, 2026
…the merged code

The mission validator runs its read-only judge session with cwd:
this.rootDir — the engine's main working copy. When a task's merge landed
on the remote or in another worktree and rootDir was never fetched/reset to
it, the judge reads PRE-merge files and returns a spurious `fail`. #1917's
premerge column guard doesn't catch this: the task column is already `done`,
so it falls through to handleValidationFail and mints a bogus Fix Feature.

Add a symmetric second guard in the fail branch, after the premerge column
check: isValidationWorkspaceStale resolves the task's integration SHA and
runs `git merge-base --is-ancestor <sha> HEAD` in rootDir. Only affirmative
staleness evidence (exit 1 = NOT an ancestor) defers the fail to
inconclusive; exit 0 (ancestor/fresh), a missing SHA, or a bad object
(exit 128) all trust the fail. Fail-open: a guard may DEFER a fail, never
SUPPRESS one on missing or unreadable data.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
gsxdsm added a commit that referenced this pull request Jul 6, 2026
…the merged code (#1929)

## Problem

The mission validator runs its read-only judge session with `cwd:
this.rootDir` — the engine's **main working copy**. When a task's merge
landed on the remote (or in another worktree) and `rootDir` was never
fetched/reset to that commit, the judge reads **pre-merge files** and
returns a spurious `fail`.

#1917's premerge column guard does not catch this case: by the time
validation runs the task column is already `done`, so execution falls
through to `handleValidationFail` and mints a **bogus Fix Feature** for
code that is actually correct and merged.

## Fix

A symmetric second guard in the `fail` branch, placed **after** the
#1917 premerge column check:

- `isValidationWorkspaceStale(feature)` resolves the task's integration
SHA and runs `git merge-base --is-ancestor <sha> HEAD` in `rootDir`.
- **Only affirmative staleness evidence defers.** `--is-ancestor` exit
`1` (the SHA is *not* an ancestor of HEAD → the workspace predates the
merge) → defer the fail to **inconclusive**, so a later validation
judges the merged code.
- Every other outcome trusts the fail: exit `0` (ancestor → workspace is
fresh), no integration SHA available, or a bad/unknown object (exit
`128`).

Fail-open doctrine, matching #1917: a guard may only ever **defer** a
fail, never **suppress** one on missing or unreadable data.

## Tests

Four real-git cases in `mission-execution-loop.test.ts` (skipped when
`git` is unavailable):

1. Judged checkout predates the merged commit → fail deferred to
inconclusive, no Fix Feature minted, emits `validation:inconclusive`.
2. Merged commit is an ancestor of HEAD (fresh workspace) → normal fail
path, Fix Feature minted, emits `validation:failed`.
3. Task carries no integration SHA → fail open (normal fail).
4. Integration SHA is an unknown object (exit 128) → fail open (normal
fail).

Verified with stash-red/restore-green discipline: with the production
guard stashed, case (1) goes red while the three fail-open guardrails
stay green — proving case (1) exercises the fix. Full file: 62 passed.
`tsc --noEmit`: clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Validation failures are now deferred when the workspace appears to be
out of date with merged changes, reducing incorrect failure reports.
* Tasks with linked work continue to use the usual failure path when the
current workspace is up to date.
* Staleness checks now avoid masking real validation failures when no
merge reference is available or when the reference can’t be verified.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants