Skip to content

Improve RLCR stagnation handling with evidence contracts and verification preflights #128

@VitalyAnkh

Description

@VitalyAnkh

Summary

A recent RLCR run showed that the methodology was generally effective: reviews caught real issues, feedback was actionable, and the loop correctly distinguished progress from final completion.

However, the loop eventually stagnated around an independently unverifiable acceptance gate. The main issue was not lack of implementation progress, but the absence of a pre-agreed alternative evidence path when a verification step could not be reproduced in the review environment.

Suggested Improvements

  1. Add dual-layer completion states:

    • implementation complete
    • independently verified complete
  2. Add evidence contracts for each round:

    • required commands
    • expected artifacts
    • acceptable substitutes when the primary verification environment is unavailable
  3. Add a repeated-blocker circuit breaker:

    • if the same blocker appears in two consecutive reviews, pause implementation and require a decision: fix, replace evidence path, or mark as environment-limited with explicit acceptance rules
  4. Add early preflight checks for high-risk acceptance gates:

    • runtime smoke tests
    • network/socket requirements
    • external tool availability
    • generated artifact consistency
  5. Clarify verification state language:

    • self-tested
    • reviewer-reproduced
    • reviewer-blocked
    • accepted by alternative evidence
  6. Strengthen review-freeze status tracking:

    • make it explicit when work is awaiting review
    • prevent summaries from implying review acceptance before review completes
  7. Add a review finding classification matrix:

    • mainline gap
    • true blocker
    • queued issue
    • environment limitation
  8. Reduce repeated full-plan restatement:

    • summaries and reviews should emphasize state deltas, unresolved evidence gaps, and next decisions
  9. Add a clean-baseline final verification checkpoint:

    • rerun all required commands after all feature work lands
    • include leakage/naming/audit checks where relevant

Expected Benefit

These changes should reduce repeated rounds, make stagnation easier to diagnose, clarify the difference between implementation progress and independent acceptance, and improve final completion confidence.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions