Skip to content

Cite canonical source in Guide anti-gaming bullets#294

Open
PunchTheDev wants to merge 1 commit into
mainfrom
punch/guide-anti-gaming-tooltips
Open

Cite canonical source in Guide anti-gaming bullets#294
PunchTheDev wants to merge 1 commit into
mainfrom
punch/guide-anti-gaming-tooltips

Conversation

@PunchTheDev
Copy link
Copy Markdown
Owner

Summary

Each of the 7 anti-gaming bullets in QuickstartGuide.tsx gets cursor-help + a tooltip naming the specific attack vector it prevents and citing the canonical source file in forge. Bullet 2 is fixed from a false claim.

Motivation

The bullets were terse one-liners with no link to the canonical threat-model doc. A first-timer reading "Determinism check" or "Similarity check" learned nothing about what attack the check stops or where in the harness it lives.

Separately, bullet 2 claimed "Full coverage — final scoring runs all 45 problems, no sampling variance". That contradicts scripts/run_eval_pool.py L37–49 and .github/workflows/eval.yml L150, which both run only 3 specs per PR (one random spec per round). The claim is replaced with the accurate "Three-round sampling — each PR scores 3 of 45 problems, one random spec per round" plus a tooltip explaining the CI-time tradeoff.

Changes

  • src/components/QuickstartGuide.tsx L838–878: 7-item bullet list converted from string array to { label, tip } objects; each label rendered with cursor-help border-b border-dotted so the hover affordance is visible. Tooltips cite scripts/run_eval_pool.py, scripts/check_source_similarity.py, forge/sdk/llm.py (FORGE_MODEL_WHITELIST), benchmark/sandbox.py, specs/generator.py, benchmark/fea.py.
  • Bullet 2 reframed from false "Full coverage — all 45 problems" to accurate "Three-round sampling — 3 of 45, one random per round".
  • BACKLOG.md L115: Anti-gaming guarantees flipped ○ ○ ○● ● ● with rationale.

Verification

Puppeteer at 1440×900 on /guide#anti-gaming:

  • 7 bullets present
  • Each has cursor-help + border-dotted + non-empty title attr (lengths 127, 223, 127, 157, 170, 159, 192 chars)
  • Bullet 2 text matches new Three-round sampling framing
  • 0 new console errors

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant