feat(skills): Add triage and digest workflow skills; enrich fix-issues#179
Draft
NicoHinderling wants to merge 27 commits into
Draft
feat(skills): Add triage and digest workflow skills; enrich fix-issues#179NicoHinderling wants to merge 27 commits into
NicoHinderling wants to merge 27 commits into
Conversation
…sues Add two new workflow skills designed to run unattended on a schedule: - sentry-groom-issues: weekly two-pass backlog hygiene (close stale, reopen regressions), MCP-only with caps, dry-run, and a fixed parseable digest. - sentry-issue-digest: read-only daily/weekly summary of what changed (top new issues, regressions, movers, optional release health). Enrich sentry-fix-issues with an autonomous (cron) mode: fixability scoring and single-candidate selection, root-cause and scope discipline, draft-PR/branch safety, a parseable summary, and emission of a shared sentry-agent-activity/v1 marker for auditability. Add allowed-tools to its frontmatter. Register both new skills in the sentry-workflow router and regenerate SKILL_TREE.md.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
![sentry-warden[bot]](https://avatars.githubusercontent.com/in/2755838?s=60&v=4){kind=small}
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
… self-correct When closing stale issues, set ignoreMode: untilEscalating instead of a plain ignore. A stale-closed issue then auto-resurfaces if it escalates again, making a wrong close self-correcting and safer for unattended runs.
…condary Frame sentry-fix-issues around interactive, point-at-a-bug use and treat the scheduled/autonomous path as a secondary capability, so it occupies a distinct niche from the queue-hygiene skills (triage, groom) rather than competing for the daily-cron slot.
Generalize the team's triage-frontend-issues pattern into a platform-agnostic, cron-capable skill. Classifies the live new-issue queue as archive / skip / needs-human, archives only with ignoreMode: untilEscalating and a stated reason, and never touches code. Includes a generic noise taxonomy plus a routed JS profile (references/triage-js-profile.md). Skip-assigned and fresh-queue scoping keep it from colliding with sentry-fix-issues (which assigns what it works on) and sentry-groom-issues (aged backlog). De-collide groom-issues' triggers (drop 'triage'/'clean up issues') and register triage in the sentry-workflow router.
NicoHinderling
changed the title
Drop the docs/superpowers/specs/ design documents from the PR diff; they were working design notes, not part of the shipped skill library.
Live testing against a real org showed the Sentry MCP's query layer can rewrite a bare relative duration (e.g. firstSeen:-14d) into an invalid >=-14d, failing the search with HTTP 400. Switch issue-digest, triage-issues, and groom Pass 2 to absolute ISO cutoffs with comparators (firstSeen:>ISO / lastSeen:>ISO), which the API accepts reliably. groom Pass 1 already used this form.
Live testing showed the Sentry MCP can be connected read-only (no update_issue mutation tool), which would make the writing skills fail or no-op confusingly for external users. Add a Pass 0 write-capability check to groom-issues and triage-issues that falls back to report-only with a clear banner when no write tool is present, and document the prerequisite. fix-issues skips the Sentry assign + activity marker on read-only connections and notes it; its draft PR (via gh) is unaffected.
…tool Audit pass: drop generic debugging filler (fix-issues Common Patterns), a Quick Reference cadence line that duplicated config (issue-digest), and a duplicated do-not-assign rule (groom-issues). Switch groom Pass 2 to the dedicated get_issue_activity tool for the resolution timestamp.
…nsistency - Never prompt unless a human is unambiguously present, so scheduled runs cannot hang on AskUserQuestion (all skills). - fix-issues: move branch preflight before code edits; skip orphaned branches / open PRs to the next candidate in autonomous mode instead of halting; use search_issues with a separate sort param (drop list_issues and sort:freq in query); keep regression tests within the scoped change. - groom: skip human-assigned issues in Pass 1; scope queries by PROJECT_SLUG; gate writes on READ_ONLY as well as DRY_RUN; report run mode in the digest; make the regression look-back window configurable. - triage: define the interactive numbered plan-table schema; require users==0 for <unknown> zero-impact archives; make single-event-fluke signals window-relative; gate writes on READ_ONLY; report run mode. - issue-digest: scope gather queries by project; window-filter the regressions section; relabel movers as most-active-in-window to match what it ranks.
Per review, skills must stand alone: remove every cross-skill reference from the four workflow skills (no more 'use sentry-X instead', interplay notes, or pipeline ordering) so each loads in isolation without dangling pointers that drift. Cross-skill routing/disambiguation lives in the sentry-workflow router, which is its purpose. Also delete the 'Invoke This Skill When' sections, whose triggers are already carried by the description frontmatter used for routing.
De-cron'ing the skill over-narrowed its description to 'a specific issue', which clashed with the plural skill name and title. Restore the general 'find and fix issues' framing while keeping the human-in-the-loop positioning; it fixes one issue at a time (its own branch/PR) as a workflow detail, not a scope limit.
Reshape the digest output to be short and glanceable: lead with a one-line TL;DR, print only sections that have content, and collapse to a single line when nothing changed. Lower the default per-section cap. Add an optional best-effort repo-correlation pass that, when a git checkout is present, ties new and regressed issues to the commits that likely introduced them. The pass is skipped silently when no repo is available, keeping the skill cron-safe. This is the differentiator from the passive, repo-blind Weekly Reports email.
Live-testing against a production org exposed three gaps in the repo-correlation pass: - Minified production stack frames defeat the top-frame path lookup. Add Seer-root-cause and error-string grep as primary signals, and use the release tag as a frame-independent commit suspect when it resolves to a SHA. - A Sentry regression is often a re-occurrence of a resolved issue, not a recent commit. Distinguish the two so the digest no longer risks blaming an old, unchanged line; emit a re-occurrence note instead of a false 'likely from' attribution. - Prefer Seer's root cause when present over git archaeology.
Triage: - Never substitute a permanent ignore for archive-until-escalating; if the mutation tool can't express that mode, run read-only and archive nothing. Verify each archive landed as until-escalating, not permanent. - Default to interactive (confirmation gate); autonomous only with --auto, so a human run can't silently auto-archive. - Distinguish read-only from dry-run in row tags and the digest header. - Populate needs_human[]/skipped[] accumulators, not just archived[]. Digest: - Rename 'New regressions' to 'Active regressions' and document that the lastSeen filter surfaces currently-active regressed issues, not newly regressed ones. - Only abort an org-wide run for a missing project when PROJECT_SLUG is actually set; label org-wide output 'Project: all'. Fix-issues: - Phase 8 report titled 'Fix proposed' (draft PR), not 'Fixed', since the Sentry issue is left unresolved pending merge.
The release tag is untrusted Sentry input; validate it against a strict SHA pattern before passing to git cat-file, closing a command-injection vector introduced in the correlation pass.
- Document that gh is used in the Phase 1 branch preflight, not only Phase 7, and never infer an orphaned branch from a missing/failed gh lookup (don't recommend deletion on a failed check). - Make Sentry assignment best-effort: only assign with a real user ID from an available tool, never a guessed one; skip and note otherwise.
The per-issue 'continue on failure' rule conflicted with Pass 3's instruction to stop after a permanent-ignore verification failure. Carve out that one case: continue on ordinary failures, but stop archiving when the until-escalating mode misbehaves.
Consolidate two within-skill restatements: the validate-event-data- against-source rule (now stated once in Phase 4, a pointer in Security Constraints), and the no-raw-data/secrets guidance (stated once in Security Constraints, pointers in Phases 2 and 5).
Cut a stray scope reminder that duplicated the intro, and drop the Mode Selection parenthetical that re-explained Pass 0/Pass 3 instead of just pointing to them.
Trim the 'vs static email' paragraph to one operational line (the correlation capability is already in the description and the Repo correlation section), and reduce the Hard Rule to its invariant since the formatting detail is fully specified in Final - Print Digest.
- fix-issues: push the branch before gh pr create (PR creation needs a pushed branch). - triage: archived[] is filled only by the archive loop; non-archive decisions populate needs_human[]/skipped[] — prevents double-listing. - triage: close the <unknown>/0-user decision-matrix gap (51-999 events). - digest: align description and intro with the 'Active regressions' rename (no more 'new/fresh regressions' wording).
Phases 5-6 ended with only an implicit 'after the user approves' precondition; add an explicit checkpoint so the agent never commits, pushes, or opens a draft PR without confirmation, and can stop at 'Fix ready (no PR)'.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 3 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit eb9c345. Configure here.
| | Our application code | — | any | `skip` | | ||
| | `<unknown>`, `users == 0` | n/a | < 1000 events | `archive` (zero-impact) | | ||
| | `<unknown>`, `users > 0` | n/a | any | `needs-human` (real users affected) | | ||
| | `<unknown>`, `users == 0` | n/a | high (≥ 1000 events) | `needs-human` (volume warrants a look) | |
There was a problem hiding this comment.
Zero users misread as zero-impact
High Severity
The decision matrix auto-archives any issue whose top frame is unknown when users == 0 and event count is under 1000, without a clean noise-category match. In Sentry, zero unique users often means missing user context, not no real impact, so first-party bugs with unsymbolicated stacks can be archived unattended under --auto.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit eb9c345. Configure here.
| - The issue has a **human assignee** — someone already owns it; leave it for them. | ||
| - The issue is assigned to a team other than yours and looks team-specific — let that team triage it. | ||
|
|
||
| ## Pass 2 — Classify each candidate |
There was a problem hiding this comment.
Pass two omits digest accumulators
Medium Severity
Pass 2 classifies issues but never says to fill needs_human[] or skipped[]. Those accumulators are only described later under Pass 3, after the interactive plan and cancel path, so a literal follow can print an empty Needs human/Skipped digest even though the plan table already listed decisions.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit eb9c345. Configure here.
| ## Hard Rules | ||
|
|
||
| - **Never write.** Do not call `update_issue` or any mutating tool. This skill only reads. | ||
| - **Cap every section at `TOP_N`.** If a section is truncated, say so in the digest so a quiet section isn't mistaken for a complete one. |
There was a problem hiding this comment.
Release health ignores TOP_N cap
Low Severity
Hard Rules require every digest section to respect TOP_N, but optional Release health only says to cap results at “a few” releases. Scheduled runs can emit an unbounded Release health block, breaking the stated scannable digest size and skipping the “(top TOP_N)” truncation cue used elsewhere.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit eb9c345. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
sentry-issue-digestsentry-triage-issuesarchive/skip/needs-human; archive noiseuntilEscalatingwith a stated reason. Generic taxonomy + routed JS profile. Autonomous (cron) or interactive.Why these exist (vs. native Sentry)
Native surfaces (Weekly Reports email, auto-resolve, Seer) are passive, in-product, or blunt — none reason and act in your repo, on your schedule.
The digest's headline differentiator over the Weekly Reports email: repo correlation — it locates each issue's source (Seer root cause → stack frame → error-string grep) and names the likely commit via
git log+ the release tag. Skipped silently with no repo, so it stays cron-safe.Live validation (real production org)
Dry-run/read-tested against a live org with a checked-out monorepo, which hardened the skills against real-data edge cases:
root_level_exceptionhad a fully minified stack — error-string grep + the release SHA found the source instead. Correlation now uses those signals.inpage.bundle.js) that Sentry itself labeled "first-party," and a third-partymixpanel-browsererror — each with a sourced reason; dry-run wrote nothing.Design principles
sentry-workflowrouter.--dry-run, read-only-MCP fallback to report-only.untilEscalatingarchives self-correct, making unattended archiving safe.Verification
scripts/build-skill-tree.sh --check→ 35 skills, 3 routers, 0 errors. Router,AGENTS.md,SKILL_TREE.mdupdated.