PR Triage

AI-powered pull request evaluation for open source maintainers.

PR Triage reads the diff, the linked issue, and your repo context — then posts a confidence score and reasoning directly on the PR. Skip the junk, review what matters.

Quick Start

1. Add your API key as a repo secret

Go to your repo > Settings > Secrets > New repository secret. Name it ANTHROPIC_API_KEY (or OPENAI_API_KEY if using OpenAI).

2. Create .github/workflows/pr-triage.yml

name: PR Triage
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  triage:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write
      contents: read
    steps:
      - uses: Elifterminal/pr-triage@v1
        with:
          api-key: ${{ secrets.ANTHROPIC_API_KEY }}
          provider: anthropic

3. Done. Next PR gets an automatic triage evaluation.

What It Checks

• Issue Relevance — Does the code actually address the linked issue?
• Scope Assessment — Is the change proportional to the problem?
• Pattern Alignment — Does the code follow your repo's conventions?
• Spam Detection — README-only edits, cosmetic changes, boilerplate dumps

Example Output

PASS (Score: 87/100)

PR directly addresses issue #342 by implementing retry logic in the HTTP client. Adds 3 test cases covering timeout, 5xx, and connection reset scenarios. Changes are scoped appropriately.

REJECT (Score: 14/100)

PR claims to fix issue #89 (authentication bug) but only modifies README.md with formatting changes. No code changes address the reported authentication failure.

Configuration

Input	Required	Default	Description
api-key	Yes	—	Your LLM API key (stored as GitHub Secret)
provider	No	anthropic	anthropic or openai
model	No	auto	Model override (e.g., gpt-4o, claude-sonnet-4-20250514)
pass-threshold	No	80	Minimum score for PASS
reject-threshold	No	39	Maximum score for REJECT
post-comment	No	true	Post results as PR comment
max-diff-size	No	12000	Max diff chars sent to LLM
ignore-paths	No	—	Comma-separated globs to exclude (e.g., *.md,docs/**)

Outputs

Use these in subsequent workflow steps:

- uses: Elifterminal/pr-triage@v1
  id: triage
  with:
    api-key: ${{ secrets.ANTHROPIC_API_KEY }}

- name: Auto-close spam
  if: steps.triage.outputs.recommendation == 'REJECT'
  run: gh pr close ${{ github.event.pull_request.number }} --comment "Closed by PR Triage"
  env:
    GH_TOKEN: ${{ github.token }}

Output	Description
score	Confidence score (0-100)
recommendation	PASS, REVIEW, or REJECT
summary	Short reasoning summary

Cost

PR Triage is free and open source. You provide your own LLM API key.

Average cost per evaluation: ~$0.002 (about 500 PRs per dollar).

How It Works

1. Triggered on pull_request events (opened, synchronize)
2. Fetches the PR diff, linked issues (from Fixes #123 patterns), and repo context
3. Sends a structured evaluation prompt to your chosen LLM
4. Parses the response and posts a formatted comment on the PR
5. Sets action outputs for use in downstream workflow steps

Draft PRs are automatically skipped.

Supported Providers

• Anthropic (Claude) — Default. Recommended model: claude-sonnet-4-20250514
• OpenAI — Recommended model: gpt-4o-mini

License

MIT