Skip to content

Audit/2026 04 29#376

Draft
jasonmorais wants to merge 2 commits into
jason.weekly-audit-pipelinefrom
audit/2026-04-29
Draft

Audit/2026 04 29#376
jasonmorais wants to merge 2 commits into
jason.weekly-audit-pipelinefrom
audit/2026-04-29

Conversation

@jasonmorais
Copy link
Copy Markdown
Contributor

@jasonmorais jasonmorais commented Apr 29, 2026

Summary by Sourcery

Introduce a new strictly enforced audit workflow with orchestrator, agents, and hooks, and record the initial 2026-04-29 repository audit.

New Features:

  • Add an Audit-Orchestrator agent and specialized audit subagents for scoping, dependency security, practice compliance, code quality, test quality, performance, documentation DX, synthesis, and publishing.
  • Introduce audit-specific hooks and shared logic to manage audit workflow phases, enforce agent sequencing, and validate analyzer report outputs.
  • Create the first generated audit report artifact under documents/audits/2026-04-29/audit.md, establishing the baseline for future audits.

Enhancements:

  • Refine the existing implementation orchestrator agent configuration to reference implementation-specific hooks and centralize model assignment usage.

Inaugural automated audit covering 50 commits (2025-12-01 → 2026-04-29)
across all packages in the ShareThrift monorepo.

Findings summary:
  Critical : 1  (missing auth on appeal-request resolvers — live in prod)
  High     : 6  (unbounded DB queries, no-op event handlers, auth control-flow)
  Medium   : 12 (N+1 resolvers, missing indexes, PII console.log, duplication)
  Low      : 6  (type safety, schema workarounds, unindexed regex search)

No auto-fixes were applied by analyzers this run.
DependencySecurity report was unavailable; re-run required.

See documents/audits/2026-04-29/audit.md for full findings and action plan.
- Full audit report with 1 Critical, 7 High, 9 Medium, 5 Low, 4 Info findings
- DependencySecurity: removed 9 obsolete Snyk waivers from .snyk (auto-fix)
- CodeQuality: 4 high (permissions layer dead, auth gaps, visa bypass, unimplemented stubs)
- Performance: 4 high (N+1 queries, unbounded collection scans, in-memory pagination)
- Establishes baseline for all future week-over-week trend comparisons
@sourcery-ai
Copy link
Copy Markdown
Contributor

sourcery-ai Bot commented Apr 29, 2026

Reviewer's Guide

Introduces a new, hook-enforced weekly audit workflow (Scoper → parallel analyzers → Synthesizer → Publisher) with dedicated agents and Node hooks, wires audit-specific shared state and report handling, adds the first generated audit report artifact, and refactors the implementation orchestrator config to use implementation-specific hooks and a central model-assignment table.

Class diagram for audit shared state and hooks

classDiagram
  class AuditState {
    +string workflow
    +string phase
    +boolean active
    +boolean scoperCompleted
    +string[] analyzersCompleted
    +boolean synthesizerCompleted
    +boolean publisherCompleted
    +number stopBlockCount
    +string[] processedEvents
  }

  class SharedModule {
    +string[] AUDIT_ANALYZERS
    +string[] VALID_AGENTS
    +number MAX_STOP_BLOCKS
    +map~string, string[]~ PHASE_ALLOWED_AGENTS
    +map~string, string~ PHASE_GUIDANCE
    +stateDir() string
    +stateFilePath(sessionId string) string
    +loadState(sessionId string) AuditState~nullable~
    +saveState(sessionId string, state AuditState) void
    +createInitialState() AuditState
    +eventKey(input any) string
    +isDuplicate(state AuditState, input any) boolean
    +extractAgentName(toolInput any) string~nullable~
    +isSubagentTool(toolName string) boolean
    +workflowSummary(state AuditState) string
    +readHookInput() any
    +runHook(handler function) void
    +reportsDir(sessionId string) string
    +analyzerReportFilename(analyzerId string) string
    +listReports(sessionId string) ReportEntry[]
    +analyzerReportStatus(sessionId string) AnalyzerStatus
    +summarizeAnalyzerStatus(status AnalyzerStatus) string
  }

  class ReportEntry {
    +string filePath
    +string filename
    +any data
  }

  class AnalyzerStatus {
    +number expected
    +string[] present
    +string[] missing
    +string[] failed
    +map~string, string~ statusById
  }

  class SessionStartHookModule {
    +handleSessionStart(input any) any
  }

  class PreToolUseHookModule {
    +handlePreToolUse(input any) any
    -denyOutOfPhase(state AuditState, agentName string) any~nullable~
    -denyIfReportGate(state AuditState, agentName string, sessionId string) any~nullable~
  }

  class SubagentStartHookModule {
    +handleSubagentStart(input any) any
  }

  class SubagentStopHookModule {
    +handleSubagentStop(input any) any
    -handleScoperStop(state AuditState) string
    -handleAnalyzerStop(state AuditState, agentType string, sessionId string) string
    -handleSynthesizerStop(state AuditState) string
    -handlePublisherStop(state AuditState) string
  }

  class StopHookModule {
    +handleStop(input any) any
  }

  class UserPromptSubmitHookModule {
    +handleUserPromptSubmit(input any) any
  }

  SharedModule <.. SessionStartHookModule : uses
  SharedModule <.. PreToolUseHookModule : uses
  SharedModule <.. SubagentStartHookModule : uses
  SharedModule <.. SubagentStopHookModule : uses
  SharedModule <.. StopHookModule : uses
  SharedModule <.. UserPromptSubmitHookModule : uses

  AuditState <.. SharedModule : created and persisted
  AnalyzerStatus <.. SharedModule : returned by
  ReportEntry <.. SharedModule : returned by
Loading

Flow diagram for the new audit workflow phases and agents

flowchart TD
  subgraph Phases
    P1["Phase init\n(Scoper only)"]
    P2["Phase scoping\n(Scoper running)"]
    P3["Phase scoping_complete\n(ready for analyzers)"]
    P4["Phase analyzing\n(analyzers and Synthesizer gate)"]
    P5["Phase synthesizing\n(Synthesizer running)"]
    P6["Phase synthesis_complete\n(ready for Publisher)"]
    P7["Phase publishing\n(Publisher running)"]
    P8["Phase done\n(workflow complete)"]
  end

  subgraph Agents
    A0["Audit_Orchestrator\n(agent: Audit-Orchestrator)"]
    A1["Scoper"]
    A2["DependencySecurity"]
    A3["PracticeCompliance"]
    A4["CodeQuality"]
    A5["TestQuality"]
    A6["Performance"]
    A7["DocumentationDx"]
    A8["Synthesizer"]
    A9["Publisher"]
  end

  subgraph Hooks_and_State
    H1["SessionStart hook\n(audit/session-start.mjs)"]
    H2["PreToolUse hook\n(audit/pre-tool-use.mjs)"]
    H3["SubagentStart hook\n(audit/subagent-start.mjs)"]
    H4["SubagentStop hook\n(audit/subagent-stop.mjs)"]
    H5["Stop hook\n(audit/stop.mjs)"]
    S1["Shared_state\n(audit/shared.mjs)"]
    R1["Reports_dir\n(reportsDir(sessionId))"]
  end

  A0 --> H1
  H1 --> S1
  H1 --> R1

  P1 -->|spawn Scoper| A1
  A0 -->|runSubagent Scoper| H2
  H2 --> H3
  H3 --> P2
  A1 -->|writes scope.json| R1
  A1 --> H4
  H4 --> P3

  P3 -->|spawn all analyzers in one response| A2
  P3 --> A3
  P3 --> A4
  P3 --> A5
  P3 --> A6
  P3 --> A7

  A0 -->|runSubagent analyzers| H2
  H2 --> H3
  H3 --> P4

  A2 -->|DependencySecurity.json| R1
  A3 -->|PracticeCompliance.json| R1
  A4 -->|CodeQuality.json| R1
  A5 -->|TestQuality.json| R1
  A6 -->|Performance.json| R1
  A7 -->|DocumentationDx.json| R1

  A2 --> H4
  A3 --> H4
  A4 --> H4
  A5 --> H4
  A6 --> H4
  A7 --> H4

  H4 -->|check analyzerReportStatus| S1
  H4 -->|all reports present| A0

  P4 -->|spawn Synthesizer| A8
  A0 -->|runSubagent Synthesizer| H2
  H2 -->|deny if missing reports| A0
  H2 -->|allow if all present| H3
  H3 --> P5

  A8 -->|read all reports + write audit.md| R1
  A8 --> H4
  H4 --> P6

  P6 -->|spawn Publisher| A9
  A0 -->|runSubagent Publisher| H2
  H2 --> H3
  H3 --> P7

  A9 -->|branch + commit + push + PR| R1
  A9 --> H4
  H4 --> P8

  A0 -->|Stop requested| H5
  H5 -->|block until Publisher done or max blocks| A0

  S1 -->|PHASE_ALLOWED_AGENTS, PHASE_GUIDANCE, analyzerReportStatus| H2
  S1 --> H3
  S1 --> H4
  S1 --> H5
Loading

File-Level Changes

Change Details Files
Refactor implementation orchestrator agent config to use implementation-specific hooks and centralized model assignment.
  • Rename orchestrator agent file and update its name to Implementation-Orchestrator.
  • Point all workflow hooks (SessionStart, UserPromptSubmit, PreToolUse, SubagentStart, SubagentStop, Stop) to implementation-specific hook paths.
  • Introduce a MODEL ASSIGNMENT table near the top and switch all step/prompt instructions to reference it instead of hard-coded model strings.
.github/agents/orchestrator.agent.md
.github/agents/implementation-orchestrator.agent.md
Introduce reusable shared audit workflow state, report directory helpers, and analyzer report validation utilities for hooks.
  • Define audit phases, valid agents, allowed agents per phase, and phase guidance strings.
  • Implement state persistence in tmpdir keyed by sessionId, including duplicate-event suppression and simple event history.
  • Add helpers for reports directory management, listing and summarizing analyzer reports, and summarizing analyzer status for guidance messages.
  • Provide generic hook runner and stdin JSON parsing utilities for other audit hooks to reuse.
.github/hooks/audit/shared.mjs
Add an audit-specific orchestrator agent definition that coordinates Scoper, analyzers, Synthesizer, and Publisher using the audit hooks.
  • Define the Audit-Orchestrator agent with its own model assignments and the full 5-step audit workflow description.
  • Document strict tool usage (runSubagent only), required sequencing, and parallel spawning of analyzers.
  • Specify required prompt shapes for Scoper, each analyzer, Synthesizer, and Publisher, including REPORTS_DIR/SCOPE_PATH conventions and safe autofix policy.
.github/agents/audit-orchestrator.agent.md
Implement audit lifecycle hooks to enforce phase ordering, analyzer report gates, and safe stop behavior.
  • On SessionStart, initialize audit state, compute a per-session reports directory, and emit detailed workflow and report protocol guidance.
  • In SubagentStart, transition phases based on which agent starts (Scoper, analyzers, Synthesizer, Publisher).
  • In SubagentStop, update completion flags, enforce that all analyzer reports exist before allowing Synthesizer, and emit analyzer status summaries.
  • In PreToolUse, deny out-of-phase or unknown agent spawns and block Synthesizer if analyzer reports are missing, while allowing valid subagent spawns with contextual messages.
  • In Stop, block premature session termination until Publisher completes or a max block count is reached, returning current progress and next required action.
  • Provide a UserPromptSubmit hook that injects phase-specific guidance into the system message for each user prompt.
.github/hooks/audit/session-start.mjs
.github/hooks/audit/subagent-start.mjs
.github/hooks/audit/subagent-stop.mjs
.github/hooks/audit/pre-tool-use.mjs
.github/hooks/audit/stop.mjs
.github/hooks/audit/user-prompt-submit.mjs
Add specialized audit agents for Scoping, Dependency Security, Practice Compliance, Code Quality, Test Quality, Performance, Documentation DX, Synthesizing, and Publishing.
  • Define each audit agent’s purpose, allowed tools, and strict input expectations centered around REPORTS_DIR and scope.json.
  • Specify per-agent responsibilities, including safe-autofix boundaries (DependencySecurity, PracticeCompliance, DocumentationDx) vs report-only behavior for others.
  • Standardize JSON report schemas for each agent, including fields like agentId, status, summary, findings, appliedFixes, and statistics.
  • Constrain agents from modifying code where inappropriate, and require that reports be written as their final action.
.github/agents/audit-scoper.agent.md
.github/agents/audit-dependency-security.agent.md
.github/agents/audit-practice-compliance.agent.md
.github/agents/audit-code-quality.agent.md
.github/agents/audit-test-quality.agent.md
.github/agents/audit-performance.agent.md
.github/agents/audit-documentation-dx.agent.md
.github/agents/audit-synthesizer.agent.md
.github/agents/audit-publisher.agent.md
Check in the generated weekly audit artifact for 2026-04-29 documenting findings and recommendations across domains.
  • Create documents/audits/2026-04-29/audit.md containing the first baseline audit, with executive summary, per-severity findings, per-agent summaries, and action plan.
  • Document auto-applied fixes (e.g., removal of obsolete .snyk waivers) and cross-reference specific files and lines for all key findings.
documents/audits/2026-04-29/audit.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@jasonmorais jasonmorais changed the base branch from main to jason.weekly-audit-pipeline April 29, 2026 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant