Skip to content

Split PluginBase: extract DiscoverablePlugin for CLI plugins #90

@renaudcepre

Description

@renaudcepre

Context

PluginBase mixes two responsibilities:

  1. Discovery/CLIname, description, add_cli_options(), activate()
  2. Lifecycle hookssetup(), on_test_pass(), on_session_end(), etc.

Internally-wired plugins (HistoryPlugin, EvalHistoryPlugin, EvalResultsWriter) only need lifecycle hooks but are forced to override activate() with return None to prevent auto-discovery. This is a code smell.

Proposal

PluginBase              → lifecycle hooks only (setup, on_*)
    ↑
DiscoverablePlugin      → adds name, description, add_cli_options, activate
  • Internal plugins inherit PluginBase
  • CLI-discoverable plugins inherit DiscoverablePlugin
  • No more defensive return None overrides

Additional motivation: circular import

core/session.py imports evals/history, evals/results_writer, and evals/wrapper to wire eval plugins manually. This creates a dependency inversion — core depends on evals, instead of evals plugging into core. This forces a __getattr__ lazy import hack in evals/__init__.py to break the cycle.

If eval plugins were auto-discovered (via DiscoverablePlugin.activate()), session wouldn't need to import them directly, eliminating the circular dependency entirely.

Files affected

  • protest/plugin.py — split the class
  • All plugins that currently override activate() -> None — simplify to PluginBase
  • Auto-discovery in session — only scan DiscoverablePlugin subclasses
  • protest/core/session.py — remove manual eval plugin wiring (_wire_eval_support)
  • protest/evals/__init__.py — remove __getattr__ hack once cycle is broken

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions