feat: evals#89

Open

renaudcepre wants to merge 21 commits intomainfrom

feat/evals-native

Commits on Mar 29, 2026

chore: entity exports, pyproject config
renaudcepre
committed

Commits on Mar 30, 2026

Commits on Mar 31, 2026

feat(evals): Judge protocol — LLM-as-judge via inversion of dependency
renaudcepre
committed
fix(reporters): show in/out token split in eval usage summary
renaudcepre
committed
fix(history): exclude error-only runs from stats, propagate is_error flag
renaudcepre
committed
refactor: remove getattr abuse — proper typing and Protocol contracts
renaudcepre
committed
fix(hashing): fail-hard canonicalization, evaluator_identity() protocol
renaudcepre
committed
refactor: replace kind string literals with SuiteKind StrEnum
renaudcepre
committed
refactor: move lazy imports to top-level, remove PLC0415 per-file ignores
renaudcepre
committed
fix: resolve all 32 mypy errors, type EvalContext generics properly
renaudcepre
committed
refactor: remove dead duck-typed evaluator markers, add typed examples
renaudcepre
committed
ci: update workflow to install dependencies and fix mypy invocation
renaudcepre
committed
refactor: remove redundant type ignores, update dependency management
renaudcepre
committed