Problem
Users must run multiple CLI subcommands (summary, tail, jitter, convergence, health-check, outlier-impact, scorecard) individually to get a complete picture of their benchmark data quality and results.
Solution
Add a comprehensive subcommand that orchestrates multiple analysis modules in a single invocation and produces a unified report with aggregated findings.
Implementation
ComprehensiveRunner class in comprehensive.py
ComprehensiveConfig, AnalysisSection, ComprehensiveReport Pydantic models
- Default suite: summary, SLA check, tail analysis, jitter, convergence, health check, outlier impact, scorecard
- Configurable sections via
--sections / --skip flags
- Aggregated findings: warnings, recommendations, key metrics from all analyses
- Overall health summary with composite grade
- CLI
comprehensive subcommand with table + JSON output
- Programmatic
run_comprehensive() API
Acceptance Criteria
Problem
Users must run multiple CLI subcommands (summary, tail, jitter, convergence, health-check, outlier-impact, scorecard) individually to get a complete picture of their benchmark data quality and results.
Solution
Add a
comprehensivesubcommand that orchestrates multiple analysis modules in a single invocation and produces a unified report with aggregated findings.Implementation
ComprehensiveRunnerclass incomprehensive.pyComprehensiveConfig,AnalysisSection,ComprehensiveReportPydantic models--sections/--skipflagscomprehensivesubcommand with table + JSON outputrun_comprehensive()APIAcceptance Criteria
comprehensivesubcommand produces unified report