Skip to content

feat: Benchmark Quality Gate (M117)#258

Merged
hlin99 merged 1 commit into
mainfrom
feat/m117-quality-gate
Apr 6, 2026
Merged

feat: Benchmark Quality Gate (M117)#258
hlin99 merged 1 commit into
mainfrom
feat/m117-quality-gate

Conversation

@hlin99

@hlin99 hlin99 commented Apr 6, 2026

Copy link
Copy Markdown
Member

Summary

Composite pass/fail quality gate for benchmark data, designed for CI/CD pipeline integration.

Changes

  • QualityGate class in quality_gate.py with YAML-configurable GateConfig
  • GateCheck, GateConfig, GateResult, GateVerdict Pydantic models
  • 5 built-in checks: min_requests, quality_score, outlier_ratio, convergence, load_profile
  • Non-zero exit code on FAIL (CI/CD pipeline friendly)
  • CLI quality-gate subcommand with --config YAML support, table + JSON output
  • Programmatic evaluate_quality_gate() and load_gate_config() APIs
  • 23 new tests
  • Updated ROADMAP.md (M116 ✅, M117 🔄)

Closes #257

- QualityGate class in quality_gate.py with YAML-configurable GateConfig
- GateCheck, GateConfig, GateResult, GateVerdict Pydantic models
- 5 composite checks: min_requests, quality_score, outlier_ratio, convergence, load_profile
- Non-zero exit code on FAIL (CI/CD pipeline friendly)
- CLI quality-gate subcommand with table + JSON output
- Programmatic evaluate_quality_gate() API
- 23 new tests

Closes #257

@hlin99-Review-Bot hlin99-Review-Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved by hlin99-Review-Bot

Idea Value: High — composite quality gate is essential for CI/CD pipeline integration. Combining validation, convergence, outlier, and load profile checks into a single pass/fail verdict is the right approach.

Code Quality:

  • Clean Pydantic models (GateConfig, GateCheck, GateResult, GateVerdict)
  • YAML-configurable thresholds with sensible defaults
  • Good separation: core logic in quality_gate.py, CLI in _quality_gate.py
  • 23 tests covering models, config loading, evaluator, programmatic API, CLI, and imports
  • Non-zero exit on FAIL is CI-friendly
  • docs/iterations/current.md updated

All 5 CI checks pass. LGTM.

@hlin99-Review-BotX hlin99-Review-BotX left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved by hlin99-Review-BotX

Idea Value: High — composite quality gate is a natural CI/CD integration point. Combining validation, convergence, outlier, and load profile checks into a single pass/fail verdict with YAML-configurable thresholds is well-designed.

Code Quality:

  • Clean Pydantic models (GateConfig, GateCheck, GateResult, GateVerdict)
  • Good separation: core quality_gate.py + CLI _quality_gate.py
  • YAML config with sensible defaults, graceful warn/fail thresholds
  • 23 tests covering models, config, evaluator, programmatic API, CLI, and imports
  • Non-zero exit on FAIL is CI-friendly
  • docs/iterations/current.md updated

All 5 CI checks pass. Second +1 — ready for merge.

@hlin99-Review-BotX hlin99-Review-BotX left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved by hlin99-Review-BotX

Idea Value: Strong — composite quality gate for CI/CD is a natural fit after the benchmark pipeline milestones. YAML-configurable thresholds and non-zero exit on FAIL make it immediately usable in pipelines.

Code Quality:

  • Clean Pydantic models (GateConfig/GateCheck/GateResult/GateVerdict)
  • Good separation: core logic in quality_gate.py, CLI in _quality_gate.py
  • 5 built-in checks with graduated PASS/WARN/FAIL verdicts
  • YAML config loader with sensible defaults
  • CLI properly registered with table + JSON output
  • docs/iterations/current.md updated
  • ROADMAP.md updated (M116 ✅, M117 🔄)
  • CI all green (lint + tests 3.10/3.11/3.12)

No issues found. LGTM.

@hlin99-Review-BotX hlin99-Review-BotX left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved (hlin99-Review-BotX)

Idea Value: Strong — composite quality gate for CI/CD pipeline integration is a natural next step after convergence/outlier/load-profile analyzers. Clean design combining existing analyzers into a unified pass/fail verdict.

Code Quality:

  • Well-structured Pydantic models (GateConfig, GateCheck, GateResult, GateVerdict)
  • 5 checks with sensible PASS/WARN/FAIL thresholds and configurable via YAML
  • Clean CLI with table + JSON output, non-zero exit on FAIL (CI-friendly)
  • Programmatic evaluate_quality_gate() API with both config-file and kwargs paths
  • 23 tests covering models, config loading, evaluator, API, CLI, and public imports
  • docs/iterations/current.md and ROADMAP.md properly updated

Minor nit: ROADMAP says '~22 new tests' but actual count is 23. Not blocking.

LGTM 🚀

@hlin99 hlin99 merged commit b545a74 into main Apr 6, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: Benchmark Quality Gate (M117)

3 participants