Skip to content

Fix benchmark manifest Git SHA#73

Merged
SPerekrestova merged 1 commit into
mainfrom
codex/benchmark-manifest-git-sha
May 22, 2026
Merged

Fix benchmark manifest Git SHA#73
SPerekrestova merged 1 commit into
mainfrom
codex/benchmark-manifest-git-sha

Conversation

@SPerekrestova

Copy link
Copy Markdown
Owner

Summary

  • Pass the workflow commit SHA into the Cloud Run benchmark job as BENCHMARK_GIT_COMMIT.
  • Make eval.run_benchmark use BENCHMARK_GIT_COMMIT/GITHUB_SHA when .git metadata is absent in the benchmark image.
  • Tighten artifact validation so manifest.git_commit cannot be the all-zero fallback.

Test Plan

  • rtk uv run pytest tests/eval tests/test_download_interaction_db.py tests/test_tier1_benchmark_workflow.py -q
  • rtk uv run pytest tests/ --ignore=tests/test_rxnorm_client.py -q
  • rtk git diff --check

@SPerekrestova SPerekrestova left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM


Generated by Claude Code

@SPerekrestova SPerekrestova merged commit 698a7ac into main May 22, 2026
5 checks passed
@SPerekrestova SPerekrestova deleted the codex/benchmark-manifest-git-sha branch May 22, 2026 22:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant