Skip to content

chore: switch from sonnet to haiku in maintainers processing [CM-1049] #3915

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mbani01 merged 3 commits into from
Mar 12, 2026
+16 −6
Merged

chore: switch from sonnet to haiku in maintainers processing [CM-1049] #3915

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5d2f0e2
feat: switch to Haiku model
mbani01 Mar 12, 2026
1dea030
chore: adapt query to Haiku model
mbani01 Mar 12, 2026
6c3dafd
fix: ai cost calculation
mbani01 Mar 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 11 additions & 5 deletions services/apps/git_integration/src/crowdgit/services/maintainer/bedrock.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ async def invoke_bedrock(
}
)

modelId = "us.anthropic.claude-sonnet-4-20250514-v1:0"
modelId = "us.anthropic.claude-haiku-4-5-20251001-v1:0"
accept = "application/json"
contentType = "application/json"

Expand Down Expand Up @@ -107,14 +107,20 @@ async def invoke_bedrock(
response_body = json.loads(body_bytes.decode("utf-8"))
raw_text = response_body["content"][0]["text"].replace('"""', "").strip()

# Expect pure JSON - no markdown handling
# Strip markdown code fences if present (Haiku sometimes ignores the system prompt)
if raw_text.startswith("```"):
raw_text = raw_text.split("\n", 1)[-1]
if raw_text.endswith("```"):
raw_text = raw_text.rsplit("```", 1)[0]
raw_text = raw_text.strip()
Comment on lines +111 to +115
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The markdown code-fence stripping can still leave a trailing closing fence in common outputs like json\n{...}\n\n (note the newline after the closing fence). In that case raw_text.endswith("```") is false before .strip(), so the closing fence remains and json.loads(raw_text) will fail. Consider stripping whitespace before checking for the closing fence (or using a small regex that removes leading .*\n and trailing with surrounding whitespace).

Suggested change
if raw_text.startswith("```"):
raw_text = raw_text.split("\n", 1)[-1]
if raw_text.endswith("```"):
raw_text = raw_text.rsplit("```", 1)[0]
raw_text = raw_text.strip()
stripped_text = raw_text.strip()
if stripped_text.startswith("```"):
# Remove the opening code fence (and optional language specifier)
stripped_text = stripped_text.split("\n", 1)[-1]
# Tolerate trailing whitespace/newlines after the closing fence
if stripped_text.rstrip().endswith("```"):
stripped_text = stripped_text.rstrip().rsplit("```", 1)[0]
raw_text = stripped_text.strip()

Copilot uses AI. Check for mistakes.

output = json.loads(raw_text)
Comment on lines +110 to 117
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new parsing behavior (stripping markdown code fences before json.loads) is a regression-prone edge case that would benefit from a unit test (e.g., outputs with ```json fences, trailing newlines, and plain JSON). There is an existing pytest suite under services/apps/git_integration/src/test, but no coverage around this Bedrock response parsing currently.

Copilot uses AI. Check for mistakes.

# Calculate cost
# Calculate cost (Claude Haiku 4.5 on AWS Bedrock: $1.00/$5.00 per 1M tokens)
input_tokens = response_body["usage"]["input_tokens"]
output_tokens = response_body["usage"]["output_tokens"]
input_cost = (input_tokens / 1000) * 0.003
output_cost = (output_tokens / 1000) * 0.015
input_cost = (input_tokens / 1_000_000) * 1.00
output_cost = (output_tokens / 1_000_000) * 5.00
total_cost = input_cost + output_cost

# Validate output with the provided model if it exists
Expand Down
6 changes: 5 additions & 1 deletion services/apps/git_integration/src/crowdgit/services/maintainer/maintainer_service.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -218,12 +218,16 @@ def get_extraction_prompt(self, filename: str, content_to_analyze: str) -> str:
- The person's role, with a maximum of two words (e.g., "Lead Reviewer", "Core Maintainer").
- The role must be about project governance, not a generic job title like "Software Engineer".
- Do not include filler words like "repository", "project", or "active".
- **If the content does not assign an explicit individual role to each person** (e.g. a flat list with no per-person labels), set the title to the capitalized form of `normalized_title` (i.e. "Maintainer" or "Contributor"). Every person in the same response MUST receive the same derived title.
4. `normalized_title`:
- Must be exactly "maintainer" or "contributor". If the role is ambiguous, use the `<filename>` as the primary hint. For example, a file named `MAINTAINERS` or `CODEOWNERS` implies "maintainer", while `CONTRIBUTORS` implies "contributor".
- Must be exactly "maintainer" or "contributor". If the role is ambiguous, use the `{filename}` as the primary hint:
- Filenames containing `MAINTAINERS`, `CODEOWNERS`, `OWNERS`, or `REVIEWERS` → "maintainer"
- All other filenames (AUTHORS, CONTRIBUTORS, CREDITS, COMMITTERS, etc.) → "contributor"
5. `email`:
- Extract the person's email address from the content. Look for patterns like `FullName <email@domain>`, `email@domain`, or email addresses in various formats.
- The email must be a valid email address format (containing @ and a domain).
- If no valid email can be found for the individual, use the string "unknown".
- **You MUST include every person found in the content regardless of whether their email is known. Never omit a person because their email is missing.**

---
Filename: {filename}
Expand Down
Loading