Skip to content

Filter system messages#177

Merged
Ankur Goyal (ankrgyl) merged 3 commits intomainfrom
filter-system-messages
Mar 1, 2026
Merged

Filter system messages#177
Ankur Goyal (ankrgyl) merged 3 commits intomainfrom
filter-system-messages

Conversation

@ankrgyl
Copy link
Contributor

preprocessors now return system messages, so we need to filter them out explicitly.

@github-actions
Copy link

github-actions bot commented Feb 28, 2026

Braintrust eval report

Autoevals (filter-system-messages-1772320745)

Score Average Improvements Regressions
NumericDiff 75% (+1pp) 7 🟢 2 🔴
Time_to_first_token 1.97tok (+0.6tok) 3 🟢 116 🔴
Llm_calls 1.55 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 279.25tok (+0tok) - -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Completion_tokens 18.49tok (+0.03tok) 15 🟢 19 🔴
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 297.73tok (+0.03tok) 15 🟢 19 🔴
Estimated_cost 0$ (+0$) - 119 🔴
Duration 2.88s (-0.76s) 109 🟢 110 🔴
Llm_duration 3.75s (+1.04s) 1 🟢 118 🔴

1 similar comment
@github-actions
Copy link

Braintrust eval report

Autoevals (filter-system-messages-1772320745)

Score Average Improvements Regressions
NumericDiff 75% (+1pp) 7 🟢 2 🔴
Time_to_first_token 1.97tok (+0.6tok) 3 🟢 116 🔴
Llm_calls 1.55 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 279.25tok (+0tok) - -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Completion_tokens 18.49tok (+0.03tok) 15 🟢 19 🔴
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 297.73tok (+0.03tok) 15 🟢 19 🔴
Estimated_cost 0$ (+0$) - 119 🔴
Duration 2.88s (-0.76s) 109 🟢 110 🔴
Llm_duration 3.75s (+1.04s) 1 🟢 118 🔴

@ankrgyl Ankur Goyal (ankrgyl) merged commit 71e61dd into main Mar 1, 2026
7 checks passed
@github-actions
Copy link

github-actions bot commented Mar 1, 2026

Braintrust eval report

Autoevals (main-1772387142)

Score Average Improvements Regressions
NumericDiff 74% (-1pp) 1 🟢 4 🔴
Time_to_first_token 1.41tok (-0.56tok) 119 🟢 -
Llm_calls 1.55 (+0) - -
Tool_calls 0 (+0) - -
Errors 0 (+0) - -
Llm_errors 0 (+0) - -
Tool_errors 0 (+0) - -
Prompt_tokens 279.25tok (+0tok) - -
Prompt_cached_tokens 0tok (+0tok) - -
Prompt_cache_creation_tokens 0tok (+0tok) - -
Completion_tokens 18.55tok (+0.06tok) - 1 🔴
Completion_reasoning_tokens 0tok (+0tok) - -
Total_tokens 297.79tok (+0.06tok) - 1 🔴
Estimated_cost 0$ (0$) 119 🟢 -
Duration 2.82s (-0.05s) 137 🟢 82 🔴
Llm_duration 2.85s (-0.9s) 118 🟢 1 🔴

@Qard Stephen Belanger (Qard) deleted the filter-system-messages branch March 3, 2026 02:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant