fix: use consistent token counting for truncation before/after display#11991
Draft
roomote-v0[bot] wants to merge 1 commit intomainfrom
Draft
fix: use consistent token counting for truncation before/after display#11991roomote-v0[bot] wants to merge 1 commit intomainfrom
roomote-v0[bot] wants to merge 1 commit intomainfrom
Conversation
The truncation UI showed "after" token count larger than "before" because they used different counting methods: prevContextTokens used API-reported values (tokensIn + tokensOut) while newContextTokensAfterTruncation used local estimation (system prompt + message content). When providers report different values than local estimation, the display became inconsistent. Now both values use the same local estimation method, ensuring the "after" count is always less than the "before" count. Fixes #11990
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related GitHub Issue
Closes: #11990
Description
This PR attempts to address Issue #11990 where the truncation UI shows "after" token count larger than "before".
Root cause: In
manageContext(),prevContextTokens(the "before" value) was calculated using API-reported token counts (tokensIn + tokensOut + lastMessageTokens), whilenewContextTokensAfterTruncation(the "after" value) was calculated using local token estimation (system prompt + message content). When API providers (e.g. OpenRouter) report different values than local estimation, the display became inconsistent -- "after" could appear larger than "before".Fix: Both "before" and "after" values now use the same local token estimation method (
estimateTokenCountvia the provider'scountTokens). The system prompt tokens are computed once and reused for both calculations. The API-reported value is still used for the threshold decision (whether to trigger truncation), but the display values are always consistent.Key changes:
src/core/context-management/index.ts: CalculatelocalPrevContextTokensusing the same method asnewContextTokensAfterTruncationin the truncation pathsrc/core/context-management/__tests__/context-management.spec.ts: Updated tests to verify the consistency invariant (prevContextTokens >= newContextTokensAfterTruncation) instead of checking against API-reported valuesFeedback and guidance are welcome.
Test Procedure
context-management.spec.tspasstruncation.spec.tsandgetApiMetrics.spec.tspassprevContextTokens >= newContextTokensAfterTruncationPre-Submission Checklist
Documentation Updates
Additional Notes
The fix only affects the truncation path in
manageContext(). The condensation path and non-action path still use the API-reportedprevContextTokensvalue, which is correct for those cases since the condensation result also uses a different token counting approach that is already consistent with its "before" value.Interactively review PR in Roo Code Cloud