Skip to content

Conversation

@roomote
Copy link
Contributor

@roomote roomote bot commented Jul 28, 2025

Summary

This PR implements the fix requested by @NaccOll in issue #6290 to handle a special case in the VS Code Language Model API token counting.

Problem

The VS Code LM API was not correctly calculating context size when using the "copilot - gpt-4.1" model. The issue was that the API's countTokens method was returning incorrect values for LanguageModelChatMessage objects.

Solution

As requested by @NaccOll, this PR:

  • Continues using the internalCountTokens method as the primary approach
  • Adds special handling: when the text type is LanguageModelChatMessage AND tokenCount equals 4, it converts the message to a string and recalculates using this.client.countTokens

Changes

  1. Modified internalCountTokens method in src/api/providers/vscode-lm.ts:

    • Added a check for when tokenCount === 4 for LanguageModelChatMessage objects
    • Extracts text content from the message and recalculates token count as a string
    • Added debug logging for transparency
  2. Added comprehensive test coverage in src/api/providers/__tests__/vscode-lm.spec.ts:

    • Test case for the special handling when tokenCount equals 4
    • Test case to ensure normal behavior when tokenCount is not 4
    • Test case for handling image blocks in content

Testing

  • All existing tests pass ✅
  • New tests specifically cover the edge case ✅
  • Ran full test suite for api/providers to ensure no regressions ✅

Fixes #6290


Important

Fixes token counting issue in vscode-lm.ts by recalculating when tokenCount is 4, with tests added in vscode-lm.spec.ts.

  • Behavior:
    • Modifies internalCountTokens in vscode-lm.ts to handle tokenCount === 4 for LanguageModelChatMessage by converting to string and recalculating.
    • Adds debug logging for recalculation process.
  • Testing:
    • Adds test for special case when tokenCount is 4 in vscode-lm.spec.ts.
    • Tests normal behavior when tokenCount is not 4.
    • Tests handling of image blocks in content.
  • Misc:
    • All existing tests pass.
    • Full test suite run to ensure no regressions.

This description was created by Ellipsis for 6433d87. You can customize this summary. It will automatically update as commits are pushed.

- Add special handling in internalCountTokens method for LanguageModelChatMessage
- When tokenCount equals 4, convert message to string and recalculate
- Add comprehensive test coverage for the special case
- Fixes #6290
@roomote roomote bot requested review from cte, jr and mrubens as code owners July 28, 2025 12:30
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. bug Something isn't working labels Jul 28, 2025
@hannesrudolph hannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Jul 28, 2025
@daniel-lxs
Copy link
Member

Closing since these are duplicates of #6424

@daniel-lxs daniel-lxs closed this Jul 30, 2025
@github-project-automation github-project-automation bot moved this from Triage to Done in Roo Code Roadmap Jul 30, 2025
@github-project-automation github-project-automation bot moved this from New to Done in Roo Code Roadmap Jul 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. size:L This PR changes 100-499 lines, ignoring generated files.

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

VS Code LM API unable to get the correct context size

4 participants