feat: add lite mode for smaller context in local models #9352
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR attempts to address Issue #9351. It implements a "lite mode" feature that significantly reduces the system prompt size from ~14k tokens to ~4k tokens, making it more suitable for local models with limited context windows.
Problem
Local models (especially those with 32k-64k context) struggle with Roo's large default system prompts, limiting their ability to process files and maintain performance. The issue reporter experienced this with Qwen3 Coder 30b, where the 14k token prompt left little room for actual work.
Solution
Added a "Lite Mode" toggle in the Context Management settings that:
Changes
liteModeoption toSystemPromptSettingssrc/core/prompts/tools/lite-descriptions.tsTesting
Token Reduction
This allows local models to have significantly more context available for actual file processing and conversation history.
Notes
Feedback and guidance are welcome!
Important
Introduces a "lite mode" to reduce system prompt size for local models, adding a UI toggle and supporting state management and tool descriptions.
ContextManagementSettings.tsx.liteModeoption toSystemPromptSettingsintypes.ts.getCapabilitiesSection()incapabilities.tsandgetRulesSection()inrules.tsto support lite mode.lite-descriptions.ts.toolDescriptionMapinindex.tsto use lite descriptions when lite mode is enabled.ExtensionStateContext.tsxandExtensionMessage.ts.This description was created by
for 9bdc844. You can customize this summary. It will automatically update as commits are pushed.