Skip to content

docs: tool system performance audit for long-session regressions - #7695

Closed
kilo-code-bot[bot] wants to merge 1 commit into
mainfrom
research/tool-system-performance-audit
Closed

docs: tool system performance audit for long-session regressions#7695
kilo-code-bot[bot] wants to merge 1 commit into
mainfrom
research/tool-system-performance-audit

Conversation

@kilo-code-bot

@kilo-code-bot kilo-code-bot Bot commented Mar 26, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Research audit of recent changes (Feb 26 - Mar 26, 2026) to the tool system, system prompts, and agent definitions that could cause performance regressions in long-running sessions
  • Documents token budget impact of each change with specific PRs, commits, file paths, and diffs
  • Identifies the session diff memory leak fix (PR fix(cli): prevent CLI memory leaks from oversized diffs and session accumulation #7617) as the most impactful change for long-session performance

Key Findings

Change PR Per-request token impact
codebase_search tool #6685 +~350 tokens (experimental flag)
environment_details injection #6225 +15 to +250 tokens (always)
Config paths in system prompt #7444 +~35 tokens (always)
PlanExitTool always registered N/A 0 (filtered by agent perms)
Session diff memory leak fix #7617 N/A (storage fix, not token)

Typical case (default settings, CLI): +~50 tokens per request
Worst case (all features enabled, VS Code with many tabs): +~635 tokens per request

Recommendations

  1. Cap open tabs included in environment_details to limit token growth
  2. Consider whether the always-present ISO 8601 timestamp is worth ~15 tokens/request
  3. bash.txt (9.6 KB) and todowrite.txt (8.8 KB) are the largest tool descriptions — prime targets for compression if token budget is a concern

Research recent changes (Feb 26 - Mar 26, 2026) to tool definitions,
system prompts, and agent metadata that could cause performance
regressions in long-running sessions by increasing token count.

Key findings:
- New codebase_search tool (+350 tokens when experimental flag on)
- environment_details injection on every request (+15-250 tokens)
- Config paths added to system prompt (+35 tokens)
- PlanExitTool always registered (filtered by agent permissions)
- Session diff memory leak fix (PR #7617) addresses major long-session issue
1. A new `codebase_search` tool was added (PR #6685), adding ~974 bytes to tool definitions when enabled via experimental flag.
2. The `environment_details` system was introduced (PR #6225), injecting per-message dynamic context into user messages — a net token increase but only on the last user message.
3. Config location awareness was added to the system prompt (PR #7444), adding ~2 lines to every `<env>` block.
4. `PlanExitTool` is now always registered in the tool registry instead of being feature-flag-gated, adding its description (~614 bytes) to every session.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Executive summary overstates the PlanExitTool token impact

The detailed analysis later in this document correctly notes that resolveTools() filters plan_exit out for agents that do not allow it, so its description is not added to most LLM requests. Saying it is added "to every session" contradicts that conclusion and overstates the reported regression risk.

Suggested change
4. `PlanExitTool` is now always registered in the tool registry instead of being feature-flag-gated, adding its description (~614 bytes) to every session.
4. `PlanExitTool` is now always registered in the tool registry instead of being feature-flag-gated, but agent permission filtering keeps its description out of most sessions.

@kilo-code-bot

kilo-code-bot Bot commented Mar 26, 2026

Copy link
Copy Markdown
Contributor Author

Code Review Summary

Status: 1 Issue Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0

Fix these issues in Kilo Cloud

Issue Details (click to expand)

WARNING

File Line Issue
docs/research/tool-system-performance-audit.md 16 Executive summary says PlanExitTool adds ~614 bytes to every session, but the later analysis correctly notes agent permission filtering keeps it out of most LLM requests.
Other Observations (not in diff)

Issues found in unchanged code that cannot receive inline comments:

None.

Files Reviewed (1 files)
  • docs/research/tool-system-performance-audit.md - 1 issue

Reviewed by gpt-5.4-20260305 · 293,893 tokens

@marius-kilocode

Copy link
Copy Markdown
Collaborator

This writes to docs/ at the repo root, but we don't have a top level docs/ tree here. Looks like a one off research note that wasn't going to any published docs surface, closing it out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant