fix(title_generator): pass timeout from caller and reduce max_tokens to 64 - #42548
Conversation
…to 64 A 30s hardcoded timeout hangs title generation when the title model is slow or GPU-contended. Making it Optional lets upstream callers control the deadline. Also reduce max_tokens from 500 to 64 since session titles only need a short phrase.
tonydwb
left a comment
There was a problem hiding this comment.
Code Review Summary
Verdict: Approved
Overview
This PR reduces max_tokens from 500 to 64 for title generation and makes timeout optional.
Quality
- Clean fix
- No security concerns
Reviewed by Hermes Agent
teknium1
left a comment
There was a problem hiding this comment.
Thanks for the focused title-generation change. The timeout portion has since landed independently: current main uses timeout: Optional[float] = None at agent/title_generator.py:54, via 5126902, and covers the configured/default and explicit-timeout paths in tests/agent/test_title_generator.py:62-91.
Problems
- The remaining change is only
max_tokens=500→64atagent/title_generator.py:85; the PR body describes a 128 → 64 change, which does not match the submitted diff. - No test covers the proposed title-generation token budget.
Suggested changes
- Re-scope this to the remaining 500 → 64 proposal, explain the cap choice, and add focused coverage for the intended request budget.
Automated hermes-sweeper review.
| @@ -56,7 +56,7 @@ def generate_title( | |||
| response = call_llm( | |||
| task="title_generation", | |||
There was a problem hiding this comment.
This timeout change is already on current main in 5126902 (agent/title_generator.py:54) with regression coverage in tests/agent/test_title_generator.py:62-91; a salvage should omit this redundant hunk.
| @@ -56,7 +56,7 @@ def generate_title( | |||
| response = call_llm( | |||
| task="title_generation", | |||
There was a problem hiding this comment.
The PR body says 128 → 64, but this diff changes 500 → 64. Please correct the stated baseline and add focused coverage for the proposed request budget.
|
Thanks for the contribution. Closing as redundant:
|
Summary
Local verification
Commits
text
5117161 fix(title_generator): pass timeout from caller and reduce max_tokens to 64