perf: add backoff delay to empty response retries (#35230) - #38002
Closed
SandroHub013 wants to merge 2 commits into
Closed
perf: add backoff delay to empty response retries (#35230)#38002SandroHub013 wants to merge 2 commits into
SandroHub013 wants to merge 2 commits into
Conversation
…ch#29590) Read max_tokens from auxiliary.vision config section instead of using hardcoded values (2000 for vision_analyze, 4000 for video_analyze). - Add max_tokens to auxiliary.vision config defaults (4000) - vision_analyze: read config, fallback to 2000 - video_analyze: read config, fallback to 8192 - Update existing config tests to verify max_tokens behavior
When the model returns an empty response (no content, no reasoning), the agent previously retried up to 3 times with zero delay. All retries fired back-to-back within seconds, wasting API calls. Add jittered exponential backoff (base=0.5s, max=4s) between empty response retries, consistent with the existing API error retry mechanism using jittered_backoff(). This prevents rapid-fire retries when the model is temporarily degraded. Co-authored-by: SandroHub013 <SandroHub013@users.noreply.github.com>
Author
|
Duplicate of #35296 — same fix already open. Closing. |
alt-glitch
added
type/perf
This was referenced Jul 29, 2026
[Enhancement] Empty response retries have no backoff delay — immediate retry wastes API calls
#35230
Closed
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds jittered exponential backoff between empty response retries in the conversation loop, preventing rapid-fire retries that waste API calls.
Problem
When the model returns an empty response (no content, no reasoning), the agent retries up to 3 times with zero delay. All retries fire back-to-back within seconds. If the model is temporarily degraded, all 3 retries are wasted. Worst case: 6 rapid-fire calls (3 primary + 3 fallback).
This is inconsistent with the existing
jittered_backoff()mechanism used for API-level errors (rate limits, timeouts, 5xx), which properly waits between attempts.Fix
Added
time.sleep(jittered_backoff(...))withbase_delay=0.5s, max_delay=4sto the empty-response retry path inagent/conversation_loop.py. This produces delays of approximately:Consistent with the existing API error retry mechanism using the same
jittered_backoff()utility.Files Changed
agent/conversation_loop.py— Added backoff delay beforecontinuein empty response retry block (~line 4163)Testing
Existing tests in
test_run_agent.pycover empty response retry paths. The test conftest already patchesjittered_backoffto return 0.0 for fast test execution, so no test changes needed.Fixes #35230