Fix issue #3454: Add proactive context length checking to prevent empty LLM responses

[devin-ai-integration]

Fix issue #3454: Add proactive context length checking to prevent empty LLM responses

Summary

Adds proactive context length checking to prevent empty responses from LLM providers like DeepInfra when input exceeds the model's context window. The current reactive approach only catches exceptions, but some providers return empty responses instead of throwing errors.

Key Changes:

• Added _check_context_length_before_call() method that estimates token count using character count ÷ 4
• Integrated proactive check into _invoke_loop() before LLM calls
• Reuses existing _handle_context_length() logic for summarization/exit behavior
• Added comprehensive unit tests covering proactive handling and token estimation

Review & Testing Checklist for Human

This is a medium-risk change that requires careful validation:

• Test with real long inputs: Verify the fix works with actual LLM providers (especially DeepInfra) using inputs that exceed context windows
• Validate token estimation: Check if the characters÷4 approximation is reasonable for your use cases - could be significantly off for certain content types
• Performance testing: Measure overhead of character counting on large message histories in production scenarios
• Edge case testing: Test with empty messages, None content, zero context window size, etc.
• Integration verification: Ensure proactive and reactive context handling don't conflict or double-trigger

Notes

• Unit tests pass but integration tests had authentication issues, so real-world validation is critical
• The character-based token estimation is intentionally conservative given LLM's existing 75% safety margin
• Requested by João ([email protected]) via Slack
• Session: https://app.devin.ai/sessions/5177f947ee8e47b19af9164c065db422

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[devin-ai-integration]

Closing due to inactivity for more than 7 days. Configure here.