fix(agent): let context length resolution fall through when failed to get context_length from provider - #6495
Open
nocoo wants to merge 1 commit into
Conversation
…dpoint lacks context_length When a custom endpoint (e.g. an API proxy) recognizes a model name in its /v1/models response but does not include a context_length field, the early return on line 912 caused get_model_context_length() to return DEFAULT_FALLBACK_CONTEXT (128K) immediately. This prevented the function from reaching later resolution steps (Anthropic API, models.dev, OpenRouter, hardcoded defaults) that could correctly resolve the context length. For example, a proxy serving 'claude-opus-4.6-1m' without context_length metadata would return 128K instead of 1M, because the hardcoded DEFAULT_CONTEXT_LENGTHS entry 'claude-opus-4.6' (a substring match) was never reached. Remove the early return and let the resolution chain continue. Keep the logger.info message for observability. Update the existing test that asserted the old 128K behavior and add two new regression tests covering the exact scenarios fixed.
nocoo
marked this pull request as ready for review
April 9, 2026 09:11
6 tasks
Collaborator
1 similar comment
Collaborator
Contributor
|
Thanks for isolating the custom-endpoint early-return issue. Problems
Suggested changes
This is an automated hermes-sweeper review. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes an early return in
get_model_context_length()that causes custom endpoints withoutcontext_lengthin their/v1/modelsresponse to default to 128K, even when later resolution steps (hardcoded defaults, models.dev, OpenRouter) could correctly resolve the context length.Example: A proxy serving
claude-opus-4.6-1mthat omitscontext_lengthfrom its model metadata would return 128K instead of 1M, because the hardcodedDEFAULT_CONTEXT_LENGTHSentry"claude-opus-4.6"(a valid substring match at step 8) was never reached.Type of Change
Changes Made
agent/model_metadata.py: Remove the earlyreturn DEFAULT_FALLBACK_CONTEXTat the end of the custom endpoint probe block (step 2/3). The function now falls through to remaining resolution steps (4–8) when the endpoint probe does not yield acontext_length. Thelogger.infomessage is preserved for observability.tests/agent/test_model_metadata.py: Updatetest_custom_endpoint_without_metadata_falls_through_to_hardcoded_default(formerly..._skips_name_based_default) to assert the new fall-through behavior. Add two new regression tests:test_custom_endpoint_no_context_length_falls_through_to_defaults— endpoint returns model metadata withoutcontext_lengthtest_custom_endpoint_no_match_falls_through_to_defaults— endpoint returns unrelated models, requested model falls through to hardcoded defaultsHow to Test
Checklist
Code
pytest tests/ -qpasses