fix(agent): default context lookup for empty model IDs (salvage #65515) — un-reds main CI - #85498
Merged
kshitijk4poor merged 1 commit intoAug 13, 2026
Conversation
An empty/blank model id reaching get_model_context_length() can't be
meaningfully resolved — and it's worse than a miss: the endpoint
metadata fuzzy matcher ('model in key or key in model') is vacuously
true for "", so it matches an ARBITRARY catalog entry from the live
/v1/models response and returns whatever context length that entry
happens to have, persisting it under a junk '@<base_url>' cache key.
This started failing CI on main when the Nous portal catalog changed:
tests/run_agent/test_primary_runtime_restore.py constructs agents with
model='' against the live portal URL, the arbitrary match now lands on
a 32K entry, and init_agent raises the 64K-floor ValueError
(test_allowed_for_nous_anthropic_messages, red on every PR's slice).
Guard early: a blank model id falls back to DEFAULT_FALLBACK_CONTEXT
immediately, before any cache write or network probe.
Salvaged from NousResearch#65515 by @whirmill (rebased onto current main; the
guard now sits after the malformed-base_url normalization added since,
and carries an explanatory comment for the fuzzy-match footgun).
Fixes the red slice on NousResearch#85444, NousResearch#85452 and every other open PR.
Co-authored-by: whirmill <5079591+whirmill@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every open PR's CI is currently red on one shard with:
(Seen on #85444 slice 7/12, #85452 slice 2/12; reproduces locally on plain
upstream/mainfiles.)Root cause: the test constructs an agent with
model=""against the live Nous portal URL.get_model_context_length("")reaches_resolve_endpoint_context_length, whose fuzzy matcheris vacuously true for an empty model —
"" in keymatches every entry — so it picks an arbitrary model from the live/v1/modelsresponse and returns that entry's context length. The portal catalog changed recently; the arbitrary match now lands on a 32K entry,init_agentraises the 64K-floorValueError, and the junk value is persisted under a@https://inference-api.nousresearch.com/v1cache key (visible in the failing log:Cached context length @https://... -> 32,000 tokens).This is a salvage of #65515 by @whirmill (open since July 16), rebased onto current main — the same guard, now positioned after the malformed-base_url normalization that landed since, plus an explanatory comment documenting the fuzzy-match footgun. Contributor authorship preserved on the commit.
Changes
agent/model_metadata.py: a blank/empty model id falls back toDEFAULT_FALLBACK_CONTEXTimmediately — before any cache write or network probe.tests/agent/test_model_metadata.py:test_empty_model_uses_fallback_context(covers""andNone).Validation
test_allowed_for_nous_anthropic_messages+test_empty_model_uses_fallback_context.tests/agent/test_model_metadata.py+tests/run_agent/test_primary_runtime_restore.py: 92 passed.test_empty_model_uses_fallback_contextfail (arbitrary live-catalog match returns non-default), restoring goes green.ruff checkclean.Closes #65515.