Skip to content

fix(models): merge curated + live catalog for Copilot provider - #44257

Open
nguyenhung2904 wants to merge 1 commit into
NousResearch:mainfrom
nguyenhung2904:fix/copilot-merge-curated-with-live-models
Open

fix(models): merge curated + live catalog for Copilot provider#44257
nguyenhung2904 wants to merge 1 commit into
NousResearch:mainfrom
nguyenhung2904:fix/copilot-merge-curated-with-live-models

Conversation

@nguyenhung2904

Copy link
Copy Markdown

Problem

When get_available_models() is called for the copilot provider, it fetches the live catalog from the GitHub Copilot API and returns it verbatim — discarding the curated static list (_PROVIDER_MODELS['copilot']) entirely whenever the live fetch succeeds.

This means models that are enabled on the account but absent from the live OAuth token's catalog response never appear in /model.

Observed symptom: claude-fable-5 (model_picker_enabled=True, type=chat, listed under /chat/completions) was missing from the model picker despite being enabled on the GitHub Copilot account. The curated list included it; the live catalog from the OAuth token did not.

Fix

Mirrors the approach already used for the anthropic provider: curated entries appear first (preserving intended ordering and pinned aliases), then any live-only models are appended so newly-enabled account models still surface automatically without requiring a Hermes release.

# Before
if live:
    return live

# After
if live:
    curated = list(_PROVIDER_MODELS.get("copilot", []))
    curated_lower = {m.lower() for m in curated}
    merged = list(curated)
    for m in live:
        if m.lower() not in curated_lower:
            merged.append(m)
    return merged

Testing

Verified locally: claude-fable-5 now appears in the /model picker after the fix, whereas it was absent before.

Previously, get_available_models() for the 'copilot' provider returned
the live API catalog verbatim and discarded the curated static list
(_PROVIDER_MODELS['copilot']) whenever the live fetch succeeded.

This mirrors the approach already used for the 'anthropic' provider:
curated entries appear first (preserving intended ordering / pinned
aliases), then any live-only models are appended so newly-enabled
account models surface automatically without requiring a Hermes release.

Root cause observed: claude-fable-5 (model_picker_enabled=True, type=chat,
supported on /chat/completions) was absent from the /model picker despite
being enabled on the GitHub Copilot account, because the OAuth token used
for live catalog fetching returned a subset of the account's allowed
models. The curated list included it; the live catalog did not.
@liuhao1024

Copy link
Copy Markdown
Contributor

✅ Verification: Clean Review

Reviewed this PR's diff. Two clean changes:

  1. Curated model list update — adds newer models (claude-fable-5, claude-opus-4.8, gpt-5.5, gemini-3.5-flash), removes stale entries. Pure data update.
  2. Merge logic in provider_model_ids() — instead of returning live alone, now merges curated-first with live-only additions. Case-insensitive dedup via curated_lower set is correct.

The merge strategy (curated order preserved, live-only models appended) is the right approach — it ensures pinned models always appear while still surfacing newly-enabled models from the live API.

No issues found.

@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/cli CLI entry point, hermes_cli/, setup wizard provider/copilot GitHub Copilot (ACP + Chat) labels Jun 11, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for isolating the Copilot-specific catalog path. The underlying issue is still present on current main: provider_model_ids() returns a nonempty Copilot live list directly at hermes_cli/models.py:2319-2323, before the generic live+curated merge path used by merged PR #46857.

Problems

  • The PR changes that result to a merged catalog but does not update tests/hermes_cli/test_model_validation.py:208-216. Those tests currently mock a two-model live response and assert that it is the complete result, which conflicts with the proposed curated-first behavior.

Suggested changes

  • Replace that live-only assertion with a Copilot regression test that mocks both the curated and live lists and verifies curated-first ordering, live-only preservation, and case-insensitive deduplication. Exercise copilot-acp too, since it shares the branch at hermes_cli/models.py:2319-2327.

Automated hermes-sweeper review.

Comment thread hermes_cli/models.py
if m.lower() not in curated_lower:
merged.append(m)
return merged
except Exception:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update tests/hermes_cli/test_model_validation.py:208-216 with this behavior: it currently asserts the mocked live list is returned verbatim, so it will fail once this merged result is returned. A regression test should mock curated and live lists and assert curated-first ordering plus live-only retention and deduplication.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have provider/copilot GitHub Copilot (ACP + Chat) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants