Skip to content

fix(auxiliary): use max_completion_tokens for Copilot GPT-5 compression - #34532

Closed
Bartok9 wants to merge 2 commits into
NousResearch:mainfrom
Bartok9:fix/34530-copilot-gpt5-max-completion-tokens
Closed

Bartok9 wants to merge 2 commits into
NousResearch:mainfrom
Bartok9:fix/34530-copilot-gpt5-max-completion-tokens

Conversation

@Bartok9

@Bartok9 Bartok9 commented May 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Auxiliary context compression now sends max_completion_tokens (not max_tokens) when the resolved request endpoint is GitHub Copilot or direct OpenAI.
  • Fixes context compression silently degrading to a static fallback marker on GPT-5 series models behind GitHub Copilot.

Motivation

Closes #34530.

When auxiliary.compression is configured with provider: github-copilot and a GPT-5 series model, the compression summary request failed with:

HTTP 400 - Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.

Hermes then inserted a static fallback context marker and dropped middle conversation turns without a semantic summary.

Root cause: _build_call_kwargs only emitted max_completion_tokens for provider == "custom" on api.openai.com. GitHub Copilot normalises to provider == "copilot" (see _normalize_aux_provider / _PROVIDER_ALIASES), so the call_llm_build_call_kwargs compression path fell through to the generic max_tokens branch. Notably auxiliary_max_tokens_param (used on the main-model path) already lists both api.openai.com and api.githubcopilot.com — the two were inconsistent.

The fix keys the parameter choice on the resolved request hostname rather than the provider label, covering both api.openai.com and api.githubcopilot.com regardless of whether the provider is copilot, custom, or openai. This brings _build_call_kwargs in line with auxiliary_max_tokens_param.

Verification

  • python3 -m pytest tests/agent/test_auxiliary_client.py tests/agent/test_unsupported_parameter_retry.py — 204 passed
  • New TestBuildCallKwargsMaxTokensParam covers: copilot provider + copilot host → max_completion_tokens; copilot host with /chat/completions path; direct OpenAI host; provider="custom" + OpenAI host (pre-existing behaviour preserved); OpenRouter still uses max_tokens; unknown/empty host still uses max_tokens (no regression for DeepSeek/etc.).
  • Did NOT change the existing reactive strip-and-retry fallback; this fix sends the correct parameter up front so the summary keeps its token budget instead of being retried with no limit.

Bartok9 added 2 commits May 29, 2026 05:50
Auxiliary context compression configured with provider=github-copilot and a
GPT-5 series model failed with HTTP 400 ("Unsupported parameter: 'max_tokens'
... Use 'max_completion_tokens'"), falling back to a static context marker and
dropping middle turns without a semantic summary.

_build_call_kwargs only emitted max_completion_tokens for provider=="custom"
on api.openai.com. github-copilot normalises to provider="copilot", so the
compression call_llm path fell through to max_tokens. Key the parameter choice
on the resolved request hostname (api.openai.com + api.githubcopilot.com),
matching auxiliary_max_tokens_param which already lists both hosts.

Closes NousResearch#34530
…isation

The auxiliary_client fix in this PR routes max_tokens to max_completion_tokens
for api.openai.com (and api.githubcopilot.com) — these endpoints reject the
legacy parameter name on newer GPT-4o/o-series/GPT-5 models.

The existing temperature-retry tests asserted retry_kwargs['max_tokens'] == 500
without considering the param-name normalisation that happens for those hosts.
The retry path correctly preserves the budget — only the key name changes.
Accept either max_tokens or max_completion_tokens in the assertion.
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/copilot GitHub Copilot (ACP + Chat) labels May 29, 2026
@alt-glitch

Copy link
Copy Markdown
Contributor

Competing with #19265 (broader scope, open since May 3) and #34534 (also targets #34530). All three fix the same max_tokensmax_completion_tokens issue for GitHub Copilot. This PR's hostname-based approach aligns with the existing auxiliary_max_tokens_param pattern.

@Bartok9

Bartok9 commented May 31, 2026

Copy link
Copy Markdown
Contributor Author

Closing as superseded by #34845 (2062a8400, fix(auxiliary): stop capping output with max_tokens by default (#34530)), which landed on main and addresses the same issue #34530 more comprehensively.

My PR converted max_tokensmax_completion_tokens for the Copilot/OpenAI GPT-5 hosts. Main's merged fix instead omits the output cap entirely by default (keeping max_tokens only on the Anthropic-compat wire where it's mandatory), which sidesteps the Copilot/GPT-5 max_tokens 400 — and the ZAI 1210 case — without sending any cap at all. Rebasing my hostname-branch approach onto current main would re-introduce a cap that #34845 deliberately removed, so the right move is to close. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists provider/copilot GitHub Copilot (ACP + Chat) type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Auxiliary context compression sends max_tokens to GitHub Copilot GPT-5 models

2 participants