openai: read meta.n_ctx from /v1/models to fix context limit for local servers

[CreamyG31337]

Summary

Fixes #9529 (related: #8835).

When Goose is pointed at a local llama.cpp or Ollama server, auto-compaction never fires correctly because the context limit falls back to DEFAULT_CONTEXT_LIMIT = 128_000 — even though the server reports the actual window via a non-standard meta.n_ctx field in the /v1/models response.

Changes:

• Add fetch_n_ctx_from_api(&self, model_name: &str) -> Option<usize> to OpenAiProvider — hits the models endpoint, finds the entry matching the active model, and returns meta.n_ctx if present.
• In from_env(), after construction, if model.context_limit is still None (no GOOSE_CONTEXT_LIMIT config and no canonical registry match), call fetch_n_ctx_from_api and set context_limit from the result.
• Fails silently — real OpenAI servers don't include meta, so this is a no-op for cloud usage.

Example response from llama.cpp:

{
  "meta": {
    "n_ctx": 32768,
    "n_ctx_train": 262144
  }
}

With this change, context_limit is set to 32768 instead of 128000, so auto-compaction fires at the correct threshold.

Test plan

• cargo check -p goose passes
• Pointing Goose at a llama.cpp server (without GOOSE_CONTEXT_LIMIT set) shows the correct context window size
• Pointing Goose at real OpenAI shows no change in behavior (field absent, falls back to canonical/default as before)
• Existing tests in openai.rs pass

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 50c4baa1f6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[CreamyG31337]

Addressed both Codex P2 items in 5e01592:

P2-1 (alias matching known OpenAI model): Changed the guard from context_limit.is_none() to !is_openai. For any non-api.openai.com host the server-reported meta.n_ctx now wins regardless of what the canonical registry pre-populated. Hosts without meta.n_ctx (Azure, OpenRouter, etc.) return None and their existing limit is unchanged.

P2-2 (single-model server without --alias): After the id-match loop, if data.len() == 1 we fall back to that sole entry's n_ctx. Covers the common llama.cpp case where the id is the model file path rather than the client-facing name.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5e01592b99

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Preserve explicit context-limit overrides

When GOOSE_CONTEXT_LIMIT (or a config-provided context_limit) is set for a non-OpenAI-compatible endpoint, ModelConfig::new_base records that user override before provider creation, but this assignment unconditionally replaces it with meta.n_ctx. That removes the existing escape hatch for users who intentionally cap compaction below the server's advertised window or correct an inaccurate /v1/models value; the existing canonical-limit path only fills the field when it is still None, so this should avoid overwriting explicit limits while still replacing registry defaults for local aliases.

Useful? React with 👍 / 👎.

[CreamyG31337]

Addressed the P2 in 0311b38.

You're right — switching the guard to !is_openai made it stomp an explicit GOOSE_CONTEXT_LIMIT, which is ironic since that's the exact workaround #9529 documents. Since provider_registry::normalize_model_config runs with_canonical_limits before from_env, an explicit override and a canonical default are indistinguishable on the ModelConfig by the time the provider sees it.

Fix: re-check GOOSE_CONTEXT_LIMIT via config.get_param the same way ModelConfig::new_base does, and skip the meta.n_ctx override when it's explicitly set (> 0). Canonical/known-model alias defaults — which don't come from that key — are still overridden as intended.

Resulting precedence for non-OpenAI hosts: GOOSE_CONTEXT_LIMIT > server meta.n_ctx > canonical/known-models default > DEFAULT_CONTEXT_LIMIT.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0311b38629

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Preserve ACP context-limit overrides

When an ACP client changes the session model with an explicit context_limit, that value is carried in ModelConfig::with_context_limit(...) before provider creation, but this assignment still replaces it whenever GOOSE_CONTEXT_LIMIT is unset and the compatible server reports meta.n_ctx. Fresh evidence in this revision is that the new guard only checks GOOSE_CONTEXT_LIMIT, while ACP per-session overrides are stored only on the incoming ModelConfig (see update_provider passing context_limit into with_context_limit), so users who intentionally cap or correct a local OpenAI-compatible server’s context window lose that override as soon as the provider is rebuilt.

Useful? React with 👍 / 👎.

[CreamyG31337]

Good catch on the ACP path — fixed in 57cd91d, but by stepping back rather than adding a fourth special case.

The ACP finding exposed the real issue: there are several explicit-override channels — GOOSE_CONTEXT_LIMIT, ACP/server per-session with_context_limit(), and GOOSE_PREDEFINED_MODELS — and by provider-construction time all of them, plus a canonical/known-models default, are indistinguishable on the ModelConfig (every one is just context_limit = Some(x)). Chasing each override channel with its own guard is a losing game.

So I've reverted the guard to context_limit.is_none(), which mirrors how with_canonical_limits fills derived limits and preserves every explicit override at once — env, ACP, and predefined all keep working.

On the earlier P2-1 (override canonical alias defaults): I'm intentionally not doing that anymore, and I'd argue it shouldn't be done at this layer. Overriding a canonical default is indistinguishable from overriding a deliberate per-session/ACP limit, so honoring P2-1 necessarily reintroduces this ACP regression. The niche case it covered — deliberately aliasing a local model to a known OpenAI name like gpt-3.5-turbo — is still fully addressable with GOOSE_CONTEXT_LIMIT, the documented escape hatch. Fully resolving P2-1 and preserving all overrides would require threading provenance (explicit vs. registry-derived) through ModelConfig + normalize_model_config, which is a larger change I'm happy to do as a follow-up if you'd prefer it.

Net precedence for local servers now: any explicit limit (env / ACP / predefined / canonical) > server meta.n_ctx > DEFAULT_CONTEXT_LIMIT (128k).

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 57cd91d899

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Avoid blocking provider creation on optional /models probe

When context_limit is unset for an unknown or custom OpenAI-compatible model, this now awaits an optional /models request before returning the provider. That request uses the normal OPENAI_TIMEOUT client timeout, which defaults to 600 seconds (DEFAULT_PROVIDER_TIMEOUT_SECS), and any error is discarded afterward; in environments where the configured endpoint is reachable but /v1/models hangs or is slow, starting Goose or switching to that model can block for up to 10 minutes before falling back to the old behavior. Consider bounding this metadata probe with a short timeout or deferring it so the best-effort lookup cannot stall provider construction.

Useful? React with 👍 / 👎.

[CreamyG31337]

Good catch, and a real one — fixed in 7281663.

The probe reused the provider's ApiClient, whose timeout is OPENAI_TIMEOUT (default DEFAULT_PROVIDER_TIMEOUT_SECS = 600s), so a reachable-but-hanging /v1/models could have blocked provider construction — and thus Goose startup / model switching — for up to 10 minutes. Wrapped the probe in a 5s tokio::time::timeout; on timeout or any error it silently falls back to the previous behavior, so a slow or missing models endpoint can't stall construction.

FWIW this revision now compiles clean locally (cargo check -p goose, only the two pre-existing unrelated warnings) and I'm running the providers::openai unit tests.

[FredApps]

The only failing check is cargo fmt. All tests/lint/MSRV pass. Just run cargo fmt --package goose and push — the long line in from_env() is almost certainly what rustfmt wants to wrap:

tokio::time::timeout(N_CTX_PROBE_TIMEOUT, provider.fetch_n_ctx_from_api(&model_name))

[CreamyG31337]

Thanks! Done in 1c23a58 — cargo fmt --package goose wrapped exactly that from_env() line as you predicted. cargo fmt --package goose --check is now clean locally. Should be green on the next CI run.

[DOsinga]

Thanks for the fix and for filing the linked issue (#9529) — this is a genuinely useful bug for local OpenAI-compatible servers, and nice work responding to the Codex feedback.

I pushed a small cleanup commit to your branch (603c9833):

• Condensed the long rationale comment in from_env and dropped the ownership shuffle (made provider mut instead).
• Extracted the meta.n_ctx parsing into a pure parse_n_ctx_from_models function so the matching/fallback logic is unit-testable without an HTTP mock.
• Added four tests covering: match-by-id, single-entry fallback when the id differs (the llama.cpp file-path case), no fallback with multiple unmatched entries, and missing meta.

One thing left before this can merge: the DCO check is failing because none of your commits carry a Signed-off-by line. Could you sign off your existing commits? The simplest way:

git rebase HEAD~6 --signoff   # or however many commits are yours
git push --force-with-lease

(Note that will re-push over my cleanup commit too — feel free to re-sign all of them, or cherry-pick mine back; it's already signed.) Once DCO is green this looks good to me.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 603c983386

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Probe n_ctx for custom OpenAI providers

When a local llama.cpp/vLLM endpoint is added through the custom OpenAI-compatible provider flow, the registry constructs it via OpenAiProvider::from_custom_config (crates/goose/src/config/declarative_providers.rs:576-586) and the custom-provider UI/CLI stores each entered model with a hard-coded 128000 context limit (create_custom_provider), so this from_env-only is_none probe never runs and auto-compaction still uses the wrong 128k window. That leaves the documented custom-provider path for local OpenAI-compatible servers unaffected by this fix unless users hand-edit JSON with the real limit; consider applying the metadata lookup to custom OpenAI providers/defaulted model limits as well.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8bc9003901

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Read llama.cpp runtime context from /props

For current llama.cpp servers, /v1/models metadata documents and examples expose meta.n_ctx_train but not the allocated runtime window; the runtime n_ctx is reported under /props as default_generation_settings.n_ctx in the llama.cpp server README (https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md). With this .get("n_ctx") lookup, Goose still returns None for a normal llama.cpp /v1/models response and falls back to 128k, so users running llama.cpp without GOOSE_CONTEXT_LIMIT keep the wrong auto-compaction threshold despite this fix.

Useful? React with 👍 / 👎.

[DOsinga]

Thanks for the careful work here, and especially for engaging so thoroughly with the codex review — addressing each point with clear reasoning (including knowing when not to chase an override path). This is a clean, well-scoped fix that reads meta.n_ctx from /v1/models for local OpenAI-compatible servers, which fixes context sizing without affecting cloud OpenAI. Approving.