fix(model-metadata): local ctx probe must not read max_tokens as the context window

[Kyzcreig]

The second half of the early-auto-compaction bug

Companion to #260. After the lm-studio-misdetection fix reclassified the fleet's Claude proxy as a generic local server, the local context-length probe (_query_local_context_length_uncached) hit the proxy's Anthropic passthrough GET /v1/models/{id}, which returns BOTH:

• max_input_tokens: 1000000 — the context window
• max_tokens: 128000 — the max output tokens

The alias chain read max_model_len or context_length or max_tokens, so it picked up max_tokens=128000 and cached it as the window → collapsed a 1M model to 128K → auto-compaction at ~96K (0.75 × 128K). This is why sessions kept compacting early even with the proxy correctly advertising context_length: 1000000.

Fix

Both the /v1/models/{id} and the /v1/models list branches now read genuine context-window fields only — max_model_len, context_length, context_window, max_input_tokens, max_position_embeddings — and drop max_tokens (output tokens, never the context window).

Verified live end-to-end against the running proxy: _query_local_context_length and the full get_model_context_length now resolve claude-fable-5 → 1000000, claude-haiku-4-5 → 200000.

Tests

Regression on both branches: an Anthropic-style passthrough with max_input_tokens+max_tokens must resolve to the input/context value, never the output cap. 36 pass in the local-ctx suite.

[greptile-apps]

Greptile Summary

This PR fixes premature auto-compaction caused by _query_local_context_length_uncached mistakenly reading max_tokens (the max output token cap) as the context window size on Anthropic-style proxy endpoints. The fix removes max_tokens from the alias chain in both the /v1/models/{id} detail branch and the /v1/models list branch, and replaces it with the correct context-window fields (context_window, max_input_tokens, max_position_embeddings).

• agent/model_metadata.py: Both GET branches now probe only genuine context-window fields (max_model_len, context_length, context_window, max_input_tokens, max_position_embeddings) and explicitly exclude max_tokens, with a detailed comment explaining why.
• tests/agent/test_model_metadata_local_ctx.py: Two regression tests are added — one for the detail-endpoint branch and one for the list-endpoint branch — each asserting that max_input_tokens (1 000 000) wins over max_tokens (128 000) in an Anthropic-style passthrough response.

Confidence Score: 5/5

Safe to merge — the change is a narrow two-line alias-chain edit that removes a misread field and adds semantically correct alternatives, backed by targeted regression tests for both affected code paths.

The root cause is clearly identified and the fix is minimal: max_tokens is removed from both the detail-endpoint and list-endpoint alias chains, replaced with fields that actually represent the context window. Both changed branches have dedicated regression tests with correctly ordered mock side-effects.

No files require special attention. Both changed files are straightforward and the test coverage directly mirrors the production failure scenario.

Important Files Changed

Filename	Overview
agent/model_metadata.py	Removes max_tokens from both alias chains and adds context_window, max_input_tokens, max_position_embeddings; change is minimal, well-commented, and correctly targeted
tests/agent/test_model_metadata_local_ctx.py	Adds two regression tests covering both the detail-endpoint and list-endpoint branches; mocks are set up correctly and assertions are specific

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[_query_local_context_length_uncached] --> B{server_type}
    B -->|ollama| C[POST /api/show\nread num_ctx or model_info]
    B -->|lm-studio| D[GET /api/v1/models\nread context_length]
    B -->|vllm or generic| E[GET /v1/models/model-id]

    E -->|200 OK| F["Context-window fields only\nmax_model_len OR context_length\nOR context_window OR max_input_tokens\nOR max_position_embeddings\nmax_tokens EXCLUDED"]
    F -->|value found| Z[return int ctx]
    F -->|no value| G[GET /v1/models list]
    E -->|non-200| G

    G -->|200 OK| H["Same field chain\nper matching model entry"]
    H -->|value found| Z
    H -->|not found| Y[return None]

    C --> Z
    D --> Z

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[_query_local_context_length_uncached] --> B{server_type}
    B -->|ollama| C[POST /api/show\nread num_ctx or model_info]
    B -->|lm-studio| D[GET /api/v1/models\nread context_length]
    B -->|vllm or generic| E[GET /v1/models/model-id]

    E -->|200 OK| F["Context-window fields only\nmax_model_len OR context_length\nOR context_window OR max_input_tokens\nOR max_position_embeddings\nmax_tokens EXCLUDED"]
    F -->|value found| Z[return int ctx]
    F -->|no value| G[GET /v1/models list]
    E -->|non-200| G

    G -->|200 OK| H["Same field chain\nper matching model entry"]
    H -->|value found| Z
    H -->|not found| Y[return None]

    C --> Z
    D --> Z

Loading

_{Reviews (1): Last reviewed commit: "fix(model-metadata): local ctx probe mus..." | Re-trigger Greptile}