[Bug] gpt-oss-20b WebGPU advertises 131072 context but is limited to 8192 tokens at runtime

[ken16anjo]

Describe the issue

The Foundry Local model gpt-oss-20b-generic-gpu:1 advertises a context length of 131072 tokens, but the actual WebGPU runtime is limited to 8192 total tokens.

This is reproducible and the model package itself contains conflicting settings:

• config.json
  • max_position_embeddings: 131072
• genai_config.json
  • context_length: 131072
  • max_length: 8192

At runtime, a request with 9196 input tokens and 2048 requested output tokens fails with:

This request requires 11244 total tokens (9196 input + 2048 output), which exceeds the model's maximum context length of 8192 tokens.

The exception originates from:

Microsoft.Neutron.OpenAI.Provider.OnnxChatGenerator.ValidateMaxTokens

GPU execution itself works correctly for short prompts using the same model and WebGPU backend, so this appears to be specifically a context-length/configuration issue rather than a general GPU execution failure.

For our use case, we need to process full meeting transcripts in one pass. An 8192-token limit is insufficient even for a 12,671-character Japanese meeting transcript.

Please clarify whether the 8192-token limit is intentional for the WebGPU variant or whether this is a packaging/runtime configuration issue.

If the 8192-token limit is intentional, the catalog metadata should reflect the actual supported context length.

If it is not intentional, we would like the GPU variant to support a substantially longer context window, ideally consistent with the advertised 131072-token context length.

To reproduce

1. Install Foundry Local 0.10.2 and foundry-local-sdk-winml 1.2.4.

2. Register the available execution providers using:

   download_and_register_eps()

3. Confirm that the following execution providers are registered:

   • WebGpuExecutionProvider
   • OpenVINOExecutionProvider

4. Refresh the Foundry Local model catalog.

5. Resolve the model:

   gpt-oss-20b-generic-gpu:1

6. Confirm that the SDK reports:

   • Device: GPU
   • Execution Provider: WebGpuExecutionProvider
   • context_length: 131072

7. Download and load the model.

8. Inspect the downloaded model package:

   ~/.gpt_oss_minutes_webgpu/cache/models/Microsoft/gpt-oss-20b-generic-gpu-1/v1/

9. Relevant model configuration values are:

   config.json

   • max_position_embeddings: 131072

   genai_config.json

   • context_length: 131072
   • max_length: 8192

10. Submit a request containing a 12,671-character Japanese meeting transcript with max_tokens=2048.

11. Foundry Local calculates:

• Input tokens: 9196
• Requested output tokens: 2048
• Total: 11244

12. The request fails with:

This request requires 11244 total tokens (9196 input + 2048 output), which exceeds the model's maximum context length of 8192 tokens.

13. As a control test, a short prompt with the same GPU model succeeds:

Prompt:
Reply with exactly: GPU_OK

Result:
GPU_OK

Inference time:
approximately 3.7 seconds.

Urgency

High.

This issue blocks evaluation of the GPT-OSS GPU variant for long-context workloads.

The current 8192-token runtime limit prevents processing documents that should fit within the advertised 131072-token context window.

A fix or clarification is important because the observed runtime behavior does not match the published model metadata.

Platform and architecture

Windows X64

OS Version

Windows 11, 10.0.26200

Installation type

Released package/binary

Foundry Local version

CLI 0.10.2 foundry-local-sdk-winml 1.2.4

API or surface area

Python SDK

Hardware acceleration/backend

WebGPU/CUDA

Backend/runtime version

WebGpuExecutionProvider Intel Graphics driver 32.0.101.8860 foundry-local-sdk-winml 1.2.4 onnxruntime-core 1.26.0 onnxruntime-genai-core 0.14.1; CPU: Intel Core Ultra 7 356H GPU: Intel Graphics GPU driver: 32.0.101.8860 RAM: 64 GB