Skip to content

fix(ai): honor an explicitly configured maxTokens - #839

Open
BILLKISHORE wants to merge 5 commits into
PrimeIntellect-ai:mainfrom
BILLKISHORE:fix/755-max-tokens-clamp
Open

fix(ai): honor an explicitly configured maxTokens#839
BILLKISHORE wants to merge 5 commits into
PrimeIntellect-ai:mainfrom
BILLKISHORE:fix/755-max-tokens-clamp

Conversation

@BILLKISHORE

@BILLKISHORE BILLKISHORE commented Aug 7, 2026

Copy link
Copy Markdown

Closes #755.

Problem

buildBaseOptions clamped every request to 32000:

maxTokens: options?.maxTokens ?? (model.maxTokens > 0 ? Math.min(model.maxTokens, 32000) : undefined),

model.maxTokens is also where user configuration lands, so a models.json entry asking for
"maxTokens": 131072 went on the wire as max_tokens: 32000, with no warning and no documented
override. This hits self-hosted and long-output setups hardest, which is where the reporter found it.

Why not just remove the clamp

The clamp is not a rare guard against bad catalog metadata. 901 of the 1162 models in
models.generated.ts that declare a maxTokens are above 32000 (128000 on 281 models, 131072 on
113), so removing it outright would change the requested output size for roughly 78% of the catalog
and the cost and latency that come with it. The default is doing real work and is worth keeping.

The bug is narrower than the clamp: an explicitly configured value should not be silently overridden
by a default.

Change

Model gains an optional maxTokensExplicit. When set, buildBaseOptions uses model.maxTokens
unchanged; otherwise the existing 32000 default applies, now named DEFAULT_MAX_OUTPUT_TOKENS.

The registry sets the flag at the three places where a maxTokens can only have come from
configuration:

  • applyModelOverride when override.maxTokens is present
  • parseModels when a models.json model definition supplies one, since the field is optional there
    and otherwise defaults to 16384
  • applyProviderConfig, where maxTokens is a required field on the programmatic provider input

Catalog models are untouched and keep the default ceiling. A per-request options.maxTokens still
takes precedence over both, as before.

Tests

  • packages/ai/test/max-tokens.test.ts covers the ceiling, explicit passthrough, an explicit value
    below the ceiling, maxTokens: 0, and per-request precedence.
  • packages/coding-agent/test/suite/regressions/755-configured-max-tokens.test.ts covers the three
    registry paths plus an untouched built-in.

Both fail without the corresponding change: the first asserts 131072 and gets 32000, the second
asserts the flag and gets undefined.

Verified with npm run check and the affected suites on Node 22.22.3.

Note

Fix maxTokens clamping to honor explicitly configured values in model registry

  • Previously, all model maxTokens values were clamped to a 32000 default ceiling, ignoring explicitly configured values from models.json or overrides.
  • Adds a maxTokensExplicit flag to the Model interface and a resolveMaxTokens helper in simple-options.ts that bypasses the clamp when the flag is set.
  • model-registry.ts now sets maxTokensExplicit: true when loading models with a configured maxTokens, both from models.json definitions and from applyModelOverride.

Macroscope summarized c7387b2.

@BILLKISHORE
BILLKISHORE force-pushed the fix/755-max-tokens-clamp branch 2 times, most recently from deb6ac4 to 0615aa7 Compare August 11, 2026 14:31
@BILLKISHORE
BILLKISHORE force-pushed the fix/755-max-tokens-clamp branch from 0615aa7 to c7387b2 Compare August 11, 2026 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Configured per-model maxTokens is silently clamped to 32000 (min(configured, 32000) in buildBaseOptions)

1 participant