Skip to content

fix(runtime): surface invalid generated token failures - #8

Merged
kanadaj merged 4 commits into
kanadaj:mainfrom
ktsaou:fix/invalid-token-failure
Sep 14, 2026
Merged

kanadaj merged 4 commits into
kanadaj:mainfrom
ktsaou:fix/invalid-token-failure

Conversation

@ktsaou

@ktsaou ktsaou commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

The scheduler currently converts an out-of-vocabulary generated token into an ordinary stop, and a speculative overrun can further hide it as a length finish. Clients can therefore receive a successful response after an engine-level token failure. This change reports the condition as an HTTP 500 InvalidTokenError, excludes the faulty token, and carries the failure through Chat, Anthropic Messages, and Responses clients.

The three-layer scope is required:

  • schedule_batch.py produces an abort and prevents the output cap from replacing it.
  • Chat recognizes the integer status produced by finish-reason serialization, emits the error, emits [DONE], and terminates.
  • Responses maps status-bearing aborts and error finishes to failed with a server_error payload and response.failed; graceful aborts without a status remain cancelled.

The publishable profile keeps the existing Responses runtime snapshot unchanged. Patch 0019 and its post-patch source snapshot are independently attested, and the Dockerfile combines them with the existing API compatibility files.

Validation:

  • Exact unpatched base: 11 methods reproduce 20 failures and 2 errors.
  • Patched exact-base container: all 11 runtime methods pass.
  • Package and attestation suite: 84/84 pass.
  • Exact-image reconstruction: patches 0015, 0016, and 0019 apply cleanly; all 4,392 source records match, followed by a second final-tree verification.
  • Python compilation, shell syntax, and patch whitespace checks pass.

This is CPU-tested only. No image was built, published, or deployed. The change overlaps serving_responses.py in #5 and serving_chat.py in #6, so it will need a mechanical rebase if either lands first.

@kanadaj
kanadaj force-pushed the fix/invalid-token-failure branch from 4059ace to fae68fb Compare September 14, 2026 14:48
@kanadaj
kanadaj merged commit facd7be into kanadaj:main Sep 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants