[None][fix] Chat Completions: accept the OpenAI store field by JunyiXu-nv · Pull Request #20864 · NVIDIA/TensorRT-LLM

JunyiXu-nv · 2026-10-23T04:57:07Z

Dev Engineer Review

ChatCompletionRequest now accepts store: Optional[bool] = None for OpenAI compatibility. The field is ignored because the server does not retain chat completions.
The change preserves rejection of unrelated unknown fields. It does not change the behavior of other request fields.
Test execution results are unavailable.

QA Engineer Review

Added tests/unittest/llmapi/apps/test_openai_protocol_chat_store.py and registered it in the CPU CI list.
Tests cover accepted values, the None default, JSON parsing, boolean-like coercion, invalid values, unchanged other fields, and rejection of unknown fields.
Coverage verdict: sufficient for the new field’s validation and unknown-field behavior. Test execution results are unavailable.

Per-File QA Perspective

tensorrt_llm/serve/openai_protocol.py: Verify that chat completion requests accept optional store, default it to None, and do not change completion storage behavior.
tests/unittest/llmapi/apps/test_openai_protocol_chat_store.py: Covers field validation, JSON parsing, coercion, and rejection behavior. The file is listed in test-db/l0_cpu.yml.
tests/integration/test_lists/test-db/l0_cpu.yml: Registers the new unit test for CPU CI.
tests/unittest/api_stability/references/trtllm_serve_api.yaml: Records store in the API stability reference. Verify that the reference remains aligned with the request model.

Description

ChatCompletionRequest forbids unknown fields (extra="forbid"), so a request that sets OpenAI's standard store field fails validation with HTTP 400. The pi coding agent sends store: false on every Chat Completions request unless its model config sets compat.supportsStore: false, so with default settings every pi turn against trtllm-serve is rejected before it reaches the model.

This PR accepts store as Optional[bool] = None and leaves it unused, like user and parallel_tool_calls: the server keeps no copy of a chat completion, so there is nothing to store. Other unknown fields are still rejected.

tensorrt_llm/serve/openai_protocol.py: the field, where the OpenAI reference orders it.
tests/unittest/api_stability/references/trtllm_serve_api.yaml: its reference row (kind: openai, status: stable).

#20368 adds prompt_cache_key the same way; the two PRs do not touch the same lines.

Test Coverage

New tests/unittest/llmapi/apps/test_openai_protocol_chat_store.py (CPU only), registered in l0_cpu.yml: store accepted as a keyword and from JSON, default None, non-boolean values rejected, other fields unchanged by it, unknown fields still rejected. On main, 6 of its 8 tests fail; the 2 that pass are the negative cases, which hold there too.
tests/unittest/api_stability/test_serve_api.py covers the reference row.
End to end: an A/B on one GB300 node with Qwen3-30B-A3B and the pi coding agent 1.4.0: main at 9bb7931 against the same main with this change (plus the other fixes from the same run), both from the same precompiled CI build, so they differ only in Python. pi over Chat Completions with default settings: on main every request was answered with HTTP 400 (4 of 4 sessions per mode); with the change all were served and the trtllm-serve-model-harness validator found no conformance violation. A replay of 4 recorded pi requests carrying store: 400 on main, 200 with the change.

PR Checklist

PR description clearly explains what and why.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

🤖 Generated with Claude Code

ChatCompletionRequest forbids unknown fields, so a request that sets the standard OpenAI `store` field failed validation with HTTP 400. The pi coding agent sends `store: false` on every Chat Completions request by default, so each of its turns was rejected before reaching the model. Accept the field and leave it unused, like `user` and `parallel_tool_calls`: the server keeps no copy of a chat completion, so there is nothing to store. Other unknown fields are still rejected. Signed-off-by: JunyiXu-nv <219237550+JunyiXu-nv@users.noreply.github.com>

coderabbitai · 2026-10-23T05:00:17Z

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

Run configuration

Source mode: Auto
Repository: NVIDIA/TensorRT-LLM
Branch: main
File: .coderabbit.yaml
Commit: b7d6e62
Layers: Repository YAML
Review profile: CHILL
Plan: Enterprise
Run ID: 807f4622-6972-4c58-a9a2-354abb6295ed

Commits

Reviewing files that changed from the base of the PR and between 9ee328b and bd133a4.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

No actionable comments were generated in the recent review.

Recent review info

Run configuration

Source mode: Auto
Repository: NVIDIA/TensorRT-LLM
Branch: main
File: .coderabbit.yaml
Commit: ecae3c9
Layers: Repository YAML
Review profile: CHILL
Plan: Enterprise
Run ID: 0e444f6e-68f9-4dae-ad49-6ac5cd70ff07

Commits

Reviewing files that changed from the base of the PR and between fd33783 and 611ee1b.

Files selected for processing: 1

tests/integration/test_lists/test-db/l0_cpu.yml

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

Walkthrough

ChatCompletionRequest now accepts an optional store field for compatibility and ignores it. The API stability reference records the field. CPU tests cover accepted values, parsing, and validation.

Changes

Chat completion store field

Layer / File(s)	Summary
Request field and API stability reference `tensorrt_llm/serve/openai_protocol.py`, `tests/unittest/api_stability/references/trtllm_serve_api.yaml`	`ChatCompletionRequest` adds an optional `store` boolean field. The API stability reference lists it as stable, optional, and null by default.
Request validation and CPU test registration `tests/unittest/llmapi/apps/test_openai_protocol_chat_store.py`, `tests/integration/test_lists/test-db/l0_cpu.yml`	Tests cover accepted values, omission, JSON parsing, invalid inputs, and model-dump behavior. The CPU integration test list includes the new test module.

Priority: Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Merge Risk: Minimal · up to 611ee

No actionable merge-blocking risk is identified in the reviewed change.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 2 files. (1 skipped: 1 …	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Description check		The description clearly explains the problem, solution, affected files, test coverage, and validation results. It includes the required Description, Test Coverage, and PR Checklist sections.
Title check		The title follows the required format and clearly identifies the fix: accepting the OpenAI store field for Chat Completions.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 2 files. (1 skipped: 1 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches

🧪 Generate unit tests (beta)

Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

zhaoyangwang-nvidia

LGTM

JunyiXu-nv · 2026-10-23T05:53:22Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-10-23T05:59:46Z

PR_Github #80073 [ run ] triggered by Bot. Commit: 8f26eba Link to invocation

tensorrt-cicd · 2026-10-23T11:35:22Z

PR_Github #80073 [ run ] completed with state FAILURE. Commit: 8f26eba
/LLM/main/L0_MergeRequest_PR pipeline #65897 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Picks up the waive of unittest/executor/test_forward_error_survival.py (NVIDIA#20895, nvbug 6981910), which fails on this branch's base in the CPU stages independently of this change. Signed-off-by: JunyiXu-nv <219237550+JunyiXu-nv@users.noreply.github.com>

JunyiXu-nv · 2026-10-23T11:40:17Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-10-23T11:47:05Z

PR_Github #80156 [ run ] triggered by Bot. Commit: b7969fe Link to invocation

brnguyen2

Approving — the comments below are optional touch-ups, not blockers.

This handles store the same way as the existing accepted-but-ignored fields (parallel_tool_calls, user), and extra="forbid" still rejects other unknown keys, which test_unknown_fields_remain_rejected checks.

Minor points:

The title uses [None] for a user-visible bug fix (requests from a common client returned HTTP 400). If an NVBug or JIRA ticket tracks the pi-agent compatibility work, please cite it.
store is a no-op here, but it has real meaning in the Responses API (ResponsesRequest.store defaults to True and controls persistence when enable_store is set). The code comment already says chat completions are never stored. A short note in the serve docs listing which OpenAI chat fields are accepted but ignored would help users who expect store: true to make a completion retrievable. This is optional.

github-actions · 2026-10-23T16:24:01Z

Automatically added "ci: full pre-merge approved" because this PR has satisfied the required GitHub review approvals. Unresolved review conversations and other required checks remain independent merge requirements.

tensorrt-cicd · 2026-10-23T16:58:21Z

PR_Github #80156 [ run ] completed with state SUCCESS. Commit: b7969fe
/LLM/main/L0_MergeRequest_PR pipeline #65966 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

JunyiXu-nv · 2026-10-23T16:59:06Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-10-23T17:05:44Z

PR_Github #80216 [ run ] triggered by Bot. Commit: b7969fe Link to invocation

tensorrt-cicd · 2026-10-23T19:20:27Z

PR_Github #80216 [ run ] completed with state SUCCESS. Commit: b7969fe
/LLM/main/L0_MergeRequest_PR pipeline #66019 completed with status: 'SUCCESS'

CI Report

Link to invocation

… rejects The request schema is not strict, so Pydantic coerces boolean-like strings and integers for store. A test now records that, and the rejection test is renamed for what it checks: a value that is not boolean-like. Signed-off-by: JunyiXu-nv <219237550+JunyiXu-nv@users.noreply.github.com>

JunyiXu-nv · 2026-10-23T19:24:04Z

@brnguyen2 Thanks for the review. The inline point about lax coercion is addressed in fd33783. On the two body points:

Title: no NVBug or JIRA ticket tracks this. It came from running the pi coding agent against trtllm-serve, so the PR keeps [None].
A docs note listing the chat fields that are accepted but ignored: I would rather keep this PR to the schema fix. Such a list would also cover parallel_tool_calls and user, so it fits a separate docs change. The field comment states that a chat completion is never stored.

JunyiXu-nv · 2026-10-23T19:24:08Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-10-23T19:31:43Z

PR_Github #80244 [ run ] triggered by Bot. Commit: fd33783 Link to invocation

tensorrt-cicd · 2026-10-24T00:19:43Z

PR_Github #80244 [ run ] completed with state FAILURE. Commit: fd33783
/LLM/main/L0_MergeRequest_PR pipeline #66042 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

JunyiXu-nv · 2026-10-24T00:20:42Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-10-24T00:28:12Z

PR_Github #80290 [ run ] triggered by Bot. Commit: fd33783 Link to invocation

tensorrt-cicd · 2026-10-24T03:41:10Z

PR_Github #80290 [ run ] completed with state SUCCESS. Commit: fd33783
/LLM/main/L0_MergeRequest_PR pipeline #66084 completed with status: 'SUCCESS'

CI Report

Link to invocation

Resolve tests/integration/test_lists/test-db/l0_cpu.yml: keep both new entries (main's test_openai_chat_multimodal_prompt.py and this branch's test_openai_protocol_chat_store.py). Signed-off-by: JunyiXu-nv <219237550+JunyiXu-nv@users.noreply.github.com>

JunyiXu-nv · 2026-10-24T03:44:24Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-10-24T03:51:19Z

PR_Github #80314 [ run ] triggered by Bot. Commit: 611ee1b Link to invocation

tensorrt-cicd · 2026-10-24T08:47:56Z

PR_Github #80314 [ run ] completed with state FAILURE. Commit: 611ee1b
/LLM/main/L0_MergeRequest_PR pipeline #66103 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Picks up the waive of unittest/_torch/modeling_v2/models/glm_moe_dsa/test_routing.py (NVIDIA#20935, nvbug 6986044), which the previous main merge brought in failing on both CPU stages independently of this change. Signed-off-by: JunyiXu-nv <219237550+JunyiXu-nv@users.noreply.github.com>

JunyiXu-nv · 2026-10-24T08:49:11Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-10-24T08:55:28Z

PR_Github #80349 [ run ] triggered by Bot. Commit: 9ee328b Link to invocation

tensorrt-cicd · 2026-10-24T14:42:15Z

PR_Github #80349 [ run ] completed with state FAILURE. Commit: 9ee328b
/LLM/main/L0_MergeRequest_PR pipeline #66131 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Picks up main's waives of the LoRA tests that NVIDIA#20774 broke (test_lora_ownership.py and the test_resource_manager.py PEFT reload tests, Signed-off-by: JunyiXu-nv <219237550+JunyiXu-nv@users.noreply.github.com> NVIDIA#20953), which fail independently of this change.

JunyiXu-nv · 2026-10-25T05:54:42Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-10-25T06:02:45Z

PR_Github #80434 [ run ] triggered by Bot. Commit: bd133a4 Link to invocation

tensorrt-cicd · 2026-10-25T10:34:06Z

PR_Github #80434 [ run ] completed with state FAILURE. Commit: bd133a4
/LLM/main/L0_MergeRequest_PR pipeline #66205 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

JunyiXu-nv · 2026-10-25T10:35:10Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-10-25T10:43:17Z

PR_Github #80458 [ run ] triggered by Bot. Commit: bd133a4 Link to invocation

tensorrt-cicd · 2026-10-25T12:24:14Z

PR_Github #80458 [ run ] completed with state SUCCESS. Commit: bd133a4
/LLM/main/L0_MergeRequest_PR pipeline #66229 completed with status: 'SUCCESS'

CI Report

Link to invocation

JunyiXu-nv requested review from a team as code owners October 23, 2026 04:57

JunyiXu-nv requested review from QiJune, YihuiLu512, brnguyen2, niukuo, yuanjingx87 and zhaoyangwang-nvidia October 23, 2026 04:57

github-actions Bot assigned JunyiXu-nv Oct 23, 2026

JunyiXu-nv added the api-compatible Accepted LLM API contract change that is backwards-compatible label Oct 23, 2026

zhaoyangwang-nvidia approved these changes Oct 23, 2026

View reviewed changes

This was referenced Oct 23, 2026

[None][fix] DeepSeek DSML tool parsers: the layout newlines around tool calls are not content #20873

Merged

[None][fix] Anthropic Messages: no whitespace-only text blocks #20874

Merged

BowenFu approved these changes Oct 23, 2026

View reviewed changes

brnguyen2 approved these changes Oct 23, 2026

View reviewed changes

Comment thread tests/unittest/llmapi/apps/test_openai_protocol_chat_store.py Resolved

trtllm-agent added the ci: full pre-merge approved label Oct 23, 2026

Sonica-B mentioned this pull request Oct 25, 2026

[#20962][fix] Accept the OpenAI safety_identifier field in Chat Completions requests #20963

Open

1 task done

coderabbitai Bot mentioned this pull request Oct 25, 2026

[#20367][fix] Accept prompt_cache_key in Chat Completions requests #20368

Open

6 tasks done

JunyiXu-nv merged commit c4dd49d into NVIDIA:main Oct 26, 2026
7 checks passed

coderabbitai Bot mentioned this pull request Oct 28, 2026

[None][fix] Chat Completions: accept service_tier and client_metadata #21154

Merged

1 task done

Conversation

JunyiXu-nv commented Oct 23, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dev Engineer Review

QA Engineer Review

Per-File QA Perspective

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

coderabbitai Bot commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Walkthrough

Changes

Uh oh!

zhaoyangwang-nvidia left a comment

Choose a reason for hiding this comment

Uh oh!

JunyiXu-nv commented Oct 23, 2026

Uh oh!

tensorrt-cicd commented Oct 23, 2026

Uh oh!

tensorrt-cicd commented Oct 23, 2026

Uh oh!

JunyiXu-nv commented Oct 23, 2026

Uh oh!

tensorrt-cicd commented Oct 23, 2026

Uh oh!

brnguyen2 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented Oct 23, 2026

Uh oh!

tensorrt-cicd commented Oct 23, 2026

Uh oh!

JunyiXu-nv commented Oct 23, 2026

Uh oh!

tensorrt-cicd commented Oct 23, 2026

Uh oh!

tensorrt-cicd commented Oct 23, 2026

Uh oh!

JunyiXu-nv commented Oct 23, 2026

Uh oh!

JunyiXu-nv commented Oct 23, 2026

Uh oh!

tensorrt-cicd commented Oct 23, 2026

Uh oh!

tensorrt-cicd commented Oct 24, 2026

Uh oh!

JunyiXu-nv commented Oct 24, 2026

Uh oh!

tensorrt-cicd commented Oct 24, 2026

Uh oh!

tensorrt-cicd commented Oct 24, 2026

Uh oh!

JunyiXu-nv commented Oct 24, 2026

Uh oh!

tensorrt-cicd commented Oct 24, 2026

Uh oh!

tensorrt-cicd commented Oct 24, 2026

Uh oh!

JunyiXu-nv commented Oct 24, 2026

Uh oh!

tensorrt-cicd commented Oct 24, 2026

Uh oh!

tensorrt-cicd commented Oct 24, 2026

Uh oh!

JunyiXu-nv commented Oct 25, 2026

Uh oh!

tensorrt-cicd commented Oct 25, 2026

Uh oh!

tensorrt-cicd commented Oct 25, 2026

Uh oh!

JunyiXu-nv commented Oct 25, 2026

Uh oh!

tensorrt-cicd commented Oct 25, 2026

Uh oh!

JunyiXu-nv commented Oct 23, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Oct 23, 2026 •

edited

Loading