Skip to content

[Frontend] Extract parse and assembly out of batch chat derender - #60073

Merged
sfeng33 merged 2 commits into
vllm-project:mainfrom
hickeyma:move-parse-assembly
Oct 5, 2026
Merged

sfeng33 merged 2 commits into
vllm-project:mainfrom
hickeyma:move-parse-assembly

Conversation

@hickeyma

@hickeyma hickeyma commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Overview

Moves the parse and message assembly steps out of _derender_chat into a _parse_and_assemble helper. It's prep work for the inline derender output level in #56851 (Phase 2). It means that it can call this code instead of copying it.

Detais

  • New OnlineDerenderer._parse_and_assemble() takes decoded text and returns the assistant ChatMessage. It covers building the parser, parsing, dropping reasoning when include_reasoning is off, tool call IDs and the empty content rule for named/required tool_choice.
  • chat_template_kwargs is now resolved once per request instead of once per choice. The streaming parser path uses the same _resolve_chat_template_kwargs() helper, so that block isn't duplicated anymore.
  • The decode call is pulled out of the parser/no-parser branches since both made the same call.
  • No behavior change.

Validation

  • pytest tests/entrypoints/scale_out/derender/test_derender.py -v
  • `pytest tests/entrypoints/scale_out/derender/test_derender_stream.py -v
  • pytest tests/entrypoints/scale_out/derender/test_derender_parity.py -v

There are no new tests. The existing derender unit and parity tests already cover this path and none of them needed changes.


Pull Request Checklist
  • I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

  • AI assistance was used during the creation of this PR.

  • Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

  • Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

  • Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

  • Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

Move the parser and message building steps in _derender_chat into
_parse_and_assemble so the inline derender level in vllm-project#56851 can reuse
them instead of copying them. Template kwargs are now resolved once
per request rather than per choice and the streaming path shares the
same helper for that. The decode call is also pulled out of the
if/else since both branches made the same call.

No behavior change.

Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
@hickeyma
hickeyma marked this pull request as ready for review October 5, 2026 15:39

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@sfeng33 sfeng33 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the work, LGTM!

@sfeng33

sfeng33 commented Oct 5, 2026

Copy link
Copy Markdown
Member

/ci run

@sfeng33
sfeng33 enabled auto-merge (squash) October 5, 2026 19:37
@github-actions github-actions Bot added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 5, 2026
@sfeng33

sfeng33 commented Oct 5, 2026

Copy link
Copy Markdown
Member

/ci run --allow-stale

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown

✅ Triggered Buildkite CI #92952 for commit f2cdd21073d2.

⚠️ This PR is 1 commit behind upstream main. Running CI at your own risk because --allow-stale was requested; outdated CI configuration may cause failures. Before merging, merge or rebase onto the latest main, then rerun /ci run on the latest PR commit.

@vllm-agent

Copy link
Copy Markdown
Collaborator

CI selector (shadow): 2 test steps (5 jobs) instead of 35 (48 jobs)

Shadow mode: this changes nothing about what CI runs. It shows what the evidence-based selector would pick for this PR, next to today's rules. How it works.

Feedback welcome: reply here if it would skip a step this change needs, or runs something unrelated.

steps (jobs) Today's rules Selector Would skip Would add
NVIDIA, CPU and others 35 (48) 2 (5) 34 (44) 1 (1)
AMD mirrors 30 (38) 2 (5) 29 (34) 1 (1)
Selector would run (2)
  • entrypoints-integration-api-server ×4
  • scale-out-ec-e2e-2-gpus
Would skip (today's rules run them) (34)
  • basic-correctness ×2
  • basic-correctness-cpu-offload
  • basic-correctness-cumem
  • basic-correctness-prefetch-offload
  • basic-correctness-sleep-mode
  • basic-models-test-other-cpu
  • basic-models-tests-initialization
  • basic-models-tests-other
  • benchmarks-cli-test
  • cpu-language-generation-and-pooling-model-tests ×3
  • cpu-multimodal-config
  • cpu-params-env-tokenizers-parser
  • cpu-reasoning-renderers
  • cpu-tool-parsers
  • entrypoints-integration-api-server-generate
  • entrypoints-integration-api-server-openai-chat_completion
  • entrypoints-integration-api-server-openai-completion
  • entrypoints-integration-llm
  • entrypoints-integration-multimodal
  • entrypoints-integration-pooling
  • entrypoints-integration-responses-api
  • entrypoints-integration-speech_to_text
  • kernels-fla-ops-test-b200
  • kernels-mhc-test-b200
  • kernels-root-misc-test-b200
  • language-models-tests-granite-l4-compatibility
  • language-models-tests-hybrid ×2
  • language-models-tests-standard
  • multi-modal-models-standard-1-qwen2
  • multi-modal-models-standard-2-qwen3-gemma
  • multi-modal-models-standard-3-llava-qwen2-vl
  • multi-modal-models-standard-4-other-whisper
  • multi-modal-processor ×4
  • multi-modal-processor-cpu ×4
Would add (today's rules do not run them) (1)
  • scale-out-ec-e2e-2-gpus (Python record)
AMD mirrors: would skip (29)
  • basic-correctness ×2
  • basic-correctness-cpu-offload
  • basic-correctness-cumem
  • basic-correctness-prefetch-offload
  • basic-correctness-sleep-mode
  • basic-models-tests-initialization
  • basic-models-tests-other
  • benchmarks-cli-test
  • entrypoints-integration-api-server-generate
  • entrypoints-integration-api-server-openai-chat_completion
  • entrypoints-integration-api-server-openai-completion
  • entrypoints-integration-llm
  • entrypoints-integration-multimodal
  • entrypoints-integration-pooling
  • entrypoints-integration-responses-api
  • entrypoints-integration-speech_to_text
  • kernels-fla-ops-test-b200
  • kernels-mhc-test-b200
  • kernels-root-misc-test-b200
  • language-models-tests-granite-l4-compatibility
  • language-models-tests-hybrid ×2
  • language-models-tests-standard
  • multi-modal-models-standard-1-qwen2
  • multi-modal-models-standard-2-qwen3-gemma
  • multi-modal-models-standard-3-llava-qwen2-vl
  • multi-modal-models-standard-4-other-whisper
  • multi-modal-processor ×4
  • platform-tests
  • pytorch-compilation-passes-unit-tests
AMD mirrors: would add (1)
  • scale-out-ec-e2e-2-gpus (Python record)

1 changed files · base 74c5cbcd7b · head f2cdd21073 · Python record: build 92838 at 92044241a0 · kernel record: table 9204424 (build 92838), map 9204424 · not counted: 10 build steps, 5 A100 steps the generator no longer emits, 0 optional steps the selector would also run

@sfeng33
sfeng33 merged commit 3fb1cd7 into vllm-project:main Oct 5, 2026
88 checks passed
@hickeyma
hickeyma deleted the move-parse-assembly branch October 6, 2026 08:07
@hickeyma hickeyma mentioned this pull request Oct 6, 2026
6 tasks done
michael-gpu pushed a commit to practical-inference/vllm-ampere-extended that referenced this pull request Oct 20, 2026
…m-project#60073)

Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants