Repository navigation
[Frontend] Extract parse and assembly out of batch chat derender - #60073
Conversation
Move the parser and message building steps in _derender_chat into _parse_and_assemble so the inline derender level in vllm-project#56851 can reuse them instead of copying them. Template kwargs are now resolved once per request rather than per choice and the streaming path shares the same helper for that. The decode call is also pulled out of the if/else since both branches made the same call. No behavior change. Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
/ci run |
|
/ci run --allow-stale |
|
✅ Triggered Buildkite CI #92952 for commit
|
CI selector (shadow): 2 test steps (5 jobs) instead of 35 (48 jobs)Shadow mode: this changes nothing about what CI runs. It shows what the evidence-based selector would pick for this PR, next to today's rules. How it works. Feedback welcome: reply here if it would skip a step this change needs, or runs something unrelated.
Selector would run (2)
Would skip (today's rules run them) (34)
Would add (today's rules do not run them) (1)
AMD mirrors: would skip (29)
AMD mirrors: would add (1)
1 changed files · base |
…m-project#60073) Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com> Co-authored-by: Flora Feng <4florafeng@gmail.com>
Overview
Moves the parse and message assembly steps out of
_derender_chatinto a_parse_and_assemblehelper. It's prep work for the inlinederenderoutput level in #56851 (Phase 2). It means that it can call this code instead of copying it.Detais
OnlineDerenderer._parse_and_assemble()takes decoded text and returns the assistantChatMessage. It covers building the parser, parsing, dropping reasoning wheninclude_reasoningis off, tool call IDs and the empty content rule for named/requiredtool_choice.chat_template_kwargsis now resolved once per request instead of once per choice. The streaming parser path uses the same_resolve_chat_template_kwargs()helper, so that block isn't duplicated anymore.Validation
pytest tests/entrypoints/scale_out/derender/test_derender.py -vpytest tests/entrypoints/scale_out/derender/test_derender_parity.py -vThere are no new tests. The existing derender unit and parity tests already cover this path and none of them needed changes.
Pull Request Checklist
I used vLLM's
/pr-checklistskill. (Mandatory for agents, optional for humans).AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.