[furionw]

Overview

Backport of #17586 to release/1.7.0, which is in QA. The fix keeps requested prompt and completion token IDs and logprobs, including the hidden EOS score, from being dropped, duplicated or reordered when the frontend buffers parser markers, queues reasoning, or emits an EOS-only chunk. It covers the SGLang processor and both Rust chat-parser routes (V1 and V2). This is a clean cherry-pick of a90469cd19: its added and removed lines are identical to the source commit.

Key Changes

Core logic: about 190 added lines (115 Rust, 76 Python), excluding tests, docs and comments.

• SGLang processor retains the hidden EOS score, emits the requested IDs, and truncates completion IDs at a local stop.
• V2 parser transfers each source choice's scores once and keeps source order across queued Qwen reasoning.
• V1 parser tracks scores per choice outside the jail and flushes what remains at EOF.

Cost, as measured on the source PR: copying V1 payloads added roughly 4-7% V1 CPU in processor fixtures. That is not a serving-latency or throughput measurement.

Validation

Each behavior is pinned by a regression test added in the source PR:

Case	Result	Test
SGLang, EOS-terminated generation of 48 tokens	every token keeps its score and ID; only the EOS text stays hidden	test_hidden_eos_keeps_score_and_requested_ids
V2, alternating raw input and queued Qwen reasoning	scores keep source order A, B, C, D, E	qwen_queued_reasoning_keeps_source_logprob_order
V1, EOS swallowed by the jail	its score arrives on a final empty-delta choice after the finish	legacy_jail_delivers_swallowed_eos_score_after_finish
Responses, score-only chunk after the finish	no new events; exactly one completed output item	post_finish_score_only_choice_emits_no_events

On this branch: the three Rust tests above pass, along with the rest of the affected Rust modules (648 passed, 0 failed, 12 ignored; built with --no-default-features). The SGLang Python test was not run locally. Release-branch CI for the current head is pending.

Issues and Ongoing Work

Refs DGH-783

• Source: fix(frontend): preserve chat token IDs and EOS logprobs #17586, merged to main as a90469cd19. Target: release/1.7.0.
• Existing work: fix(frontend): gate SGLang tool-call parsing on delimiter token ids #17467 is open and changes SGLang tool-marker gating in the same preprocessing file. It is not a prerequisite for this backport.

Pull Request Checklist

• I used Dynamo's /pr-checklist skill.

[github-actions]

🌿 Fern Docs Preview: https://nvidia-preview-01a1a7a0-dcfe-75d8-8f2a-2578dcfc185b.docs.buildwithfern.com/dynamo/dev

[devin-ai-integration]

❌ Dynamo PR CI failed — run 41878089492 (attempt 1) on 10ad2aabb4

Gate checks: ✅ backend-status-check · ✅ deploy-status-check · ❌ dynamo-status-check

Other	Jobs
dynamo-runtime	❌ 1 ✅ 6

Failure details

1 job failed: the arm64 CPU-only pytest stage stopped partway through tests/router/test_router_e2e_with_mockers.py::test_router_decisions_disagg_router_ais and never recorded an exit code, so the stage verification step failed it. No test reported FAILED or ERROR.

❌ dynamo-runtime / test / sequential cuda13.0, arm64: pre_merge_cpu cut off mid-run (no exit code recorded)

Job: dynamo-runtime / test / sequential cuda13.0, arm64 · Failed step: Verify test stages completed · Logs: gh run view --job 126748492979 -R ai-dynamo/dynamo --log-failed

tests/router/test_router_e2e_with_mockers.py::test_disagg_per_role_session_affinity PASSED [ 95%]
WARN dynamo_runtime::transports::etcd::connector: Reconnecting to ETCD cluster at: ["http://localhost:2624"]
WARN dynamo_runtime::transports::etcd::lease: Failed to establish keep-alive stream ... tcp connect error, 127.0.0.1:2624 ... Connection refused
tests/router/test_router_e2e_with_mockers.py::test_router_decisions_disagg_router_ais
##[group]Run actions/upload-artifact@...            <- pytest output ends here, no session summary
##[warning]No files were found with the provided path: test-results/pytest_test_report_*.xml. No artifacts will be uploaded.
❌ pre_merge_cpu: reported success but recorded no exit code.
   Its script was cut off mid-run, so the result is
   unknown and cannot be treated as a pass.
##[error]A pytest stage did not record a passing result. See this step's log for the stage.

Step Run CPU-only tests (parallelized) ran for about 30 minutes (17:44–18:14 UTC), below its 45-minute timeout. It ended while test_router_decisions_disagg_router_ais was running (95% through), shortly after etcd connection-refused warnings. Pytest printed no summary and wrote no JUnit XML, but the step still reported success, so Verify test stages completed failed the job. The process seems to have been cut off, not failed by an assertion, so it looks like a runner/container interruption and a rerun of failed jobs is likely enough. If it happens again at the same test, check that test for a hang or crash.

For agents

{"pr": 18085, "run_id": 41878089492, "run_attempt": 1, "head_sha": "10ad2aabb4d0f5dc1b5b7a751f72eecca90717b6",
 "failures": [{"job": "dynamo-runtime / test / sequential cuda13.0, arm64", "job_id": 126748492979, "failed_step": "Verify test stages completed", "signature": "pre_merge_cpu: reported success but recorded no exit code (pytest cut off mid-run)", "tests": ["tests/router/test_router_e2e_with_mockers.py::test_router_decisions_disagg_router_ais"], "log_cmd": "gh run view --job 126748492979 -R ai-dynamo/dynamo --log-failed"}]}

_{Posted automatically by Devin for run 41878089492. Updated on every full-CI run of this PR.}