[yifjiang]

Summary

Use the unambiguous --router-mo abbreviation in the worker rejection and golden parser cases. Preserve the existing rejection and real-parser equivalence assertions. The other parser-mode and captured-environment corrections are inherited from the updated public main base.

Part of #16747 and #16746. Review slice D03 of #16748. This draft targets main; its complete review diff is 4 additions plus deletions, including tests. It leads the review stack without activating decode-to-prefill reuse.

Validation

All configured pre-commit hooks passed for this exact slice on allocated SLURM. Stack-level CPU regression coverage is summarized on D31; each draft still requires its own applicable CI in the Checks tab. Historical combined-head and hardware results retain their original source scope.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[devin-ai-integration]

❌ Dynamo PR CI failed — run 41910227323 (attempt 2) on c8d275fc2f

Gate checks: ❌ backend-status-check · ✅ deploy-status-check · ❌ dynamo-status-check

Other	Jobs
dynamo-runtime	❌ 4 ✅ 3
planner	❌ 1 ✅ 4

Failure details

5 jobs failed the same way: the Run CPU-only tests (parallelized) step (pre_merge_cpu) was cut off mid-run and recorded no exit code, so Verify test stages completed failed. None of the logs contain a pytest FAILED/ERROR line or a summary line; the pytest output just stops. There are no cancelled jobs.

❌ dynamo-runtime / test / parallel cuda13.0, amd64: pre_merge_cpu cut off mid-run, no exit code recorded

Job: dynamo-runtime / test / parallel cuda13.0, amd64 · Failed step: Verify test stages completed · Logs: gh run view --job 126887825763 -R ai-dynamo/dynamo --log-failed

# last pytest output in 'Run CPU-only tests (parallelized)':
components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_recommend_cli_cases_round_trip[01-no-adapters-throughput] PASSED [ 36%]
...
##[group]Run actions/upload-artifact@...
# 'Verify test stages completed':
❌ pre_merge_cpu: reported success but recorded no exit code.
   Its script was cut off mid-run, so the result is
   unknown and cannot be treated as a pass.
##[error]A pytest stage did not record a passing result. See this step's log for the stage.

The pytest step was reported as successful, but its output stops at 36% (components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_recommend_cli_cases_round_trip[01-no-adapters-throughput] PASSED); step ran 22:03:06→22:08:50. No test failure, assertion or summary line was logged before the output stopped.

❌ planner / CPU Test cpu, arm64: pre_merge_cpu cut off mid-run, no exit code recorded

Job: planner / CPU Test cpu, arm64 · Failed step: Verify test stages completed · Logs: gh run view --job 126887827665 -R ai-dynamo/dynamo --log-failed

# last pytest output in 'Run CPU-only tests (parallelized)':
[Datadog Test Optimization] WARNING  ddtrace.testing.internal.pytest.plugin:plugin.py:1647 rerunfailures is installed alongside Datadog Test Optimization. ...
...
##[group]Run actions/upload-artifact@...
# 'Verify test stages completed':
❌ pre_merge_cpu: reported success but recorded no exit code.
   Its script was cut off mid-run, so the result is
   unknown and cannot be treated as a pass.
##[error]A pytest stage did not record a passing result. See this step's log for the stage.

The pytest step was reported as successful, but its output stops right after xdist worker startup, before any test result line; step ran 21:31:58→21:32:34 (36 s). No test failure, assertion or summary line was logged before the output stopped.

❌ dynamo-runtime / test / sequential cuda13.0, amd64: pre_merge_cpu cut off mid-run, no exit code recorded

Job: dynamo-runtime / test / sequential cuda13.0, amd64 · Failed step: Verify test stages completed · Logs: gh run view --job 126887827849 -R ai-dynamo/dynamo --log-failed

# last pytest output in 'Run CPU-only tests (parallelized)':
lib/bindings/python/tests/test_request_plane_admission_rank.py::test_a_request_keeps_the_role_its_endpoint_had_when_it_arrived[push] PASSED [ 90%]
...
##[group]Run actions/upload-artifact@...
# 'Verify test stages completed':
❌ pre_merge_cpu: reported success but recorded no exit code.
   Its script was cut off mid-run, so the result is
   unknown and cannot be treated as a pass.
##[error]A pytest stage did not record a passing result. See this step's log for the stage.

The pytest step was reported as successful, but its output stops at 90% (lib/bindings/python/tests/test_request_plane_admission_rank.py::test_a_request_keeps_the_role_its_endpoint_had_when_it_arrived[push] PASSED); step ran 21:16:59→21:28:51. No test failure, assertion or summary line was logged before the output stopped.

❌ dynamo-runtime / test / parallel cuda13.0, arm64: pre_merge_cpu cut off mid-run, no exit code recorded

Job: dynamo-runtime / test / parallel cuda13.0, arm64 · Failed step: Verify test stages completed · Logs: gh run view --job 126887829137 -R ai-dynamo/dynamo --log-failed

# last pytest output in 'Run CPU-only tests (parallelized)':
components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_recommend_cli_cases_round_trip[05-router-planner-pareto] PASSED [ 36%]
...
##[group]Run actions/upload-artifact@...
# 'Verify test stages completed':
❌ pre_merge_cpu: reported success but recorded no exit code.
   Its script was cut off mid-run, so the result is
   unknown and cannot be treated as a pass.
##[error]A pytest stage did not record a passing result. See this step's log for the stage.

The pytest step was reported as successful, but its output stops at 36% (components/src/dynamo/replay/tests/e2e/test_unified_cli_dynamo.py::test_dynamo_recommend_cli_cases_round_trip[05-router-planner-pareto] PASSED); step ran 21:17:41→21:29:05. No test failure, assertion or summary line was logged before the output stopped.

❌ dynamo-runtime / test / sequential cuda13.0, arm64: pre_merge_cpu cut off mid-run, no exit code recorded

Job: dynamo-runtime / test / sequential cuda13.0, arm64 · Failed step: Verify test stages completed · Logs: gh run view --job 126887829295 -R ai-dynamo/dynamo --log-failed

# last pytest output in 'Run CPU-only tests (parallelized)':
lib/bindings/python/tests/test_request_plane_admission_rank.py::test_the_response_that_ends_the_wait[text: a role-only delta, then its generated text, held open-push-json] PASSED [ 89%]
...
##[group]Run actions/upload-artifact@...
# 'Verify test stages completed':
❌ pre_merge_cpu: reported success but recorded no exit code.
   Its script was cut off mid-run, so the result is
   unknown and cannot be treated as a pass.
##[error]A pytest stage did not record a passing result. See this step's log for the stage.

The pytest step was reported as successful, but its output stops at 89% (lib/bindings/python/tests/test_request_plane_admission_rank.py::test_the_response_that_ends_the_wait[...-push-json] PASSED); step ran 21:17:44→21:29:05. No test failure, assertion or summary line was logged before the output stopped.

The same signature appears in all 5 jobs, on separate amd64 and arm64 runners, at different points in the suite, with no test-level error. This looks like the test script inside the job container was interrupted (runner/container exec infrastructure). A rerun of failed jobs is likely enough. If it happens again at the same spot, check the last test running in that job.

For agents

{"pr": 18020, "run_id": 41910227323, "run_attempt": 2, "head_sha": "c8d275fc2f322951d84a9878c16a821fc124c313", "failures": [{"job": "dynamo-runtime / test / parallel cuda13.0, amd64", "job_id": 126887825763, "failed_step": "Verify test stages completed", "signature": "pre_merge_cpu cut off mid-run, no exit code recorded", "tests": [], "log_cmd": "gh run view --job 126887825763 -R ai-dynamo/dynamo --log-failed"}, {"job": "planner / CPU Test cpu, arm64", "job_id": 126887827665, "failed_step": "Verify test stages completed", "signature": "pre_merge_cpu cut off mid-run, no exit code recorded", "tests": [], "log_cmd": "gh run view --job 126887827665 -R ai-dynamo/dynamo --log-failed"}, {"job": "dynamo-runtime / test / sequential cuda13.0, amd64", "job_id": 126887827849, "failed_step": "Verify test stages completed", "signature": "pre_merge_cpu cut off mid-run, no exit code recorded", "tests": [], "log_cmd": "gh run view --job 126887827849 -R ai-dynamo/dynamo --log-failed"}, {"job": "dynamo-runtime / test / parallel cuda13.0, arm64", "job_id": 126887829137, "failed_step": "Verify test stages completed", "signature": "pre_merge_cpu cut off mid-run, no exit code recorded", "tests": [], "log_cmd": "gh run view --job 126887829137 -R ai-dynamo/dynamo --log-failed"}, {"job": "dynamo-runtime / test / sequential cuda13.0, arm64", "job_id": 126887829295, "failed_step": "Verify test stages completed", "signature": "pre_merge_cpu cut off mid-run, no exit code recorded", "tests": [], "log_cmd": "gh run view --job 126887829295 -R ai-dynamo/dynamo --log-failed"}]}

_{Posted automatically by Devin for run 41910227323. Updated on every full-CI run of this PR.}

[yifjiang]

/ok to test c8d275f