[TRTLLM-13696][test] Part2.1: Migrate CPU only tests - runtime by tongyuantongyu · Pull Request #16498 · NVIDIA/TensorRT-LLM

tongyuantongyu · 2026-07-16T11:51:30Z

Dev Engineer Review

Updated CI pytest argument construction so --unittest-markexpr for stages whose stageName starts with CPU- now targets cpu_only (instead of cpu_only and not disabled); non-CPU-* stages still use not cpu_only.
Adjusted CPU stage mappings in jenkins/L0_Test.groovy so CPU-Generic-x86-1 and CPU-Generic-arm-1 use the l0_cpu test-db context (replacing l0_cpu_x86 / l0_cpu_arm).
Fixed test_unittests_v2 mark-expression handling in tests/integration/defs/test_unittests.py:
- If a per-case -m <expr> already exists, it is rewritten to (<case_expr>) and (<unittest_markexpr>).
- Otherwise, -m <unittest_markexpr> is appended to the per-case pytest arg_list.
Removed CUDA compute-capability-dependent dtype normalization/validation from BaseLlmArgs in tensorrt_llm/llmapi/llm_args.py by deleting the validate_dtype validator that previously rewrote dtype="auto"→"float16" on pre-SM80 CUDA and rejected dtype="bfloat16" on those devices.
Migrated the first runtime/unit-test wave to CPU-only execution via pytest.mark.cpu_only (mostly module-level pytestmark, plus a few decorator-level marks) and small determinism/compatibility tweaks where needed (e.g., async URL fetching switched to asyncio.run(), MPI/tiktoken cache fixtures added/adjusted for CPU determinism).
Updated tests/README.md to clarify CPU-only CI guidance and the GPU-independent unit-test marking approach, and to improve the CI-bridge documentation (including updated wording and examples).

Key risk areas to review:

Verify that every cpu_only-marked test genuinely avoids CUDA/GPU-only code paths and doesn’t rely on prior GPU-dependent dtype rewrite/rejection behavior.
Confirm the new l0_cpu tier plus the CPU-Generic-*→l0_cpu mapping fully covers the intended runtime/unit migration.
Validate that removing SM80/pre-SM80 dtype auto-rewrite/rejection doesn’t cause failures in tests expecting the old behavior.

QA Engineer Review

Test-db / CI list changes (test-list-only scope)

Modified/added/removed files under tests/integration/test_lists/test-db/:
- Added: l0_cpu.yml (new system_gpu_count: 0 / cpu_only tier; stage: pre_merge, backend: generic, orchestrator: mpi).
- Removed: l0_cpu_arm.yml and l0_cpu_x86.yml.
- Updated selections:
  - l0_a10.yml (reduced executor/llmapi module sets to match CPU-only migration targets).
  - l0_a100.yml (removed unittest/llmapi/test_executor.py; split unittest/llmapi/test_llm.py into part0 vs not-part0 runs).
  - l0_b200.yml (removed specific tests from KV-cache stats and post-merge backend pytorch lists).
  - l0_dgx_h100.yml (swap/adjust selected distributed and llmapi tests).
  - l0_gh200.yml (update backend pytorch entry from test_llm_quant.py to unittest/bindings).
  - l0_h100.yml (swap/remove a few Ray and post-merge PyTorch entries).
Verdict: needs follow-up (CBTS/coverage data and final migration run outcomes weren’t provided here; earlier CI comments indicate iterative failures before later partial success).

Test code changes (outside `test_lists/`)

Touched test logic/selection mechanics and key determinism points:

tests/integration/defs/test_unittests.py
- Modified: test_unittests_v2 to merge -m mark expressions correctly into per-case arg_list.
- Coverage in test-db: intended to be exercised by the updated CPU stage/test-db targeting; needs follow-up to confirm full coverage.
tests/unittest/llmapi/test_mpi_session.py
- Modified: added an autouse fixture to enable MPI (removes TLLM_DISABLE_MPI), and broadened CPU-only gating for specific tests.
- Coverage in test-db: intended to be covered by the new CPU tier; needs follow-up.
tests/unittest/inputs/test_url_validation.py
- Modified: switched asyncio coroutine driving to asyncio.run() and marked module CPU-only.
- Coverage in test-db: intended to be covered by CPU-only stage selections; needs follow-up.
tests/unittest/_torch/modules/dwdp/test_dwdp_mapping.py
- Modified: marked CPU-only and added an autouse fixture to patch tensorrt_llm.mapping.mpi_disabled for consistent behavior under CPU.
- Coverage in test-db: intended to be covered by CPU tier; needs follow-up.
tests/unittest/llmapi/test_kv_cache_dtype_override.py
- Modified: adjusted get_llm_args(...) invocation to pass gpus_per_node=1 alongside kv_cache_dtype="nvfp4" and marked CPU-only.
- Coverage in test-db: intended to be covered by CPU-only selections; needs follow-up.

Widespread marking-only changes:

Many modules under tests/unittest/** now set pytestmark = pytest.mark.cpu_only (module-level or per-test decorators), restricting execution to CPU environments without otherwise changing assertions.
Verdict (test code): needs follow-up (large CPU-only test surface expanded; requires confirming CI stability across the new l0_cpu tier and the CPU-Generic-* mappings).

Description

First wave of tests that can be moved to run without GPU.

Test Coverage

The 2 CPU pipelines.

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

tongyuantongyu · 2026-07-16T11:52:26Z

/bot run --stage-list "CPU-Generic-x86-1, CPU-Generic-arm-1" --disable-fail-fast

tensorrt-cicd · 2026-07-16T11:57:52Z

PR_Github #59698 [ run ] triggered by Bot. Commit: 1a554c9 Link to invocation

tensorrt-cicd · 2026-07-16T13:45:50Z

PR_Github #59698 [ run ] completed with state SUCCESS. Commit: 1a554c9
/LLM/main/L0_MergeRequest_PR pipeline #48129 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

tongyuantongyu · 2026-07-17T07:13:14Z

/bot run --stage-list "CPU-Generic-x86-1, CPU-Generic-arm-1" --disable-fail-fast

tensorrt-cicd · 2026-07-17T07:20:22Z

PR_Github #59921 [ run ] triggered by Bot. Commit: f4d41dd Link to invocation

tensorrt-cicd · 2026-07-17T08:45:17Z

PR_Github #59921 [ run ] completed with state SUCCESS. Commit: f4d41dd
/LLM/main/L0_MergeRequest_PR pipeline #48323 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

tongyuantongyu · 2026-07-17T08:46:42Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-17T08:51:56Z

PR_Github #59942 [ run ] triggered by Bot. Commit: 4ba739a Link to invocation

tensorrt-cicd · 2026-07-17T13:52:34Z

PR_Github #59942 [ run ] completed with state FAILURE. Commit: 4ba739a
/LLM/main/L0_MergeRequest_PR pipeline #48343 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

tongyuantongyu · 2026-07-20T09:03:58Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-20T09:10:15Z

PR_Github #60333 [ run ] triggered by Bot. Commit: e35de74 Link to invocation

tensorrt-cicd · 2026-07-20T12:47:32Z

PR_Github #60333 [ run ] completed with state SUCCESS. Commit: e35de74
/LLM/main/L0_MergeRequest_PR pipeline #48678 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

tongyuantongyu · 2026-07-21T04:12:20Z

/bot run --stage-list "CPU-Generic-x86-1, CPU-Generic-arm-1" --disable-fail-fast

tensorrt-cicd · 2026-07-21T04:18:49Z

PR_Github #60568 [ run ] triggered by Bot. Commit: b137093 Link to invocation

tensorrt-cicd · 2026-07-21T05:58:30Z

PR_Github #60568 [ run ] completed with state SUCCESS. Commit: b137093
/LLM/main/L0_MergeRequest_PR pipeline #48880 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

tongyuantongyu · 2026-07-21T10:11:45Z

/bot run --stage-list "CPU-Generic-x86-1, CPU-Generic-arm-1" --disable-fail-fast

tensorrt-cicd · 2026-07-21T10:17:22Z

PR_Github #60668 [ run ] triggered by Bot. Commit: b4df526 Link to invocation

tongyuantongyu · 2026-07-21T10:30:37Z

/bot run --stage-list "CPU-Generic-x86-1, CPU-Generic-arm-1" --disable-fail-fast

tensorrt-cicd · 2026-07-21T10:35:57Z

PR_Github #60675 [ run ] triggered by Bot. Commit: 6236369 Link to invocation

tensorrt-cicd · 2026-08-03T01:41:08Z

PR_Github #63320 [ run ] triggered by Bot. Commit: 6daf6e5 Link to invocation

tensorrt-cicd · 2026-08-03T06:23:06Z

PR_Github #63320 [ run ] completed with state FAILURE. Commit: 6daf6e5
/LLM/main/L0_MergeRequest_PR pipeline #51315 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

tongyuantongyu · 2026-08-03T06:31:37Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-03T06:37:46Z

PR_Github #63379 [ run ] triggered by Bot. Commit: 6daf6e5 Link to invocation

tensorrt-cicd · 2026-08-03T09:21:08Z

PR_Github #63379 [ run ] completed with state FAILURE. Commit: 6daf6e5
/LLM/main/L0_MergeRequest_PR pipeline #51360 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

tongyuantongyu · 2026-08-03T11:02:28Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-03T11:08:49Z

PR_Github #63431 [ run ] triggered by Bot. Commit: 6daf6e5 Link to invocation

tensorrt-cicd · 2026-08-03T13:04:49Z

PR_Github #63431 [ run ] completed with state SUCCESS. Commit: 6daf6e5
/LLM/main/L0_MergeRequest_PR pipeline #51405 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Signed-off-by: Yuan Tong <13075180+tongyuantongyu@users.noreply.github.com>

tongyuantongyu · 2026-08-04T03:03:02Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-04T03:08:44Z

PR_Github #63635 [ run ] triggered by Bot. Commit: 7409d59 Link to invocation

tensorrt-cicd · 2026-08-04T07:21:39Z

PR_Github #63635 [ run ] completed with state FAILURE. Commit: 7409d59
/LLM/main/L0_MergeRequest_PR pipeline #51590 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

tongyuantongyu · 2026-08-04T07:32:59Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-04T07:39:09Z

PR_Github #63703 [ run ] triggered by Bot. Commit: 7409d59 Link to invocation

tensorrt-cicd · 2026-08-04T09:41:50Z

PR_Github #63703 [ run ] completed with state SUCCESS. Commit: 7409d59
/LLM/main/L0_MergeRequest_PR pipeline #51655 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

tongyuantongyu · 2026-08-04T09:45:31Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-04T09:52:00Z

PR_Github #63735 [ run ] triggered by Bot. Commit: 7409d59 Link to invocation

tensorrt-cicd · 2026-08-04T13:12:10Z

PR_Github #63735 [ run ] completed with state SUCCESS. Commit: 7409d59
/LLM/main/L0_MergeRequest_PR pipeline #51686 completed with status: 'SUCCESS'
Pipeline passed with automatic retried tests. Check the rerun report for details.

CI Report

Link to invocation

NVIDIA#16498 moved tests/unittest/_torch/distributed from l0_dgx_h100 to l0_cpu, so this file now runs with zero GPUs. The three tests that build an AllReduce skip when the trtllm custom ops are not registered, since AllReduce.__init__ resolves torch.ops.trtllm.allreduce before any of the logic under test. test_lamport_skipped still needs a GPU because the workspace helper it drives allocates device tensors. Signed-off-by: pjdurden <prajjwalchittori1@gmail.com>

github-actions Bot assigned tongyuantongyu Jul 16, 2026

github-actions Bot added the VisualGen label Jul 16, 2026

tongyuantongyu force-pushed the ytong/host-only-test-cases branch 3 times, most recently from b8c05bf to f4d41dd Compare July 17, 2026 07:12

tongyuantongyu added api-compatible Accepted LLM API contract change that is backwards-compatible and removed VisualGen labels Jul 17, 2026

tongyuantongyu force-pushed the ytong/host-only-test-cases branch from 4ba739a to e35de74 Compare July 20, 2026 09:03

tongyuantongyu force-pushed the ytong/host-only-test-cases branch from e35de74 to b137093 Compare July 21, 2026 04:12

tongyuantongyu force-pushed the ytong/host-only-test-cases branch from b137093 to b4df526 Compare July 21, 2026 10:11

tongyuantongyu force-pushed the ytong/host-only-test-cases branch from b4df526 to 6236369 Compare July 21, 2026 10:30

tongyuantongyu added 2 commits August 4, 2026 10:55

Extract CPU only unittests - trunk

5bef6ad

Signed-off-by: Yuan Tong <13075180+tongyuantongyu@users.noreply.github.com>

Extract CPU only unittests - runtime

7409d59

Signed-off-by: Yuan Tong <13075180+tongyuantongyu@users.noreply.github.com>

tongyuantongyu force-pushed the ytong/host-only-test-cases branch from 6daf6e5 to 7409d59 Compare August 4, 2026 02:57

tongyuantongyu merged commit a2387bc into NVIDIA:main Aug 4, 2026
7 checks passed

tongyuantongyu deleted the ytong/host-only-test-cases branch August 5, 2026 04:03

coderabbitai Bot mentioned this pull request Aug 5, 2026

[TRTLLM-13696][test] Part2.2: Migrate CPU only tests - generic, models, disagg #17290

Merged

1 task

pjdurden mentioned this pull request Aug 5, 2026

[None][fix] Fall back to NCCL when CUDA IPC handles cannot be exchanged #17034

Open

This was referenced Aug 6, 2026

[None][test] Run thop custom-op schema checks in the CPU pre-merge stage #17322

Merged

[TRTLLM-13696][test] Part2.3: Migrate CPU only tests - auto deploy #17314

Merged

[TRTLLM-13696][test] Part2.4: Migrate CPU only tests - others #17339

Merged

lonexreb mentioned this pull request Aug 28, 2026

[None][fix] validate trtllm-serve --port with ValueError not assert #16260

Open

Uh oh!

Conversation

tongyuantongyu commented Jul 16, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dev Engineer Review

QA Engineer Review

Test-db / CI list changes (test-list-only scope)

Test code changes (outside test_lists/)

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

tongyuantongyu commented Jul 16, 2026

Uh oh!

tensorrt-cicd commented Jul 16, 2026

Uh oh!

tensorrt-cicd commented Jul 16, 2026

Uh oh!

tongyuantongyu commented Jul 17, 2026

Uh oh!

tensorrt-cicd commented Jul 17, 2026

Uh oh!

tensorrt-cicd commented Jul 17, 2026

Uh oh!

tongyuantongyu commented Jul 17, 2026

Uh oh!

tensorrt-cicd commented Jul 17, 2026

Uh oh!

tensorrt-cicd commented Jul 17, 2026

Uh oh!

tongyuantongyu commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

tongyuantongyu commented Jul 21, 2026

Uh oh!

tensorrt-cicd commented Jul 21, 2026

Uh oh!

tensorrt-cicd commented Jul 21, 2026

Uh oh!

tongyuantongyu commented Jul 21, 2026

Uh oh!

tensorrt-cicd commented Jul 21, 2026

Uh oh!

tongyuantongyu commented Jul 21, 2026

Uh oh!

tensorrt-cicd commented Jul 21, 2026

Uh oh!

tensorrt-cicd commented Aug 3, 2026

Uh oh!

tensorrt-cicd commented Aug 3, 2026

Uh oh!

tongyuantongyu commented Aug 3, 2026

Uh oh!

tensorrt-cicd commented Aug 3, 2026

Uh oh!

tensorrt-cicd commented Aug 3, 2026

Uh oh!

tongyuantongyu commented Aug 3, 2026

Uh oh!

tensorrt-cicd commented Aug 3, 2026

Uh oh!

tensorrt-cicd commented Aug 3, 2026

Uh oh!

tongyuantongyu commented Aug 4, 2026

Uh oh!

tensorrt-cicd commented Aug 4, 2026

Uh oh!

tensorrt-cicd commented Aug 4, 2026

Uh oh!

tongyuantongyu commented Aug 4, 2026

Uh oh!

tensorrt-cicd commented Aug 4, 2026

Uh oh!

tensorrt-cicd commented Aug 4, 2026

Uh oh!

tongyuantongyu commented Aug 4, 2026

tongyuantongyu commented Jul 16, 2026 •

edited by coderabbitai Bot

Loading

Test code changes (outside `test_lists/`)