Skip to content

[https://nvbugs/7050983][infra] Waive 11 failed case(s) on DGX_B200 - #21722

Draft
trtllm-agent wants to merge 1 commit into
NVIDIA:mainfrom
trtllm-agent:auto-waive/45d9bbf79d716de1ae87699bf890a2a9
Draft

trtllm-agent wants to merge 1 commit into
NVIDIA:mainfrom
trtllm-agent:auto-waive/45d9bbf79d716de1ae87699bf890a2a9

Conversation

@trtllm-agent

Copy link
Copy Markdown
Collaborator

Description

Temporarily waive 11 failed case(s) on DGX_B200. Opened automatically by auto-waive (draft; a human reviews and merges).

The native main-break detector (native-p3/k3-w12h-fresh12h-nopass-hostclock/1.0.1) reports unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_fused_indexer_q_uses_serial_prepare broken on main for DGX_B200: failure signature failed on teardown with runtimeerror: invalid argument to exchangedevice, onset 2026-11-05T08:45:49Z, latest failure 2026-11-05T08:52:47Z. Policy native_p3@1.0.0 selected it; https://nvbugs/7050983 tracks the fix. Scope DGX_B200 is the stage prefix that ran the failing test.

  • Bug: https://nvbugs/7050983 (details, logs and 7-day test history are in the bug)
  • Restore coverage when: Remove this line when https://nvbugs/7050983 is fixed or closed and the test passes on DGX_B200 post-merge; un-waive follows the bug, not the PR.
  • Scope reviewed by: auto-waive at 2026-11-05T12:05:48.558424+00:00

Tests in this break event:

  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_fused_indexer_q_uses_serial_prepare on DGX_B200
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_compress_fp4_halves_pool_footprint on DGX_B200
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_compress_rejects_unknown_dtype on DGX_B200
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_compress_token_bytes_fp4 on DGX_B200
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_compress_token_bytes_fp8 on DGX_B200
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_k_dtype_accepts_fp4 on DGX_B200
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_k_dtype_default_is_fp4 on DGX_B200
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_k_dtype_rejects_non_128_head_dim on DGX_B200
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_overlapped_prepare_records_aux_outputs_on_consumer_stream on DGX_B200
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_v4_has_no_separate_indexer_k_cache_dtype_field on DGX_B200
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_v4_inherits_indexer_k_dtype_field on DGX_B200

PRs whose pre-merge runs hit this break (rebase or re-run after this merges):

full:DGX_B200/unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_fused_indexer_q_uses_serial_prepare SKIP (https://nvbugs/7050983)
full:DGX_B200/unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_compress_fp4_halves_pool_footprint SKIP (https://nvbugs/7050983)
full:DGX_B200/unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_compress_rejects_unknown_dtype SKIP (https://nvbugs/7050983)
full:DGX_B200/unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_compress_token_bytes_fp4 SKIP (https://nvbugs/7050983)
full:DGX_B200/unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_compress_token_bytes_fp8 SKIP (https://nvbugs/7050983)
full:DGX_B200/unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_k_dtype_accepts_fp4 SKIP (https://nvbugs/7050983)
full:DGX_B200/unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_k_dtype_default_is_fp4 SKIP (https://nvbugs/7050983)
full:DGX_B200/unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_k_dtype_rejects_non_128_head_dim SKIP (https://nvbugs/7050983)
full:DGX_B200/unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_overlapped_prepare_records_aux_outputs_on_consumer_stream SKIP (https://nvbugs/7050983)
full:DGX_B200/unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_v4_has_no_separate_indexer_k_cache_dtype_field SKIP (https://nvbugs/7050983)
full:DGX_B200/unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_v4_inherits_indexer_k_dtype_field SKIP (https://nvbugs/7050983)

What to check before merging:

  • The bug's test-history links show the cases failing on main, not only on the trigger PR.
  • The scope DGX_B200 matches the failing stage and no narrower waiver already exists.
  • Nothing else is needed from you after merging: the bug owner removes these lines with the fix.
Provenance
  • Detector native-p3/k3-w12h-fresh12h-nopass-hostclock/1.0.1, policy native_p3@1.0.0
  • Run e2720be2-2960-4b4f-8b4d-e44a5b4d75ca; snapshot SHA-256 19a4487913c2b0de0cd050147e2bb9cf651108766e2ab3ed423f9a0786744687; config SHA-256 09cf0ee078a35cefbb2080f3d321976a2a18bc5d6e6c6b3de9d637db1c92730b
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_fused_indexer_q_uses_serial_prepare: incident 3faf7ba10ec9afaea68b96d0857c3fcc7936e3db657423b3ee8e526cd3887aa1, observation f9f578b850783048b1dea77b89c444ef358455bbcd6784f4baf4c17ee12faf97
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_compress_fp4_halves_pool_footprint: incident 423c50b702965ec9ac3493bdb00184eeac2ddd14a28bff4c71e64616d0daf271, observation 9b5f07a636080682e55b333d2a5b018a4f44cb7beafc6b56813885ddb3c73cb3
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_compress_rejects_unknown_dtype: incident cdf47d2f30b3a76ea68d10b755ca6fa1119f9ea9c1b43d3e3b14d8edfabd0c44, observation 272db2e37b184010c2e56b3904edfffbe032844d597df883b8a427731355ded8
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_compress_token_bytes_fp4: incident 3a0efcf0f5a776c08dc454683b8c2fe65790891052beccc9c70eb9cf4f32e705, observation 3a30c88d36ca7613293a57e1edb3c8aac6a739332ad7eb43cb547242c8cd0fa6
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_compress_token_bytes_fp8: incident 34eefe54b51a5c2e4863410ade674aeb304cad1e249ca1077ed52e27bd733b02, observation 93f514cc49778cd2445ff5ecba68011e1fc23b1818f1894cc757847e9a4c5469
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_k_dtype_accepts_fp4: incident b372946efaf89b81dbfde5fa50146dd3385fabe5c183ae397a9fa87000abb5b9, observation 608094e2feaf39c5545ea7038f5826f0d970d736144279675e138bd207881358
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_k_dtype_default_is_fp4: incident ee03e13334a3de3f682b2c0f445978cceddda9ee6ce103ed3ca50afa2a07c00a, observation 26b097b2b1c52f389a24c81acf06b644e0751561fe9d390ceff968817cf1c90b
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_indexer_k_dtype_rejects_non_128_head_dim: incident e967e25400455dadbe27d56d72663bffa2bb42fd034082231515a0d9a84615b4, observation db2a8b78b89567cc4989794626297f7af564066d9f39542f3857a48ea51b46e4
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_overlapped_prepare_records_aux_outputs_on_consumer_stream: incident f0f055c4f5ab1e3c2556fbc6bf24ac341e07b256253847cc88ec7c6239f20094, observation 77bc3db45edacdacfc359750b4ea54534744f57f6c2a65f2bfc2bb3d1df50dac
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_v4_has_no_separate_indexer_k_cache_dtype_field: incident 9c9b0782b15ae573588f36533af07a7e5f02fa0c98db3d781514aad337b1c78b, observation dced056f5abd9b1fbd9a324f843fd5984fe313393dfc4e7cc44280b76f72e7fb
  • unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_fp4_indexer.py::test_v4_inherits_indexer_k_dtype_field: incident 2e37a40871186b29ba678dc3fa003fe4993e65da815f11d1b196f98deb94abca, observation 478a739d1042d5e6747696c9a9db823a4a97e00091b0750a6c82c3a6c6820afc

Test Coverage

Review the incident evidence and selector scope before merging. This proposal does not
establish that main is fixed. Confirm restoration separately after the corrective change.

PR Checklist

  • Please check this after reviewing the above items as appropriate for this PR.

Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant