Skip to content

Fix nondeterministic FlashInfer GDN alignment test - #37343

Merged
hnyls2002 merged 1 commit into
sgl-project:mainfrom
nvpohanh:codex/fix-gdn-alignment-test-uninitialized
Sep 2, 2026
Merged

hnyls2002 merged 1 commit into
sgl-project:mainfrom
nvpohanh:codex/fix-gdn-alignment-test-uninitialized

Conversation

@nvpohanh

@nvpohanh nvpohanh commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

[by Codex]

Summary

  • initialize the misaligned a and b test views with deterministic finite values
  • keep their values distinct so the test still verifies that both FlashInfer arguments are copied independently and without corruption
  • avoid comparing allocator-provided torch.empty contents, which can contain NaNs and make assert_close fail nondeterministically

What failure does this fix?

This fixes the base-a-test-cpu failure observed while testing #33922:

The same test failed standalone on latest main in lmsysorg/sglang:dev-cu13 on a computelab node. In that run, captured["b"] reported 8/8 mismatched elements with a greatest absolute difference of nan. Which tensor/elements fail depends on the contents returned by the allocator, but both failures have the same root cause.

Additional confirmed CI occurrences

I scanned 600 failed PR Test Base workflow runs after #35758 introduced this test and inspected the logs of their failed base-a-test-cpu root jobs. The exact failure was confirmed in three pipeline runs across two unrelated PRs:

All three failed in TestFlashInferGDNAlignment::test_decode_repairs_read_only_arguments_before_flashinfer with the same signature: 2/8 mismatched elements and a nan difference at index (0, 0, 2). The recurrence across unrelated PRs and separate workflow runs supports that this is an intermittent main-branch test defect rather than a regression caused by any one target PR.

Root cause

_view_with_pointer_mod() intentionally builds misaligned views from torch.empty backing storage. In this test, a and b were never initialized before kernel.decode():

a = _view_with_pointer_mod((1, 1, 8), torch.bfloat16, 16)
b = _view_with_pointer_mod((1, 1, 8), torch.bfloat16, 16)

The alignment-repair path correctly copies these views into aligned scratch buffers. The test then compares the copied tensors against the original uninitialized tensors with torch.testing.assert_close. If the allocator-provided bit patterns contain NaNs, the comparison fails because assert_close does not consider corresponding NaNs equal by default.

Therefore:

  • the failure is intermittent in manifestation, because it depends on allocator reuse and the uninitialized bit patterns;
  • it is not an infrastructure-only flake—there is a concrete test-code defect;
  • it is not caused by Fix Qwen3.5 GDN multi-item scoring #33922; the exact test also reproduces on latest main without that PR;
  • the defective test was introduced by #35758 (qwen 3.8 rebase), merge commit 5f216fc33f315ade378bc7cd295ee88f5322d4d3, which added this test file and the uninitialized setup.

Fix

Initialize a and b with distinct finite sequences before calling decode(). This removes allocator-dependent NaNs while preserving the test's intended coverage:

  • both source views remain deliberately 16-byte-offset/misaligned;
  • the test still verifies 32-byte alignment at the FlashInfer call boundary;
  • distinct values ensure the two live argument buffers cannot be accidentally aliased or swapped;
  • the final assert_close checks still verify lossless copy behavior.

Validation

After rebasing onto 60f881b40c:

  • exact failing pytest node: passed
  • full test_gdn_flashinfer_alignment.py: 10 passed
  • 500 allocator-poison stress iterations: passed
  • BLACK_NUM_WORKERS=1 SKIP=no-commit-to-branch pre-commit run --all-files --show-diff-on-failure: passed

CI States

Latest PR Test (Base): ❌ Run #33467025020
Latest PR Test (Extra): ❌ Run #33467024850
Latest PR Test (AMD ROCm 7.2): ❌ Run #33467024983

@nvpohanh

nvpohanh commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-tests test/registered/unit/layers/attention/test_gdn_flashinfer_alignment.py

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Results for /rerun-tests test/registered/unit/layers/attention/test_gdn_flashinfer_alignment.py:

🚀 ubuntu-latest (1 test): ✅ View workflow run

cd test/ && python3 registered/unit/layers/attention/test_gdn_flashinfer_alignment.py

@hnyls2002
hnyls2002 merged commit d585cec into sgl-project:main Sep 2, 2026
110 of 123 checks passed
StevenChenSE pushed a commit to StevenChenSE/sglang that referenced this pull request Sep 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants