[CI] Add M3 MSA tests to CI - #49143
Conversation
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
github-actions
Bot
added
the
ready
Head branch was pushed to by a user without write access
| num_v_heads, | ||
| device="cuda", | ||
| dtype=dtype, | ||
| 1, total_tokens, num_v_heads, device="cuda", dtype=dtype, generator=rng | ||
| ) | ||
| # Match upstream FLA GatedDeltaNet synthetic initialization: | ||
| # https://github.com/fla-org/flash-linear-attention/blob/main/fla/layers/gated_deltanet.py | ||
| A = torch.empty(num_v_heads, device="cuda", dtype=torch.float32).uniform_(0, 16) |
There was a problem hiding this comment.
Probably here should be uniform_(0, 16, generator=rng)?
Also I am wondering why these changes in tests/kernels/mamba/test_gdn_prefill_cutedsl.py are needed for? Is it some bug fix? How this test is connected with updating CI tests list?
There was a problem hiding this comment.
thank you, i missed it
the test passes on my machine locally, but is failing in CI https://buildkite.com/vllm/ci/builds/78822/list?jid=019f7e6e-a40e-47c9-ae8f-2950ce8843d0&tab=output. We don't see this previously because the test has never run in CI before.
i thought it was due to non-deterministic inputs, so i added RNG seed. looks like even with seeded input, I need to relax the tolerance.
There was a problem hiding this comment.
This might be due to different model of GPU that you use locally and that is used in the CI for this test case.
When I recently did a bump of cutlass to 4.6.0 link, I had a similar tolerance problem with one of the CI tests. This test failed for me locally on main branch on GB200, but in the CI it passed on Nvidia L4.
Is you see such a problem it would be great if you can grab info about the environment you run locally (use python vllm/collect_env.py) as well what environment CI uses for this test case (from your CI logs I see it uses B200), and then report a CI issue as I did and link this issue to this PR. If you decide to adjust the tolerance for this test case at first I would ask about this and describe situation in #sig-si to let others know what the problem is. Probably they would help you to decide what to do next.
There was a problem hiding this comment.
The GDN prefill test passes now after I fixed the missing rng in uniform_() as you have pointed out. https://buildkite.com/vllm/ci/builds/78845/list?jid=019f7edd-b964-481a-94dc-b8f15252c8ff&tab=output
So everything is good?
There was a problem hiding this comment.
Super! Then there is no need to relax tolerance. Please, mention in the PR description please that this PR not only enables some of the existing unit tests to CI but also fixed test_gdn_prefill_cutedsl.py that failed on different environments and this PR made inputs of this test to be deterministic as a fix.
|
Thanks! Is the plan to add the MSA unit tests to the CI? I suppose need to wait for landing of #49016? |
|
@wzhao18 Yes, will need to wait for #49016. Now CI shows the failing MSA tests as expected https://buildkite.com/vllm/ci/builds/78845/list?jid=019f7edd-b964-481a-94dc-b8f15252c8ff&tab=output |
4de3be2 to
b5615d6
Compare
gau-nernst
changed the title
|
Ugh, looks like the MSA test are broken. I will likely break this into 2 PRs:
Likely this PR will remain for MSA test, and another PR for GDN prefill |
|
@gau-nernst, could you please pull main to your PR? |
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
b5615d6 to
8c8dfd4
Compare
|
@arpera I have rebased and fixed the failing test. PR description is updated. |
|
LGTM. Let's wait until all the CI tests would finish and if everything ok then merge this PR. |
|
@gau-nernst, could you please check what is wrong with failed CI jobs? |
|
The errors look unrelated. Just merged main to re-run CI |
|
CI looks good now. There is only one failure buildkite/ci/pr/kernels-fusedmoe-layer-test-2-b200s that is also failing on main 81480. I think we can merge this item then. Thanks! |
5 participants
Purpose
Follow up from #47442. Turns out some SM100 kernel tests did not run in CI because they were not included in the buildkite config.
cc @arpera
Edit:
This PR also makes RNG inputs forDone separately in #49388test_gdn_prefill_cutedsl.pydeterministic to make the test deterministic (different RNG inputs may shift the tolerance a bit. In the original test, I intentionally put the tolerance quite tight).Edit 2: This PR also fixes the MSA kernel test. It has been broken since Minimax M3 model code merge but has never surfaced since the test has never run in CI.
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.