[ocss884]

Motivation

Relevant to #5997 . Update VerlEngine test to a a more comprehensive one, which uses 8-gpu (dp==4, tp==2) . @merrymercy @zhaochenyang20

Modifications

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

[zhaochenyang20]

I think only a small part of test_verl_engine.py should use 8-gpu. We should split it, to save 8-gpu time.

[ocss884]

I split the tests by requirements of gpus <=2 or >2. Random choice (although only 1 model for 8-gpus ci now) is added in the 8-gpus one. The 2-gpu one will not have any model for ci test, only for local usage. @zhaochenyang20

[zhaochenyang20]

we can remove gpt2, nobody care about it

[zhaochenyang20]

I mean, just delete it

[zhaochenyang20]

Just rebased and after the CI, we can merge it

[zhaochenyang20]

Now there is already a 4 - gpu CI, which can be used directly. Note that you need to modify the current test_verl_engine_8_gpu.py. One thing is to change the name, and the other is to check the content. There should be no unit tests that actually use 8 gpus.

[zhaochenyang20]

hey, what's the difference in unit-test-backend-4-gpu (line 92) and unittest-test-backend-4-gpu (line 110)? I think we should only take the second one, which has more dedicated rules to reduce CI waste.

[zhaochenyang20]

I mean, just delete it

[zhaochenyang20]

    "per-commit-4-gpu": [
        TestFile("test_verl_engine.py", 64),
    ],
    "per-commit-2-gpu-amd": [
        TestFile("test_mla_tp.py", 170),
    ],
    "per-commit-4-gpu": [
        TestFile("test_local_attn.py", 250),
        TestFile("test_pp_single_node.py", 150),
        TestFile("test_verl_engine_4_gpu.py", 64),
    ],

redundant.

[zhaochenyang20]

We do not need to save CI time for 4-gpu test now. So we can make a bigger ALL_MODELS list, and randomly choose one to test on CI. and Test all locally.

Say, how about Qwen 2.5 model and llama 3.1 model?

[zhaochenyang20]

If we can have a EP test later, it would be optimal. But not for now 😂