[Bug]: CI not running all tests/compile tests

[zou3519]

Your current environment

n/a

🐛 Describe the bug

Examples of things that are not tested:

• tests/compile/test_config.py
• tests/compile/test_functionalization.py

A partial list of things that are tested:

• vllm/.buildkite/test-pipeline.yaml

  Lines 324 to 331 in 497e763

  	- pytest -v -s compile/test_pass_manager.py
  	- pytest -v -s compile/test_config.py
  	- pytest -v -s compile/test_fusion.py
  	- pytest -v -s compile/test_fusion_attn.py
  	- pytest -v -s compile/test_silu_mul_quant_fusion.py
  	- pytest -v -s compile/test_sequence_parallelism.py
  	- pytest -v -s compile/test_async_tp.py
  	- pytest -v -s compile/test_fusion_all_reduce.py

As mentioned in #22682, we should really just update the test command to pytest tests/compile, with some opt-outs for some tests that actually need special configurations. This will make this one test take longer but will make things a lot saner for us to maintain.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[ZJY0516]

Hi! I'd be happy to help with this issue.

[gmagogsfm]

@zou3519 I am almost done with fixing it, could you assign this issue to me?

[gmagogsfm]

@zou3519

All tests are now fixed and enabled in CI, we could close this issue now.

[zou3519]

@gmagogsfm as a final item, could you make it so that the ci runs pytest tests/compile --exclude {the_tests_that_need_distributed} instead please? This way when someone adds a new test they don't have to manually add it to CI.