-
Notifications
You must be signed in to change notification settings - Fork 1k
[Test] L4 complete diffusion feature test for Bagel models #1938
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
e0d9cb4
[Test] L4 complete diffusion feature test for Bagel models
NumberWan 1ca862c
Fixed format
NumberWan 2b6c223
Fixed format
NumberWan ad0c6b2
Edited Comments
NumberWan fb78bb4
Edited Comments
NumberWan 752d2f2
Edited Comments
NumberWan 321c634
Changes for the Temp CI test
NumberWan 01a6af2
Revert "Changes for the Temp CI test"
NumberWan 27addc4
Merge branch 'main' into bagel_example
NumberWan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,132 @@ | ||
| """ | ||
| Comprehensive tests of diffusion features that are available in online serving mode | ||
| and are supported by the following models: | ||
| - ByteDance-Seed/BAGEL-7B-MoT | ||
| Coverage: | ||
| - TeaCache | ||
| - Cache-DiT | ||
| - CFG-Parallel | ||
| - Tensor-Parallel | ||
|
|
||
| assert_diffusion_response validates successful generation and the expected | ||
| 512x512 resolution. | ||
| """ | ||
|
|
||
| import pytest | ||
|
|
||
| from tests.conftest import ( | ||
| OmniServer, | ||
| OmniServerParams, | ||
| OpenAIClientHandler, | ||
| dummy_messages_from_mix_data, | ||
| ) | ||
| from tests.utils import hardware_marks | ||
|
|
||
| PROMPT = "A futuristic city skyline at twilight, cyberpunk style, ultra-detailed, high resolution." | ||
| NEGATIVE_PROMPT = "low quality, blurry, distorted, deformed, watermark" | ||
|
|
||
| SINGLE_CARD_FEATURE_MARKS = hardware_marks(res={"cuda": "H100"}) | ||
| PARALLEL_FEATURE_MARKS = hardware_marks(res={"cuda": "H100"}, num_cards=2) | ||
|
NumberWan marked this conversation as resolved.
|
||
|
|
||
|
|
||
| def _get_diffusion_feature_cases(model: str): | ||
| """Return L4 diffusion feature cases for Bagel. | ||
| TeaCache, Cache-DiT, CFG-Parallel, Tensor-Parallel. | ||
| """ | ||
|
|
||
| return [ | ||
| # TeaCache (single-card) | ||
| pytest.param( | ||
| OmniServerParams( | ||
| model=model, | ||
| server_args=[ | ||
| "--cache-backend", | ||
| "tea_cache", | ||
| ], | ||
| ), | ||
| id="single_card_teacache", | ||
| marks=SINGLE_CARD_FEATURE_MARKS, | ||
| ), | ||
| # Cache-DiT (single-card) | ||
| pytest.param( | ||
| OmniServerParams( | ||
| model=model, | ||
| server_args=[ | ||
| "--cache-backend", | ||
| "cache_dit", | ||
| ], | ||
| ), | ||
| id="single_card_cache_dit", | ||
| marks=SINGLE_CARD_FEATURE_MARKS, | ||
| ), | ||
| # CFG-Parallel size 2 (2 GPUs, TeaCache backend) | ||
| pytest.param( | ||
| OmniServerParams( | ||
| model=model, | ||
| server_args=[ | ||
| "--cache-backend", | ||
| "tea_cache", | ||
| "--cfg-parallel-size", | ||
| "2", | ||
| ], | ||
| ), | ||
| id="parallel_cfg_2", | ||
| marks=PARALLEL_FEATURE_MARKS, | ||
| ), | ||
| # Tensor-Parallel size 2 (2 GPUs, Cache-DiT backend) | ||
| pytest.param( | ||
| OmniServerParams( | ||
| model=model, | ||
| server_args=[ | ||
| "--cache-backend", | ||
| "cache_dit", | ||
| "--tensor-parallel-size", | ||
| "2", | ||
| ], | ||
| ), | ||
| id="parallel_tp_2", | ||
| marks=PARALLEL_FEATURE_MARKS, | ||
| ), | ||
| ] | ||
|
|
||
|
|
||
| @pytest.mark.advanced_model | ||
| @pytest.mark.diffusion | ||
| @pytest.mark.parametrize( | ||
| "omni_server", | ||
| _get_diffusion_feature_cases("ByteDance-Seed/BAGEL-7B-MoT"), | ||
| indirect=True, | ||
| ) | ||
| def test_bagel( | ||
| omni_server: OmniServer, | ||
| openai_client: OpenAIClientHandler, | ||
| ): | ||
| """L4 diffusion feature coverage for Bagel on H100. | ||
|
|
||
| This test exercises: | ||
| - TeaCache | ||
| - Cache-DiT | ||
| - CFG-Parallel (size=2) | ||
| - Tensor-Parallel (size=2) | ||
|
|
||
| Validation is delegated to assert_diffusion_response in tests.conftest, | ||
| which checks output dimensions and basic correctness. | ||
| """ | ||
|
|
||
| messages = dummy_messages_from_mix_data(content_text=PROMPT) | ||
|
|
||
| request_config = { | ||
| "model": omni_server.model, | ||
| "messages": messages, | ||
| "extra_body": { | ||
| "height": 512, | ||
| "width": 512, | ||
| "num_inference_steps": 2, | ||
| # Enable CFG for models that use classifier-free guidance | ||
| "negative_prompt": NEGATIVE_PROMPT, | ||
| "true_cfg_scale": 4.0, | ||
| "seed": 42, | ||
| }, | ||
| } | ||
|
|
||
| openai_client.send_diffusion_request(request_config) | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need compare some specific pixel value to ensure these feature can output expect result, so you can refer to bagel's e2e test, and I also think only use
OmniDiffusionis enough to test these feature.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the suggestion!
But according to the internal agreement that:
This Bagel L4 suite follows exactly the same pattern/infra as test_qwen_image_edit_expansion.py and #1682 to keep the behavior consistent across models.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#1832 This is the RFC @princepride
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you, I will check it later
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think maybe you can unify the bagel test cases in the current test-ready.yml into test_bagel.py, and use the current code style
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@princepride Thanks! This PR follows the current L4 diffusion e2e scope in RFC #1832 / template #1682 (online serving, shape checks). . If this is acceptable, could you please dismiss the “changes requested” so this PR can merge?