-
Notifications
You must be signed in to change notification settings - Fork 843
[Engine] Add entrypoint class and stage management #13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
20cfd15
Add entrypoint class and stage management
Gaohan123 6111c28
fix some reviews from gemini
Gaohan123 87d6414
supplement some imports
Gaohan123 98a85d4
fix for some reviews
Gaohan123 2c08c04
fix for reviews of copilot
Gaohan123 61a215f
fix file name
Gaohan123 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| from typing import Optional | ||
| from dataclasses import dataclass | ||
| from vllm.engine.arg_utils import EngineArgs | ||
| from vllm.utils import FlexibleArgumentParser | ||
|
|
||
| from vllm_omni.config import OmniModelConfig | ||
|
|
||
|
|
||
| @dataclass | ||
| class OmniEngineArgs(EngineArgs): | ||
| stage_id: int = 0 | ||
| model_stage: str = "thinker" | ||
| model_arch: str = "Qwen2_5OmniForConditionalGeneration" | ||
| engine_output_type: Optional[str] = None | ||
|
|
||
| @staticmethod | ||
| def add_cli_args(parser: FlexibleArgumentParser) -> FlexibleArgumentParser: | ||
| """Shared CLI arguments for vLLM engine.""" | ||
| parser.add_argument( | ||
| "--engine-output-type", | ||
| type=str, | ||
| default=EngineArgs.engine_output_type, | ||
| help=( | ||
| "Declare EngineCoreOutput.output_type (e.g., 'text', 'image', " | ||
| "'text+image', 'latent'). This will be written into " | ||
| "model_config.engine_output_type for schedulers to use." | ||
| ), | ||
| ) | ||
| parser.add_argument("--model-stage", type=str, default=OmniEngineArgs.model_stage, | ||
| help="Declare model stage (e.g., 'thinker', 'talker', 'token2wav'). This will be written into model_config.model_stage for schedulers to use.") | ||
| return parser | ||
|
|
||
| def create_model_config(self) -> OmniModelConfig: | ||
| # First, get the base ModelConfig from the parent class | ||
| base_config = super().create_model_config() | ||
|
|
||
| # Create OmniModelConfig by copying all base config attributes | ||
| # and adding the new omni-specific fields | ||
| config_dict = base_config.__dict__.copy() | ||
|
|
||
| # Add the new omni-specific fields | ||
| config_dict['stage_id'] = self.stage_id | ||
| config_dict['model_stage'] = self.model_stage | ||
| config_dict['model_arch'] = self.model_arch | ||
| config_dict['engine_output_type'] = self.engine_output_type | ||
|
|
||
| # Create and return the OmniModelConfig instance | ||
| omni_config = OmniModelConfig(**config_dict) | ||
| omni_config.hf_config.architectures = omni_config.architectures | ||
|
|
||
| return omni_config |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider adding a docstring to explain what the
@configdecorator does and why it's being used here. This will improve readability and understanding of the code.