[model] feat: add registration and config converter for Qwen 2.5-Omni#5120
[model] feat: add registration and config converter for Qwen 2.5-Omni#5120martinzhang03 wants to merge 1 commit intoverl-project:mainfrom
Conversation
|
We're now use mbridge and NVIDIA-NeMo/Megatron-Bridge to convert hf model to mcore GPTModel, please submit a PR to them. |
Thanks for the heads-up, @wuxibin89! I'll port the configuration logic over to that repository and submit a PR there instead. Should I keep this PR open for a moment in case the registration part (the Enum changes) is still needed here, or should I close this entirely? |
Close this PR since we're going to cleanup mcore model registration in verl #4496 #4530 |
What does this PR do?
This PR introduces the initial registration and configuration conversion logic for the Qwen 2.5-Omni model within the veRL framework.
Key Changes
Model Registration: Added
QWEN2_5_OMNIto theSupportedVLMandSupportedModelenums inverl/models/mcore/registry.py.Config Converter: Implemented
hf_to_mcore_config_qwen2_5_omniinverl/models/mcore/config_converter.py.Qwen2_5OmniConfig→thinker_config→text_config.mrope_sectionfromrope_parametersto support multimodal rotary positional embeddings (mRoPE).Forward Registry Skeleton: Added commented-out entries for
MODEL_FORWARDregistries as placeholders for the upcoming implementation.Implementation Details
The implementation was developed by:
Qwen2_5OmniForConditionalGenerationstructure in the Hugging Face transformers library.Future Work (Next Steps)
This PR serves as the foundation for full Qwen 2.5-Omni support. Immediate follow-up PRs will include:
Technical Summary (中文)
本次 PR 实现了 Qwen 2.5-Omni 模型在 veRL 中的基础注册与配置转换逻辑:
thinker_config嵌套结构,准确提取核心文本参数。