Skip to content

[PluggableLayer][MM] Add PluggableLayer for CustomQwen2Decoder#37293

Merged
ProExpertProg merged 6 commits intovllm-project:mainfrom
Wangbei25:main
Mar 20, 2026
Merged

[PluggableLayer][MM] Add PluggableLayer for CustomQwen2Decoder#37293
ProExpertProg merged 6 commits intovllm-project:mainfrom
Wangbei25:main

Conversation

@Wangbei25
Copy link
Contributor

@Wangbei25 Wangbei25 commented Mar 17, 2026

Purpose

Add PluggableLayer for CustomQwen2Decoder. As a result, this kind of encoder can be replaced with custom OOT implementations.

Test Plan

vllm serve /weights/DeepSeek-OCR-2
--served-model-name deepseekocr2
--trust-remote-code

curl http://localhost:1055/v1/chat/completions
-H "Content-Type: application/json"
-d '{"model": "deepseekocr2","messages": [{"role": "user","content": [
{"type": "image_url","image_url": {"url": "file:///datasets/textvqa_val/train_images/1080p.jpg","detail": "high"}},
{"type": "text","text": "Explain the contents of the picture with more than 500 words"}]}],
"max_tokens": 300
}'

Test Result

{"id":"chatcmpl-9be9f1663df4218e","object":"chat.completion","created":1773060077,"model":"deepseekocr2","choices":[{"index":0,"message":{"role":"assistant","content":"Ixar sampled a moment on the edge of a sandy beach. The white dog, sitting on its hind legs, looks directly at the camera. The person, dressed in a blue and white plaid shirt, sits on the sand with their legs crossed. In the background, the vast expanse of the ocean stretches out. The sun is setting, adding a warm glow to the scene. The image captures a peaceful moment between a dog and a person on a beach at sunset.","refusal":null,"annotations":null,"audio":null,"function_call":null,"tool_calls":[],"reasoning":null},"logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":559,"total_tokens":656,"completion_tokens":97,"prompt_tokens_details":null},"prompt_logprobs":null,"prompt_token_ids":null,"kv_transfer_params":null}


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@mergify
Copy link

mergify bot commented Mar 17, 2026

Documentation preview: https://vllm--37293.org.readthedocs.build/en/37293/

@mergify mergify bot added documentation Improvements or additions to documentation qwen Related to Qwen models labels Mar 17, 2026
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request correctly refactors CustomQwen2Decoder to be a PluggableLayer, which is a good step for extensibility. The documentation is also updated to reflect this and other related changes. My review includes a couple of suggestions to fix path separators in the documentation, which will ensure the documentation builds correctly across different operating systems.

**12. Encoder:**

```python
--8<-- "vllm\model_executor\models\deepencoder2.py:qwen2_decoder"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The file path here uses a backslash (\) as a separator. For cross-platform compatibility and consistency with other paths in this document, please use a forward slash (/).

Suggested change
--8<-- "vllm\model_executor\models\deepencoder2.py:qwen2_decoder"
--8<-- "vllm/model_executor/models/deepencoder2.py:qwen2_decoder"

Signed-off-by: Wangbei25 <wangbei41@huawei.com>
@shen-shanshan
Copy link
Contributor

@ProExpertProg Would you mind take a look at this PR? Just a minor refactor for pluggable layer.

@ProExpertProg ProExpertProg enabled auto-merge (squash) March 20, 2026 04:23
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Mar 20, 2026
@ProExpertProg ProExpertProg merged commit 0674d1f into vllm-project:main Mar 20, 2026
55 of 56 checks passed
chooper26 pushed a commit to intellistream/vllm-hust that referenced this pull request Mar 21, 2026
…project#37293)

Signed-off-by: Wangbei25 <wangbei41@huawie.com>
Signed-off-by: Wangbei25 <wangbei41@huawei.com>
Co-authored-by: Wangbei25 <wangbei41@huawie.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation qwen Related to Qwen models ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants