[Docs] Add InternVL series tutorial for single NPU by gcanlin · Pull Request #3664 · vllm-project/vllm-ascend

gcanlin · 2025-10-23T07:34:23Z

What this PR does / why we need it?

Closes #3508.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

No need to test. All scripts and Python code in the docs has been tested.

vLLM version: v0.11.0rc3
vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0

Signed-off-by: gcanlin <canlinguosdu@gmail.com>

github-actions · 2025-10-23T07:36:05Z

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

A PR should do only one thing, smaller PRs enable faster reviews.
Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

gemini-code-assist

Code Review

This pull request adds a new tutorial for running InternVL series models on a single NPU. The tutorial is well-structured and provides clear instructions for both offline inference and online serving. However, there is a significant issue with the model path used in the examples, which points to a non-public model. This will prevent users from following the tutorial. My review includes a comment to address this.

Signed-off-by: gcanlin <canlinguosdu@gmail.com>

Yikun · 2025-10-24T14:22:48Z

Please add an accuracy test with same parameter with doc before merge.

https://github.com/vllm-project/vllm-ascend/tree/main/tests%2Fe2e%2Fmodels%2Fconfigs

github-actions · 2025-10-27T08:58:48Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

### What this PR does / why we need it? As a validation for #3664, add end-to-end tests to monitor the InternVL model and ensure its continuous proper operation. This PR is only for single-card. So the models that have more parameters than 8B like 78B are needed to test using multi-cards. ### Does this PR introduce _any_ user-facing change? None. ### How was this patch tested? `pytest -sv tests/e2e/singlecard/multi-modal/test_internvl.py` - vLLM version: v0.11.0 - vLLM main: vllm-project/vllm@83f478b --------- Signed-off-by: gcanlin <canlinguosdu@gmail.com>

ApsarasX · 2025-11-03T03:57:02Z

+]
+
+# Same image for all prompts
+images = [image] * len(prompts)


This images variable is not used, should it be deleted?

Sorry for the late response. It seems that I lost some code here. I will complete it later. Thanks!

gcanlin · 2025-11-14T01:11:36Z

Now the e2e test(#3796, InternVL2, InternVL2_5, InternVL3, InternVL3_5) and accuracy/correctness test(#3964, only InternVL3 for now because of the limit of resources, maybe we should add more models in future). I will reconstruct this doc referring to the new docs of DeepseekV3.2.

### What this PR does / why we need it? As a validation for vllm-project#3664, add end-to-end tests to monitor the InternVL model and ensure its continuous proper operation. This PR is only for single-card. So the models that have more parameters than 8B like 78B are needed to test using multi-cards. ### Does this PR introduce _any_ user-facing change? None. ### How was this patch tested? `pytest -sv tests/e2e/singlecard/multi-modal/test_internvl.py` - vLLM version: v0.11.0 - vLLM main: vllm-project/vllm@83f478b --------- Signed-off-by: gcanlin <canlinguosdu@gmail.com> Signed-off-by: luolun <luolun1995@cmbchina.com>

### What this PR does / why we need it? As a validation for vllm-project#3664, add end-to-end tests to monitor the InternVL model and ensure its continuous proper operation. This PR is only for single-card. So the models that have more parameters than 8B like 78B are needed to test using multi-cards. ### Does this PR introduce _any_ user-facing change? None. ### How was this patch tested? `pytest -sv tests/e2e/singlecard/multi-modal/test_internvl.py` - vLLM version: v0.11.0 - vLLM main: vllm-project/vllm@83f478b --------- Signed-off-by: gcanlin <canlinguosdu@gmail.com> Signed-off-by: hwhaokun <haokun0405@163.com>

### What this PR does / why we need it? As a validation for vllm-project#3664, add end-to-end tests to monitor the InternVL model and ensure its continuous proper operation. This PR is only for single-card. So the models that have more parameters than 8B like 78B are needed to test using multi-cards. ### Does this PR introduce _any_ user-facing change? None. ### How was this patch tested? `pytest -sv tests/e2e/singlecard/multi-modal/test_internvl.py` - vLLM version: v0.11.0 - vLLM main: vllm-project/vllm@83f478b --------- Signed-off-by: gcanlin <canlinguosdu@gmail.com> Signed-off-by: nsdie <yeyifan@huawei.com>

### What this PR does / why we need it? As a validation for vllm-project#3664, add end-to-end tests to monitor the InternVL model and ensure its continuous proper operation. This PR is only for single-card. So the models that have more parameters than 8B like 78B are needed to test using multi-cards. ### Does this PR introduce _any_ user-facing change? None. ### How was this patch tested? `pytest -sv tests/e2e/singlecard/multi-modal/test_internvl.py` - vLLM version: v0.11.0 - vLLM main: vllm-project/vllm@83f478b --------- Signed-off-by: gcanlin <canlinguosdu@gmail.com>

wangxiyuan · 2026-01-05T03:21:16Z

Any progress? If this PR is still alive, please rebase to main and make CI happy. Thanks

gcanlin · 2026-01-05T03:25:36Z

Any progress? If this PR is still alive, please rebase to main and make CI happy. Thanks

YES, I will refactor this PR later, maybe will need about 2 days.

Yikun · 2026-03-01T14:16:51Z

Any progress?

wangxiyuan · 2026-03-10T01:15:32Z

Please rebase and fix the merge conflicts

wangxiyuan · 2026-04-10T06:26:25Z

no update for long time, close this now. Feel free to reopen if it's still needed.

[Docs] Add InternVL series tutorial for single NPU

07f908a

Signed-off-by: gcanlin <canlinguosdu@gmail.com>

gemini-code-assist bot reviewed Oct 23, 2025

View reviewed changes

Comment thread docs/source/tutorials/single_npu_internvl.md

update

83b7e3d

Signed-off-by: gcanlin <canlinguosdu@gmail.com>

github-actions bot added the documentation Improvements or additions to documentation label Oct 23, 2025

gcanlin added 2 commits October 23, 2025 08:01

fix lint

fe83006

Signed-off-by: gcanlin <canlinguosdu@gmail.com>

Update

94426fc

Signed-off-by: gcanlin <canlinguosdu@gmail.com>

gcanlin mentioned this pull request Oct 27, 2025

[E2E][MM] Add e2e tests for InternVL model #3796

Merged

github-actions bot added the merge-conflicts label Oct 27, 2025

gcanlin mentioned this pull request Oct 29, 2025

[Bug]: vllm推理internvl3的时候，报错 “selfRef expected dtype is DT_INT32 but found DT_BFLOAT16.” #1883

Open

ApsarasX reviewed Nov 3, 2025

View reviewed changes

shen-shanshan mentioned this pull request Jan 13, 2026

[RFC]: Multi-Modal Tasks #3508

Open

28 tasks

wangxiyuan closed this Apr 10, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Docs] Add InternVL series tutorial for single NPU#3664

[Docs] Add InternVL series tutorial for single NPU#3664
gcanlin wants to merge 4 commits intovllm-project:mainfrom
gcanlin:internvl-docs

gcanlin commented Oct 23, 2025 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Oct 23, 2025

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

Yikun commented Oct 24, 2025

Uh oh!

github-actions bot commented Oct 27, 2025

Uh oh!

ApsarasX Nov 3, 2025

Uh oh!

gcanlin Nov 14, 2025

Uh oh!

gcanlin commented Nov 14, 2025

Uh oh!

wangxiyuan commented Jan 5, 2026

Uh oh!

gcanlin commented Jan 5, 2026

Uh oh!

Yikun commented Mar 1, 2026

Uh oh!

wangxiyuan commented Mar 10, 2026

Uh oh!

wangxiyuan commented Apr 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

gcanlin commented Oct 23, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

github-actions bot commented Oct 23, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Yikun commented Oct 24, 2025

Uh oh!

github-actions bot commented Oct 27, 2025

Uh oh!

ApsarasX Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

gcanlin Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

gcanlin commented Nov 14, 2025

Uh oh!

wangxiyuan commented Jan 5, 2026

Uh oh!

gcanlin commented Jan 5, 2026

Uh oh!

Yikun commented Mar 1, 2026

Uh oh!

wangxiyuan commented Mar 10, 2026

Uh oh!

wangxiyuan commented Apr 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

gcanlin commented Oct 23, 2025 •

edited by github-actions bot

Loading