Skip to content

[CI]add internvl ci test #10002

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nakairika wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[CI]add internvl ci test #10002

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/workflows/schedule_nightly_test_a3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -302,6 +302,12 @@ jobs:
- name: hy3-preview
os: linux-aarch64-a3-16
config_file_path: Hy3-preview.yaml
- name: InternVL3.5-38B
os: linux-aarch64-a3-16
config_file_path: InternVL3.5-38B.yaml
- name: InternVL3.5-241B-A28B
os: linux-aarch64-a3-16
config_file_path: InternVL3.5-241B-A28B.yaml
# pytest-driven tests
- name: custom-multi-ops
os: linux-aarch64-a3-16
Expand Down
70 changes: 70 additions & 0 deletions tests/e2e/nightly/single_node/models/configs/InternVL3.5-241B-A28B.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# ==========================================
# Shared Configurations
# ==========================================

_envs: &envs
OMP_NUM_THREADS: "1"
OMP_PROC_BIND: "false"
TASK_QUEUE_ENABLE: "1"
HCCL_OP_EXPANSION_MODE: "AIV"
HCCL_BUFFSIZE: "1536"
PYTORCH_NPU_ALLOC_CONF: "expandable_segments:True"
VLLM_ASCEND_ENABLE_FLASHCOMM1: "1"
VLLM_ASCEND_ENABLE_FUSED_MC2: "1"
VLLM_ASCEND_ENABLE_NZ: "2"
VLLM_ASCEND_BALANCE_SCHEDULING: "1"
SERVER_PORT: "DEFAULT_PORT"

_server_cmd: &server_cmd
- "--quantization"
- "ascend"
- "--no-enable-prefix-caching"
- "--mm-processor-cache-gb"
- "0"
- "--tensor-parallel-size"
- "4"
- "--data-parallel-size"
- "4"
- "--enable-expert-parallel"
- "--port"
- "$SERVER_PORT"
- "--max-model-len"
- "32768"
- "--max-num-batched-tokens"
- "16384"
- "--max-num-seqs"
- "32"
- "--trust-remote-code"
- "--gpu-memory-utilization"
- "0.92"

_benchmarks: &benchmarks
acc:
case_type: accuracy
dataset_path: vllm-ascend/textvqa-lite
request_conf: vllm_api_stream_chat
dataset_conf: textvqa/textvqa_gen_base64
max_out_len: 2048
batch_size: 128
baseline: 83
temperature: 0
top_k: -1
top_p: 1
repetition_penalty: 1
threshold: 5

# ==========================================
# ACTUAL TEST CASES
# ==========================================

test_cases:
- name: "InternVL3.5-241B-A28B"
model: "Eco-Tech/InternVL3.5-241B-A28B-w8a8-QuaRot"
envs:
<<: *envs
server_cmd: *server_cmd
server_cmd_extra:
- "--compilation_config"

@gemini-code-assist gemini-code-assist Bot Jun 4, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The command-line argument for compilation configuration in vLLM is --compilation-config (with a hyphen) rather than --compilation_config (with an underscore). Using the underscore version will result in an unrecognized argument error when starting the server.

Suggested PR Title:

[CI][Test][Feature] Add InternVL3.5 CI tests

Suggested PR Summary:

### What this PR does / why we need it?

This PR adds end-to-end (e2e) nightly single-node CI test configurations for the InternVL3.5 models, specifically `InternVL3.5-241B-A28B` and `InternVL3.5-38B`. These configurations define the environment variables, server commands, and accuracy benchmarks to ensure the models run correctly and meet accuracy thresholds on Ascend NPUs.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

This patch adds test configuration files (`InternVL3.5-241B-A28B.yaml` and `InternVL3.5-38B.yaml`) to be run as part of the nightly CI suite.
      - "--compilation-config"
References
  1. The PR Title and PR Summary must follow the specific formats defined in the Repository Style Guide. (link)

- '{"cudagraph_mode": "FULL_DECODE_ONLY", "cudagraph_capture_sizes": [1,2,4,8,16,24,32]}'
benchmarks:
<<: *benchmarks
67 changes: 67 additions & 0 deletions tests/e2e/nightly/single_node/models/configs/InternVL3.5-38B.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# ==========================================
# Shared Configurations
# ==========================================


_envs: &envs
OMP_NUM_THREADS: "1"
OMP_PROC_BIND: "false"
TASK_QUEUE_ENABLE: "1"
HCCL_OP_EXPANSION_MODE: "AIV"
VLLM_ASCEND_ENABLE_FLASHCOMM1: "1"
PYTORCH_NPU_ALLOC_CONF: "expandable_segments:True"
VLLM_ASCEND_ENABLE_PREFETCH_MLP: "1"
SERVER_PORT: "DEFAULT_PORT"


_server_cmd: &server_cmd
- "--quantization"
- "ascend"
- "--no-enable-prefix-caching"
- "--mm-processor-cache-gb"
- "0"
- "--tensor-parallel-size"
- "2"
- "--port"
- "$SERVER_PORT"
- "--max-model-len"
- "20000"
- "--max-num-batched-tokens"
- "8192"
- "--trust-remote-code"
- "--gpu-memory-utilization"
- "0.9"


_benchmarks: &benchmarks
acc:
case_type: accuracy
dataset_path: vllm-ascend/textvqa-lite
request_conf: vllm_api_stream_chat
dataset_conf: textvqa/textvqa_gen_base64
max_out_len: 2048
batch_size: 128
baseline: 80
temperature: 0
top_k: -1
top_p: 1
repetition_penalty: 1
threshold: 5


# ==========================================
# ACTUAL TEST CASES
# ==========================================


test_cases:
- name: "InternVL3.5-38B"
model: "Eco-Tech/InternVL3.5-38B-w8a8-QuaRot"
envs:
<<: *envs
server_cmd: *server_cmd
server_cmd_extra:
- "--compilation_config"

@gemini-code-assist gemini-code-assist Bot Jun 4, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The command-line argument for compilation configuration in vLLM is --compilation-config (with a hyphen) rather than --compilation_config (with an underscore). Using the underscore version will result in an unrecognized argument error when starting the server.

      - "--compilation-config"

- '{"cudagraph_mode": "FULL_DECODE_ONLY", "cudagraph_capture_sizes": [1,12,16,20,24,32,48,64,68,72,76,80,128]}'
benchmarks:
<<: *benchmarks
Loading