Internvl and MMMU dataset evaluation

[Fxycst1213]

Motivation

1.Based on the #3351 branch, I implemented the internvl model with the language part from Qwen.
2.When evaluating the model using the MMMU dataset, we encountered some issues with multi-image inputs. The generated prompt was incomplete. We conducted a comparison using VLMEvalKIT, and the results are as follows.
VLMevalKIT:

bench_hf:

Additionally, different models require different prompt modifications. For example, InternVL requires the use of "Image-i" as a separator for multi-image inputs.When evaluating the InternVL model using bench_hf, I found that its prompt differs from the one generated by VLMEvalKIT.The correct prompt format should be as follows:

The Qwen2-VL-7B model performed lower compared to the test results on the VLMevalKIT.

Modifications

Added the InternVL model and conducted tests on it. After modifying the MMMU evaluation script, we evaluated InternVL-38B and Qwen2VL-38B, and updated the results in the README.

InternVL38B

VLMevalKIT

bench_hf

bench_sglang

Qwen2-VL-7B

VLMevalKIT

bench_hf

bench_sglang

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

[tonylt]

@Fxycst1213 Well done, maybe you can update qwen2-vl and internvl's mmmu accuracy results in #4456

[Fxycst1213]

@zhaochenyang20 I have fixed some bugs. Could you help me test it again?

[zhaochenyang20]

@zhaochenyang20 I have fixed some bugs. Could you help me test it again?

reruned. thanks! if you need immediate rerun, ping me in wechat.

[github-actions]

This pull request has been automatically closed due to inactivity. Please feel free to reopen it if needed.