Skip to content

[CI] Reorganization pooling_mteb_test#31265

Merged
DarkLight1337 merged 2 commits intovllm-project:mainfrom
noooop:reorganization_pooling_mteb_test
Dec 24, 2025
Merged

[CI] Reorganization pooling_mteb_test#31265
DarkLight1337 merged 2 commits intovllm-project:mainfrom
noooop:reorganization_pooling_mteb_test

Conversation

@noooop
Copy link
Collaborator

@noooop noooop commented Dec 24, 2025

Purpose

  1. rm CLSPoolingModelInfo & LASTPoolingModelInfo because this name is confusing

Following #26623, we test pooling_type, attn_type, is_prefix_caching_supported, is_chunked_prefill_supported

  1. split mteb_utils to mteb_embed_utils.py & mteb_score_utils.py

Test Plan

tests/models/language/pooling_mteb_test/
tests/entrypoints/pooling/

Test Result

pass


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

Signed-off-by: wang.yuqi <noooop@126.com>
@mergify mergify bot added the qwen Related to Qwen models label Dec 24, 2025
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request is a well-executed refactoring of the MTEB pooling tests. The key changes include splitting mteb_utils.py into more focused mteb_embed_utils.py and mteb_score_utils.py files, which improves code organization. Additionally, the removal of specialized CLSPoolingModelInfo and LASTPoolingModelInfo classes in favor of adding explicit configuration fields like pooling_type, attn_type, is_prefix_caching_supported, and is_chunked_prefill_supported to the base ModelInfo and RerankModelInfo dataclasses is a significant improvement. This makes the test configurations clearer and more explicit. The tests have been updated to assert these new properties, which enhances the test coverage for model configurations. Overall, these changes improve the clarity, maintainability, and robustness of the test suite. The refactoring appears solid, and I found no issues.

@noooop noooop marked this pull request as ready for review December 24, 2025 10:43
@chatgpt-codex-connector
Copy link

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.

@noooop noooop added the ready ONLY add when PR is ready to merge/full CI is needed label Dec 24, 2025
@DarkLight1337 DarkLight1337 merged commit 1ff67df into vllm-project:main Dec 24, 2025
22 of 23 checks passed
@noooop noooop deleted the reorganization_pooling_mteb_test branch December 24, 2025 16:22
yiliu30 pushed a commit to yiliu30/vllm-fork that referenced this pull request Dec 30, 2025
Signed-off-by: wang.yuqi <noooop@126.com>
dsuhinin pushed a commit to dsuhinin/vllm that referenced this pull request Jan 21, 2026
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: dsuhinin <suhinin.dmitriy@gmail.com>
ItzDEXX pushed a commit to ItzDEXX/vllm that referenced this pull request Feb 19, 2026
Signed-off-by: wang.yuqi <noooop@126.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

qwen Related to Qwen models ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants