[Frontend][4/n] Improve pooling entrypoints | pooling.

[noooop]

Purpose

Improve pooling entrypoints

Test Plan

keep ci green

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

This pull request refactors the pooling IO processor architecture by introducing task-specific processors and a plugin-based system, moving logic out of the main LLM engine. Key changes include the addition of token-level classification and embedding tasks, and the delegation of pre/post-processing to specialized classes. Feedback highlights several critical bugs, including a missing parameter in the offline context and an incorrect attribute access on a sequence object. Further improvements are needed to remove duplicate code blocks, fix malformed function signatures, and provide more descriptive error messages for better user feedback.

[mergify]

Hi @noooop, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?

mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

[noooop]

/gemini review

[gemini-code-assist]

Code Review

This pull request refactors the pooling and IO processor architecture to improve modularity and consistency between offline and online entrypoints. Key changes include the introduction of specialized PoolingIOProcessor implementations and the removal of direct io_processor dependencies from core serving classes. The review feedback highlights several critical issues: the use of assert for public API input validation which could lead to regressions, a logic bug in the plugin processor that breaks deprecated compatibility by unconditionally overwriting responses, and a potential KeyError when handling unsupported pooling tasks in the serving layer.

[noooop]

/gemini review

[gemini-code-assist]

Code Review

This pull request significantly refactors the handling of IO processors and pooling tasks within the vLLM serving infrastructure. Key changes include centralizing IO processor management within a new PoolingServing class hierarchy, removing direct io_processor attributes from core engine and rendering components, and introducing specialized PoolingIOProcessor implementations for different tasks, including a new 'plugin' task. The review highlights several critical issues: a regression in the offline encode API's inference of the 'plugin' task for {"data": ...} prompts, the bypassing of error checks for missing IOProcessor plugins due to a dummy class registration, and the presence of assertions that should be replaced with explicit ValueErrors for improved error handling in production.