[Model] Introduce verify_and_update_model_config for VerifyAndUpdateConfig.#31131
[Model] Introduce verify_and_update_model_config for VerifyAndUpdateConfig.#31131noooop merged 14 commits intovllm-project:mainfrom
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request refactors the model configuration update mechanism to allow for earlier updates. It introduces a new method verify_and_update_model_config on ModelConfig and migrates several model-specific configuration classes to use it. The changes also include simplifying ModelInfo classes in tests by removing default_pooling_type.
While the refactoring is a good idea, it is incomplete. Several configuration classes in vllm/model_executor/models/config.py have not been migrated to the new mechanism. This will lead to lost functionality for the corresponding models if the old configuration update path is removed. I've left a critical comment detailing the issue and suggesting potential solutions.
noooop
changed the title
noooop
requested review from
DarkLight1337,
ProExpertProg,
WoosukKwon,
hmellor,
houseroad,
mgoin,
robertgshaw2-redhat,
tlrmchlsmth,
yewentao256,
youkaichao and
ywang96
as code owners
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
Signed-off-by: wang.yuqi <noooop@126.com>
DarkLight1337
added
the
ready
…onfig. (vllm-project#31131) Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com>
…onfig. (vllm-project#31131) Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Signed-off-by: dsuhinin <suhinin.dmitriy@gmail.com>
…onfig. (vllm-project#31131) Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com>
2 participants
Purpose
VerifyAndUpdateConfig.verify_and_update_config too late to _set_default_chunked_prefill_and_prefix_caching_args.
Causing the logs to not match the actual behavior, and requiring the use of some non-obvious failback to get the correct results (which sounds like a bug)
e.g.
nvidia/llama-nemotron-embed-1b-v2
nvidia/llama-nemotron-rerank-1b-v2
google/embeddinggemma-300m
Let's introduce verify_and_update_model_config for VerifyAndUpdateConfig to make the logic here clearer.
Test Plan
tests/models/language/pooling_mteb_test/
tests/entrypoints/pooling/
Test Result
pass
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.