[CI Failure] Fix NomicBert max_model_len validation

[noooop]

Purpose

• Fix NomicBert max_model_len validation.

• rm VllmConfig.recalculate_max_model_len.

It was introduced in #18755 and is only used in the NomicBert max_model_len validation. Deleting it prevents potential misuse.

Adapted from #31632

Test Plan

tests/models/language/pooling_mteb_test/test_nomic.py

Test Result

pass

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

This pull request correctly fixes a bug in the max_model_len validation for NomicBert models, particularly when context extension is used. The core of the fix involves updating the cached derived_max_model_len before validation, which is a crucial and correct change. Additionally, the PR includes a nice refactoring by removing the VllmConfig.recalculate_max_model_len method and adjusting the NomicBertModelConfig method signature to reduce dependencies. The changes are sound, improve code clarity, and I have no further comments.

[noooop]

hi @charlotte12l

The functionality of model_arch_config_convertor and verify_and_update_model_config is quite similar. Using verify_and_update_model_config can help avoid placing the logic for NomicBertModel in two separate locations.

[noooop]

hi @charlotte12l

Now model_arch_config_convertor and verify_and_update_model_config have the same functionality.

Could you have a good way to combine them?

---

verify_and_update_model_config Introduce in #31131
VerifyAndUpdateConfig Introduce in #20012

[charlotte12l]

Moving the discussion from #31632 (comment) to here.

For context, model_arch_config was primarily for explicitly defining which fields will be used by vllm engine, so users can work with an arbitrary hf_config easily.
If I understand correctly, verify_and_update_config is for vllm_config and includes cache_config, parallel_config, etc.

My initial thoughts are:

• We could mainly focus on consolidating the update/verify about the hf_config , while leave the other parts unchanged.

For those logics that update hf_config:

• move the updates for the fields -- e.g. hidden_size, num_hidden_layers, max_len, rope related (note that I haven't include it into model_arch_config, but that's a plan to do ) etc. - -- that are used by vllm engine into model_arch_config_convertor
• for fields updates that's only impact the model definition(e.g. layer_norm_eps,

  vllm/vllm/model_executor/models/config.py

  Line 136 in 268b1c5

  config.layer_norm_eps = config.layer_norm_epsilon

  ) , we could just let the corresponding model definition to read hf_config.layer_norm_epsilon . Noob question for this: why we need to update hf_config.layer_norm_eps in the first place, is somewhere in vllm engine checking layer_norm_eps?
• For some tricky fields like method (

  vllm/vllm/model_executor/models/config.py

  Line 243 in 268b1c5

  model_config.hf_config.method = "from_2_way_softmax"

  ), is_causal (

  vllm/vllm/model_executor/models/config.py

  Line 100 in 268b1c5

  hf_config.is_causal = False

  ), I need to take a deeper look for the design.

Appreciate any suggestions!

[noooop]

Moving the discussion from #31632 (comment) to here.

For context, model_arch_config was primarily for explicitly defining which fields will be used by vllm engine, so users can work with an arbitrary hf_config easily. If I understand correctly, verify_and_update_config is for vllm_config and includes cache_config, parallel_config, etc.

My initial thoughts are:

• We could mainly focus on consolidating the update/verify about the hf_config , while leave the other parts unchanged.

For those logics that update hf_config:

• move the updates for the fields -- e.g. hidden_size, num_hidden_layers, max_len, rope related (note that I haven't include it into model_arch_config, but that's a plan to do ) etc. - -- that are used by vllm engine into model_arch_config_convertor
• for fields updates that's only impact the model definition(e.g. layer_norm_eps,

  vllm/vllm/model_executor/models/config.py

  Line 136 in 268b1c5

  config.layer_norm_eps = config.layer_norm_epsilon

  
  ) , we could just let the corresponding model definition to read hf_config.layer_norm_epsilon . Noob question for this: why we need to update hf_config.layer_norm_eps in the first place, is somewhere in vllm engine checking layer_norm_eps?
• For some tricky fields like method (

  vllm/vllm/model_executor/models/config.py

  Line 243 in 268b1c5

  model_config.hf_config.method = "from_2_way_softmax"

  
  ), is_causal (

  vllm/vllm/model_executor/models/config.py

  Line 100 in 268b1c5

  hf_config.is_causal = False

  
  ), I need to take a deeper look for the design.

Appreciate any suggestions!

Unfortunately, is_causal, hf_config.method, and others were added by me......

I will gradually mv these field into model_arch_config_convertor.py

---

I'm not sure whether to directly add these fields to ModelArchitectureConfig or to create ModelArchitectureConfigPooling.

[charlotte12l]

Thank you! Let's still use ModelArchitectureConfig but maybe putting all pooling related fields in a class. Which parts from hf_config are important for vllm engine if it's a pooling model, is_causal, hidden_act, pooling? I didn't find the usage of method though.

[mergify]

Hi @noooop, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[AndreasKaratzas]

cc @DarkLight1337