[Mypy Fix] Mypy fix for "vllm/model_executor/models/[tT]" - #53466
Conversation
Continues the split in vllm-project#48490, after [aA][bB] (vllm-project#48977), [cC][dD] (vllm-project#52003) and [eE][fF] (vllm-project#53381). Fixes the 10 mypy errors under `vllm/model_executor/models/[tT]` so the group can be enforced. - `terratorch.py`: annotate `loaded_buffers` as `list[str]`. - `telechat2.py`: narrow `layer.self_attn` / `layer.mlp` to `LlamaAttention` and `LlamaMLP` before touching their submodules. `Module.__getattr__` is typed `Tensor | Module`, so the attribute chain could not be checked. The layer types are fixed: `TeleChat2Model` does not pass `layer_type`, so `LlamaModel` uses `LlamaDecoderLayer`, whose `attn_layer_type` defaults to `LlamaAttention` and whose `mlp` is always `LlamaMLP`. - `telechat2.py`: drop the bias with `register_parameter("bias", None)` instead of assigning `None` to a `Parameter`-typed attribute. This is the idiom `linear.py` itself uses for bias-less layers and is exactly what `nn.Module.__setattr__` did for the previous assignment. - `telechat2.py`: `attribute_map` is a `ClassVar` on `PretrainedConfig`; the instance-level shadow is intentional, so it is marked `type: ignore[misc]` rather than restructured. `EXCLUDE` entries are prefix patterns, so `models/[tT]` also covered the `models/transformers` package, which still has errors. The entry is narrowed to `models/transformers` rather than dropped, which enforces the three `[tT]` files while leaving that package excluded. No behavior change. Signed-off-by: Jensen Chen <a1043904820@gmail.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
@yewentao256 continuing the One thing worth flagging: Verified locally with |
| # `[tT]` is otherwise clean; the `transformers` package is still excluded. | ||
| r"vllm/model_executor/models/transformers", |
There was a problem hiding this comment.
| # `[tT]` is otherwise clean; the `transformers` package is still excluded. | |
| r"vllm/model_executor/models/transformers", | |
| r"vllm/model_executor/models/transformers", |
Comment not needed
| # `attribute_map` is a ClassVar on `PretrainedConfig`; shadowing it on the | ||
| # instance is intentional here so LlamaModel can read TeleChat2's fields. | ||
| vllm_config.model_config.hf_config.attribute_map = { # type: ignore[misc] |
There was a problem hiding this comment.
| # `attribute_map` is a ClassVar on `PretrainedConfig`; shadowing it on the | |
| # instance is intentional here so LlamaModel can read TeleChat2's fields. | |
| vllm_config.model_config.hf_config.attribute_map = { # type: ignore[misc] | |
| vllm_config.model_config.hf_config.attribute_map = { # type: ignore[misc] |
comment not needed
Signed-off-by: Jensen Chen <a1043904820@gmail.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Thanks for the review! Both comments dropped in 8e3e891. Re-verified after the change: |
|
✅ @ZHIHANCHEN03, CI is now available for this PR.
|
|
/ci retry |
|
✅ No earlier CI build exists for this PR. Use |
|
/ci run |
|
✅ Triggered Buildkite CI #85436 for commit |
|
/ci run |
|
✅ Triggered Buildkite CI #85649 for commit |
…ct#53466) Signed-off-by: Jensen Chen <a1043904820@gmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
…ct#53466) Signed-off-by: Jensen Chen <a1043904820@gmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Signed-off-by: mikeshawcode <michaelwshaw2@gmail.com>
…ct#53466) Signed-off-by: Jensen Chen <a1043904820@gmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Purpose
Mypy fix for
vllm/model_executor/models/[tT], continuing the split in #48490 after[aA][bB] (#48977), [cC][dD] (#52003) and [eE][fF] (#53381).
10 errors across
telechat2.pyandterratorch.py:terratorch.py— annotateloaded_buffersaslist[str].telechat2.py— narrowlayer.self_attn/layer.mlptoLlamaAttentionandLlamaMLPbefore touching their submodules.Module.__getattr__is typedTensor | Module, so the attribute chain could not be checked (8 of the 10 errors).The layer types are fixed here:
TeleChat2Modeldoes not passlayer_type, soLlamaModelusesLlamaDecoderLayer, whoseattn_layer_typedefaults toLlamaAttentionand whosemlpis alwaysLlamaMLP.telechat2.py— drop the bias withregister_parameter("bias", None)instead ofassigning
Noneto aParameter-typed attribute. This is the idiomlinear.pyitself uses for bias-less layers (e.g.
linear.pyL373/L518/L1611), and it isexactly what
nn.Module.__setattr__already did for the previous assignment.telechat2.py—attribute_mapis aClassVaronPretrainedConfig; theinstance-level shadow is intentional, so it is marked
type: ignore[misc]ratherthan restructured.
No behavior change.
Note on the
EXCLUDEentryEXCLUDEentries are prefix patterns (re.compile(f"^{'|'.join(EXCLUDE)}.*")), somodels/[tT]also covered themodels/transformerspackage, which still has errors.Simply dropping the entry fails the full-repo run. The entry is therefore narrowed to
models/transformers, which enforces the three[tT]files and leaves that packageexcluded.
models/transformersis the only subdirectory undermodels/, so earlierletter groups did not hit this.
Duplicate-work check
Searched open PRs for
mypyin the title and formodel_executor/modelsmypy work.The only open mypy PRs are #45296 (kimi /
[kK]), #51043 (tests/directories) and#30960 (mypy → ty migration); none touch
[tT].AI assistance disclosure: Claude Code assisted with the audit, implementation and
review. I reviewed and understand all changed lines and validated the behavior locally.
Test Plan
# the four versions CI runs, on the whole repo pre-commit run mypy-3.10 --all-files --hook-stage manual pre-commit run mypy-3.11 --all-files --hook-stage manual pre-commit run mypy-3.12 --all-files --hook-stage manual pre-commit run mypy-3.13 --all-files --hook-stage manual pre-commit run --files tools/pre_commit/mypy.py \ vllm/model_executor/models/telechat2.py \ vllm/model_executor/models/teleflm.py \ vllm/model_executor/models/terratorch.pyTest Result
Before the change the same three files reported 10 errors; the naive version of this
patch (dropping the
[tT]entry outright) fails the full-repo run withFound 145 errors in 12 files, which is what motivated narrowing the entry instead.I also verified that
register_parameter("bias", None)is exactly equivalent to theprevious
bias = Noneassignment — samebiasattribute, same_parametersentry,same
state_dict()andnamed_parameters()— sincenn.Module.__setattr__routes aNoneassigned over a registered parameter throughregister_parameter.Model-level tests for TeleChat2/TeleFLM/Terratorch were not run locally: they need real
weights, and this was developed on an Apple Silicon CPU build.
terratorch.pyalsoimports the optional
terratorchpackage, which is not installed locally; mypy checks itunder
--follow-imports silent, unaffected.