Skip to content

[Rust Frontend] Drop tensorizer_config_dict from the LoRA request wire layout - #29

Closed
richard-for-bugen[bot] wants to merge 1 commit into
mainfrom
richard/parity-fix-lora-request-drop-tensorizer-config
Closed

richard-for-bugen[bot] wants to merge 1 commit into
mainfrom
richard/parity-fix-lora-request-drop-tensorizer-config

Conversation

@richard-for-bugen

Copy link
Copy Markdown

Motivation

vllm-project#60024 ("[LoRA] Remove tensorizer", merged 2026-10-05) removed tensorizer_config_dict from Python's LoRARequest (vllm/lora/request.py). It is a msgspec array_like=True, omit_defaults=True struct, so it is encoded positionally and every field after the removed one moved down by one: the wire layout is now [lora_name, lora_int_id, lora_path, base_model_name, load_inplace, is_3d_lora_weight].

The Rust mirror LoraRequest (rust/src/engine-core-client/src/protocol/lora.rs) still carried tensorizer_config_dict: Option<OpaqueValue> at position 4. Rust therefore encoded seven positional fields and sent null where the engine now expects load_inplace: bool:

msgspec.ValidationError: Expected `bool`, got `null` - at `$[4]`

Concrete effect: with VLLM_USE_RUST_FRONTEND=1, vllm serve <model> --enable-lora --lora-modules Alice=... dies at startup (failed to load LoRA adapter Alice from --lora-modules), and /v1/load_lora_adapter, the gRPC LoadLora and per-request lora_request hit the same decode error, since all of them encode a LoraRequest.

This PR removes the field so the Rust struct matches Python's six-field layout. Rust never decodes a LoraRequest from Python in production (only the mock engine and tests decode what Rust encoded, so the derived Deserialize_tuple stays for them), and no other Rust struct mirrors anything else changed by vllm-project#60024 (the rest of it is engine-side tensorizer/PEFT code with no wire presence).

Ownership and layering

The consumer is the engine core's msgspec decoder of EngineCoreRequest.lora_request and the add_lora utility argument, so the owner is the wire struct in vllm-engine-core-client (protocol/lora.rs); the two construction sites (vllm-server LoraManager, a vllm-llm test) only drop the None initializer. No compatibility shim for pre-vllm-project#60024 engines: frontend and engine ship together.

Regression test

The existing python_msgpack_fixtures_match_rust_encoding mechanism (src/tests/python_compat.py) had no LoRA coverage, and its lora_request slot was an untyped object. This PR adds a Python LoRARequest mirror and an EngineCoreRequest fixture with every LoRA field non-default (so omit_defaults trims nothing and the full positional layout is pinned), and asserts both directions: Python wire -> Rust struct, and Rust struct -> wire for the lora_request slot of the request. The mirror's bytes were checked to equal the real vllm.lora.request.LoRARequest encoding of the same values.

Validation

  • cargo fmt --check, cargo sort --check, cargo deny check bans, cargo clippy --workspace --all-targets --all-features --locked -D warnings: clean.
  • cargo nextest run --workspace --all-features --locked --no-fail-fast: 2255 passed, 13 skipped.
  • Red-green: restoring tensorizer_config_dict (with the test's expected value adjusted to compile) makes python_msgpack_fixtures_match_rust_encoding fail: the Rust decode of the Python fixture yields tensorizer_config_dict: Some(true), is_3d_lora_weight: false, i.e. fields shifted; with the fix it passes.
  • End to end (1 GPU, --gpu-memory-utilization 0.12 via a pytest plugin because the GPU is shared), pytest entrypoints/scale_out/token_in_token_out/test_serving_tokens.py -k lora (test_generate_with_lora_adapter, server with --enable-lora --lora-modules Alice=... Bob=...):
    • before (base aa31fa038f): 1 error, server dies at startup with Expected bool, got null - at $[4].
    • after (this PR): 1 passed (Alice and Bob load at startup, /v1/models lists them, generation through the adapter succeeds). Python frontend on the same test: 1 passed.

Codex review

Verdict: approve. No blocking findings, no nits. It confirmed the problem against the Python decoder, that both Rust consumers (generation via EngineCoreRequest encoding, adapter loading via add_lora utility args) go through the changed serializer, that the new test fails on the base with the shifted fields and passes on HEAD (140 engine-core-client tests), and that the fixture bytes match the real LoRARequest for default and mixed-boolean values. Its only remark was that the E2E result and workspace-wide checks were author-reported, which this body now carries.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XTSySQLs9bSuTKD9ScKhDb

…e layout

vllm-project#60024 removed `tensorizer_config_dict` from the
array_like `LoRARequest`, shifting `load_inplace` and `is_3d_lora_weight`
down by one position. The Rust mirror still encoded the removed field, so
the engine decoded `null` into `load_inplace: bool` and rejected every
LoRA request (`Expected bool, got null - at $[4]`), including the
`--lora-modules` adapters loaded at startup.

Remove the field and extend the Python msgpack fixture to pin the full
six-field layout in both directions.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: richard-for-bugen[bot] <342439978+richard-for-bugen[bot]@users.noreply.github.com>
@BugenZhao

Copy link
Copy Markdown
Owner

Duplicated with vllm-project#63295

@BugenZhao BugenZhao closed this Oct 23, 2026
@richard-for-bugen
richard-for-bugen Bot deleted the richard/parity-fix-lora-request-drop-tensorizer-config branch October 23, 2026 09:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant