Repository navigation
[Rust Frontend] Drop tensorizer_config_dict from the LoRA request wire layout - #29
Closed
richard-for-bugen[bot] wants to merge 1 commit into
Closed
richard-for-bugen[bot] wants to merge 1 commit into
richard-for-bugen[bot] wants to merge 1 commit into
Conversation
…e layout vllm-project#60024 removed `tensorizer_config_dict` from the array_like `LoRARequest`, shifting `load_inplace` and `is_3d_lora_weight` down by one position. The Rust mirror still encoded the removed field, so the engine decoded `null` into `load_inplace: bool` and rejected every LoRA request (`Expected bool, got null - at $[4]`), including the `--lora-modules` adapters loaded at startup. Remove the field and extend the Python msgpack fixture to pin the full six-field layout in both directions. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: richard-for-bugen[bot] <342439978+richard-for-bugen[bot]@users.noreply.github.com>
BugenZhao
approved these changes
Oct 22, 2026
This was referenced Oct 22, 2026
Owner
|
Duplicated with vllm-project#63295 |
richard-for-bugen
Bot
deleted the
richard/parity-fix-lora-request-drop-tensorizer-config
branch
October 23, 2026 09:25
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
vllm-project#60024 ("[LoRA] Remove tensorizer", merged 2026-10-05) removed
tensorizer_config_dictfrom Python'sLoRARequest(vllm/lora/request.py). It is a msgspecarray_like=True, omit_defaults=Truestruct, so it is encoded positionally and every field after the removed one moved down by one: the wire layout is now[lora_name, lora_int_id, lora_path, base_model_name, load_inplace, is_3d_lora_weight].The Rust mirror
LoraRequest(rust/src/engine-core-client/src/protocol/lora.rs) still carriedtensorizer_config_dict: Option<OpaqueValue>at position 4. Rust therefore encoded seven positional fields and sentnullwhere the engine now expectsload_inplace: bool:Concrete effect: with
VLLM_USE_RUST_FRONTEND=1,vllm serve <model> --enable-lora --lora-modules Alice=...dies at startup (failed to load LoRA adapterAlicefrom --lora-modules), and/v1/load_lora_adapter, the gRPCLoadLoraand per-requestlora_requesthit the same decode error, since all of them encode aLoraRequest.This PR removes the field so the Rust struct matches Python's six-field layout. Rust never decodes a
LoraRequestfrom Python in production (only the mock engine and tests decode what Rust encoded, so the derivedDeserialize_tuplestays for them), and no other Rust struct mirrors anything else changed by vllm-project#60024 (the rest of it is engine-side tensorizer/PEFT code with no wire presence).Ownership and layering
The consumer is the engine core's msgspec decoder of
EngineCoreRequest.lora_requestand theadd_lorautility argument, so the owner is the wire struct invllm-engine-core-client(protocol/lora.rs); the two construction sites (vllm-serverLoraManager, avllm-llmtest) only drop theNoneinitializer. No compatibility shim for pre-vllm-project#60024 engines: frontend and engine ship together.Regression test
The existing
python_msgpack_fixtures_match_rust_encodingmechanism (src/tests/python_compat.py) had no LoRA coverage, and itslora_requestslot was an untypedobject. This PR adds a PythonLoRARequestmirror and anEngineCoreRequestfixture with every LoRA field non-default (soomit_defaultstrims nothing and the full positional layout is pinned), and asserts both directions: Python wire -> Rust struct, and Rust struct -> wire for thelora_requestslot of the request. The mirror's bytes were checked to equal the realvllm.lora.request.LoRARequestencoding of the same values.Validation
cargo fmt --check,cargo sort --check,cargo deny check bans,cargo clippy --workspace --all-targets --all-features --locked -D warnings: clean.cargo nextest run --workspace --all-features --locked --no-fail-fast: 2255 passed, 13 skipped.tensorizer_config_dict(with the test's expected value adjusted to compile) makespython_msgpack_fixtures_match_rust_encodingfail: the Rust decode of the Python fixture yieldstensorizer_config_dict: Some(true), is_3d_lora_weight: false, i.e. fields shifted; with the fix it passes.--gpu-memory-utilization 0.12via a pytest plugin because the GPU is shared),pytest entrypoints/scale_out/token_in_token_out/test_serving_tokens.py -k lora(test_generate_with_lora_adapter, server with--enable-lora --lora-modules Alice=... Bob=...):aa31fa038f): 1 error, server dies at startup withExpected bool, got null - at $[4].AliceandBobload at startup,/v1/modelslists them, generation through the adapter succeeds). Python frontend on the same test: 1 passed.Codex review
Verdict: approve. No blocking findings, no nits. It confirmed the problem against the Python decoder, that both Rust consumers (generation via
EngineCoreRequestencoding, adapter loading viaadd_lorautility args) go through the changed serializer, that the new test fails on the base with the shifted fields and passes on HEAD (140 engine-core-client tests), and that the fixture bytes match the realLoRARequestfor default and mixed-boolean values. Its only remark was that the E2E result and workspace-wide checks were author-reported, which this body now carries.🤖 Generated with Claude Code
https://claude.ai/code/session_01XTSySQLs9bSuTKD9ScKhDb