build(vllm): consume upstream vllm-proto crate

[JulienDarve]

Summary

Use the official vllm-proto = "=0.1.0" crate for the vLLM sidecar and mock server, removing copied schemas and the local generation script. Centralize the required Tonic/Prost 0.14 dependencies through workspace aliases and retain the shared connection policy for the existing 0.13 consumers.

Rebased onto current main, preserving supported typed sampling controls, skip_special_tokens, priority conversion, and top_k = -1 normalization. LoRA operations remain explicitly unsupported by the mock server.

Native Generate compatibility

Main's #14398 added native sampling JSON fields from the still-open companion vLLM PR #56421. The published protocol crate does not contain those fields. This change deliberately uses the released upstream protocol: the sidecar does not advertise vllm_inference_v1_generate, and aggregated/decode requests carrying extra_args.vllm_tito.sampling_params return an explicit unsupported-request error instead of silently dropping settings. Chat/completions retain supported typed controls. Prefill/encode retain their canonical one-token behavior.

The frontend request types and projection code remain available. Native JSON passthrough can be enabled after an upstream protocol release supplies the payload and capability flag; no local schema fork or panic stub is retained.

Validation

• cargo test -p dynamo-sidecar-common -p dynamo-vllm-sidecar -p dynamo-vllm-mocker --all-targets --locked: 67 passed.
• Clippy with --all-targets --locked -- -D warnings for common, vLLM sidecar/mocker, SGLang sidecar, and TensorRT-LLM sidecar: passed.
• cargo check --manifest-path lib/bindings/python/Cargo.toml --locked: passed.
• cargo fmt --all -- --check, pre-commit on changed files, and CI filter tests: passed (71 filter tests).

Local Rust checks use the host's system protoc with --experimental_allow_proto3_optional. No new GPU smoke run was performed after this rebase; the original PR's pre-rebase live validation is recorded in its earlier description.

Related issues

• DYN-4422
• vLLM #56365: published protocol crate
• vLLM #56421: proposed native sampling extension

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 866ed081-153c-45e5-9021-697c47b3ed56

📥 Commits

Reviewing files that changed from the base of the PR and between f1683cc and 333558e.

⛔ Files ignored due to path filters (2)

• Cargo.lock is excluded by !**/*.lock
• lib/bindings/python/Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (24)

• .github/scripts/test-filters.js
• .github/workflows/copyright-check.ps1
• .pre-commit-config.yaml
• Cargo.toml
• lib/mocker/servers/vllm/Cargo.toml
• lib/mocker/servers/vllm/src/server.rs
• lib/mocker/servers/vllm/src/server_request.rs
• lib/mocker/servers/vllm/src/server_tests.rs
• lib/sidecar/Dockerfile
• lib/sidecar/common/Cargo.toml
• lib/sidecar/common/src/error.rs
• lib/sidecar/common/src/lib.rs
• lib/sidecar/vllm/Cargo.toml
• lib/sidecar/vllm/README.md
• lib/sidecar/vllm/build.rs
• lib/sidecar/vllm/deploy/agg.yaml
• lib/sidecar/vllm/proto/README.md
• lib/sidecar/vllm/proto/control.proto
• lib/sidecar/vllm/proto/inference.proto
• lib/sidecar/vllm/src/client.rs
• lib/sidecar/vllm/src/convert.rs
• lib/sidecar/vllm/src/lib.rs
• lib/sidecar/vllm/src/proto.rs
• lib/sidecar/vllm/src/tests.rs

💤 Files with no reviewable changes (5)

• lib/sidecar/vllm/src/proto.rs
• lib/sidecar/vllm/build.rs
• lib/sidecar/vllm/proto/README.md
• lib/sidecar/vllm/proto/control.proto
• lib/sidecar/vllm/proto/inference.proto

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

Walkthrough

The vLLM sidecar now uses the pinned vllm-proto crate instead of vendored protobuf generation. Tonic v14 compatibility was added. Mock and sidecar services explicitly reject unsupported LoRA operations.

Changes

vLLM protocol integration

Layer / File(s)	Summary
External protocol crate adoption Cargo.toml, lib/sidecar/vllm/..., .github/..., .pre-commit-config.yaml	The sidecar uses the pinned vllm-proto crate. Local protobuf files, generated bindings, and the build script were removed. Compatibility documentation and validation filters were updated.
Tonic v14 compatibility lib/sidecar/common/..., lib/sidecar/vllm/src/client.rs	The common library provides feature-gated Tonic v14 status and transport adapters. The vLLM client uses these adapters.
LoRA capability handling lib/mocker/servers/vllm/..., lib/sidecar/vllm/src/convert.rs, lib/sidecar/vllm/src/tests.rs	Mock and sidecar services report no LoRA support, reject LoRA requests, and return UNIMPLEMENTED for LoRA RPCs. A regression test covers request rejection.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: ⚪ Minimal · up to 33355

The protocol migration, Tonic v14 adapters, and unsupported-LoRA behavior are internally aligned and ready to merge.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 15.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 20 functions across 10 files. (9 skipped:…	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly identifies the main change: the vLLM integration now consumes the upstream vllm-proto crate.
Description check	✅ Passed	The description explains the migration, compatibility decisions, unsupported features, validation results, and related issues. It does not use the template headings exactly and omits an explicit revie…

Full details: Docstring Coverage

Explanation

Docstring coverage is 15.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 20 functions across 10 files. (9 skipped: 9 unsupported.)

• Fix all pre-merge checks with AI

---

_{Comment @coderabbitai help to get the list of available commands.}

[dmitry-tokarev-nv]

Reviewed at 591a939 against merge-base 702f532. Read DYN-4422 and the two linked vLLM PRs before the diff, then checked the PR out into a worktree and verified by execution on macOS ARM64 / Rust 1.96.1 (nothing here needs a GPU).

What I verified rather than read

• Wire compatibility is preserved. I pulled vllm-proto 0.1.0 from crates.io and diffed its .proto files against the deleted vendored copies. inference.proto differs by exactly one field — Dynamo's native_sampling_params_json = 16 is gone, upstream's lora_name = 15 is new. control.proto differs only by additions (the three LoRA RPCs, ServerInfo.max_loras = 10, ModelInfo.supports_lora = 22) plus the removal of Dynamo's supports_native_sampling_params_json = 12. Every delta is field-number-disjoint and additive, and the sidecar sends lora_name: "", a proto3 default that is never serialized — so a v0.28.0 engine sees a byte-identical request. The agg.yaml comment claim holds.
• The crate needs no protoc. vllm-proto's build.rs compiles its bundled protos with protox (pure Rust), so protobuf-compiler in the Dockerfile is now only there for the SGLang and TRT-LLM sidecars — which is what the rewritten comment says. License is Apache-2.0, inside deny.toml's allow list.
• cargo test -p dynamo-sidecar-common -p dynamo-vllm-sidecar -p dynamo-vllm-mocker --all-targets --locked → 67 passed, 0 failed, and node .github/scripts/test-filters.js → 71/71. Both match the PR body exactly.
• The v14 status adapter is correct. status.code() as i32 → tonic::Code::from_i32 round-trips all 17 gRPC codes to an identical ErrorType and an identical message string versus the 0.13 path. Details in the P3 on error.rs.
• The v14 connection pool is genuinely exercised. include!("transport.rs") carries the test module in with it, so v14::tests::startup_deadline_caps_connection_retries runs alongside transport::tests::… — the two pools are the same source and cannot drift.
• Dropping the native-Generate capability costs nothing against a released vLLM. supports_native_sampling_params_json was Dynamo's own field 12, absent from upstream control.proto, so no released vLLM ever set it: the capability was already never advertised and /inference/v1/generate already 404'd on main for sidecar workers. With one exception, below, this PR removes nothing that worked.
• LoRA really is rejected, not silently downgraded. validate_request (convert.rs:797) already errors on routing.lora_name, so the hardcoded lora_name: String::new() is unreachable-by-construction rather than a silent switch to base weights.

CI

DGDR Deploy Test / CPU / profiling and the deploy-status-check aggregator it feeds are red. Triaged as unrelated and not reported as a finding: it fails in wait_for_phase_at_least on a Kubernetes profiling Job for backend='vllm', nothing in this diff reaches the profiler, the same job is green on main at four surrounding commits and on another PR from the same window, and rust-tests (.), rust-clippy (.), dynamo-sidecar / Build multi-arch cpu, vllm-runtime / Test, and all four vllm Deploy Test legs are green at this SHA. Worth a re-run.

Findings — one P2, two P3, none blocking

1. P2 — lib/sidecar/vllm/src/convert.rs:33. The new guard also breaks the v1.4-frontend compatibility path that normalize_response_options still exists to serve. A/B measured with a control. This one is a decision, not a typo: narrow the guard, or delete the shim and document the break.
2. P3 — .github/scripts/test-filters.js:130. The replaced case was the only test of the lib/sidecar/**/*.proto → rust rule, which still has two live consumers. Mutation-tested in both directions; verified drop-in suggestion attached.
3. P3 — lib/sidecar/common/src/error.rs:92. status_to_dynamo_v14 is untested. I proved it correct today, so this is only about pinning it across a future tonic bump. Verified snippet attached.

Deliberately not raised

The prost-types skew between the root and bindings lockfiles (0.14.3 vs 0.14.4) — I checked the base tree and it already skews the same way, along with tonic 0.14.6 already being in both trees, so neither is introduced here. Likewise the stale .gitattributes:22 entry pointing at the long-gone lib/backend/vllm-sidecar/proto/vllm_grpc.proto, and the dynamo-frontend:1.3.0 pin in deploy/agg.yaml: both pre-existing.

Where verification stopped

No live serving run. Everything above is a local cargo test, a crate-source diff, or a CI-log read on ARM64 macOS — so "wire compatible" here means the schemas and the encoder agree, not measured against a running engine. The PR body is equally explicit that no post-rebase GPU smoke run was done, and the pre-rebase validation lives in the earlier description. Given the protos differ by two additive fields neither side populates, I did not judge an engine run necessary to approve, but it is the gap.

Approving: zero P0/P1, three combined P2+P3.

[JulienDarve]

I reviewed the rebased changes and CI. Rust and sidecar/vLLM checks passed at the PR head; deployment profiling failed from node DiskPressure before the container started, so that needs an infrastructure rerun once disk capacity is healthy.

The legacy sampling guard fix, restored proto filter fixture, and README conflict resolution are implemented locally. The README retains main's vLLM-Omni guidance, and full native Generate capability remains disabled. The regression fails before the guard fix and passes afterward; all 68 common/sidecar/mock tests, all 71 filter tests, Clippy, formatting, and repository hooks pass.

The fixes and merge resolution are staged together, uncommitted and not pushed. I replied to and resolved the three review conversations; the current PR head does not yet contain these local fixes.