Skip to content

build(vllm): consume protocol SDKs from Buf - #12980

Draft
connorcarpenter15 wants to merge 14 commits into
mainfrom
build/vllm-bsr-proto
Draft

build(vllm): consume protocol SDKs from Buf#12980
connorcarpenter15 wants to merge 14 commits into
mainfrom
build/vllm-bsr-proto

Conversation

@connorcarpenter15

@connorcarpenter15 connorcarpenter15 commented Aug 10, 2026

Copy link
Copy Markdown
Member

Overview

This build layer removes the vendored vLLM protocol and consumes the generated Rust SDKs directly from the Buf Schema Registry.

Details

  • Install the pinned vllm-project_vllm_community_neoeinstein-prost and vllm-project_vllm_community_neoeinstein-tonic crates for BSR commit 7726adbdafb34bda85e25c8fc5e192f4.
  • Configure Buf's sparse Cargo registry and cargo:token credential provider in .cargo/config.toml.
  • Pass the repository BUF_TOKEN secret to Cargo as CARGO_REGISTRIES_BUF_TOKEN="Bearer ..." in pre-merge, nightly, and reusable Rust CI.
  • Forward the raw token through required BuildKit secret mounts for shared runtime images, EPP and compliance cache-miss builds, standalone sidecars, mocker examples, and source-building recipes. The credential is never stored in an image layer.
  • Remove the proto download/build script and its checksum, code-generation, and offline-directory machinery.
  • Keep the existing tonic 0.13 shared transport for other sidecars while reusing the same transport implementation through a tonic 0.14 pool for the BSR-generated vLLM SDK.

Authentication

  • CI expects a repository Actions secret named BUF_TOKEN.
  • Local builds set CARGO_REGISTRIES_BUF_TOKEN="Bearer ${BUF_TOKEN}" or run cargo login --registry buf "Bearer ${BUF_TOKEN}".
  • Docker and Buildx builds pass --secret id=buf_token,env=BUF_TOKEN.
  • Authentication is workspace-wide because Cargo resolves the BSR packages from the root lockfile even when a command targets another workspace package.

Validation

  • Exact published prost and tonic crate archives: 41 focused tests plus doc tests passed.
  • cargo clippy -p dynamo-sidecar-common -p dynamo-vllm-sidecar -p dynamo-vllm-mocker --all-targets -- -D warnings
  • cargo fmt --all -- --check
  • cargo metadata --locked --offline --no-deps --format-version 1
  • pre-commit run check-yaml --files .github/workflows/pre-merge.yml .github/workflows/nightly-ci.yml .github/workflows/dynamo-pipeline.yml
  • BuildKit --check passed for the rendered production image, EPP images, all three sidecars, mocker images, and both source-building recipe images.
  • docker compose config --quiet passed for the authenticated mocker build.

Where should the reviewer start?

  • .cargo/config.toml and lib/sidecar/vllm/Cargo.toml for the authenticated, pinned BSR dependencies.
  • lib/sidecar/common/src/transport.rs for tonic 0.13/0.14 shared connection handling.
  • .github/workflows/pre-merge.yml, .github/actions/docker-remote-build/action.yml, and container/templates/wheel_builder.Dockerfile for CI and secret-safe image-build credential injection.

Related Issues

This PR is NOT linked to an issue:

  • Confirmed — no related issue

Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
@datadog-official

datadog-official Bot commented Aug 10, 2026

Copy link
Copy Markdown

Pipelines

⚠️ Warnings

🚦 5 Pipeline jobs failed

PR | backend-status-check   View in Datadog   GitHub Actions

PR | dynamo-status-check   View in Datadog   GitHub Actions

Pre Merge | pre-merge-status-check   View in Datadog   GitHub Actions

View all 5 failed jobs.

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 05cd5d7 | Docs | Datadog PR Page | Give us feedback!

Signed-off-by: Connor Carpenter <connorc@nvidia.com>
@connorcarpenter15 connorcarpenter15 changed the title build(vllm): source protocol from Buf build(vllm): consume protocol SDKs from Buf Aug 11, 2026
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
@github-actions

Copy link
Copy Markdown
Contributor

@connorcarpenter15
connorcarpenter15 force-pushed the feat/vllm-sidecar-control-discovery branch 2 times, most recently from b501531 to 9b0962b Compare August 12, 2026 00:24
Base automatically changed from feat/vllm-sidecar-control-discovery to main August 12, 2026 01:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

actions build container documentation Improvements or additions to documentation size/XXL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant