Skip to content
Merged
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion scripts/ci_install_vllm.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,21 @@ fi

echo "Using uv version: $(uv --version)"

# Pin vLLM below 0.19.1. vLLM 0.19.1 bundled the transformers v5 upgrade,
# which breaks intfloat/e5-mistral-7b-instruct embedding quality
# (self-similarity ~0.33 instead of ~1.0) via an EOS / last-token pooling
# regression. Last-known-good combo is:
# vllm==0.19.0 + transformers==4.57.6 (run 24591985132, 82a3fb1a)
# Regression first appeared in run 24608587304 (dcede344). Failure signature:
# https://github.com/lightseekorg/smg/actions/runs/24644816475/job/72068881582
#
# We rely on vllm 0.19.0's own wheel metadata (transformers<5,>=4.56.0) to
# force transformers back to 4.x. Not pinning transformers separately — if
# we ship vllm we don't want to second-guess the library's own dep range;
# drop this pin whenever vllm publishes a release that re-verifies embedding
# correctness post-transformers-v5.
echo "Installing vLLM..."
uv pip install vllm
uv pip install "vllm<0.19.1"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To ensure CI stability and consistency with the project's metadata, consider pinning to the exact last-known-good version (vllm==0.19.0). This matches the version cited in the comments and the lower bound defined in grpc_servicer/pyproject.toml. Using an exact pin is generally preferred in CI to ensure reproducibility, especially when a specific version is known to cause regressions. Additionally, since this regression affects core functionality (embedding correctness), this constraint should ideally be reflected in the package metadata (grpc_servicer/pyproject.toml) as well to protect users from installing the broken combination. This approach also adheres to the requirement of using uv pip install for Python package installations in CI scripts for consistency and performance.

Suggested change
uv pip install "vllm<0.19.1"
uv pip install "vllm==0.19.0"
References
  1. When installing Python packages in CI scripts, use uv pip install for consistency and performance if uv is already in use for other installations.
  2. Ensure that all external resource versions specified in scripts are accurate and exist, as incorrect versions can lead to build failures.

Comment thread
slin1237 marked this conversation as resolved.
Resolved

# Install nixl for vLLM PD disaggregation (NIXL KV transfer)
echo "Installing nixl..."
Expand Down
Loading