[Core][BugFix] Tag prefix-cache extra keys by source

[KernelClint]

What this fixes

Prefix-cache block hashes mix LoRA names, cache_salt, multimodal identifiers and
prompt-embeds digests into a single untagged extra_keys tuple. For example, a LoRA
request for adapter foo and a base-model request with cache_salt="foo" produce the
same first-block hash for the same prompt tokens. Either request can then be served KV
computed with the other's weights, and a caller can choose a salt that lands in an
adapter's cache namespace. Fixes #44701.

Each extra-key component is now tagged with its source:

Source	Before	After
LoRA	name	("lora", name)
cache_salt	salt	("cache_salt", salt)
Multimodal	(identifier, offset)	("mm", identifier, offset)
Prompt embeds	digest	("prompt_embeds", digest)

Equal values from different fields can no longer produce the same key.

KV cache events keep publishing the previous untagged shapes (see Compatibility), via
to_event_extra_keys() where BlockStored events are built.

An earlier revision of this PR also changed how an image's EXIF ImageID is used by the
multimodal hasher. That part was dropped, so this PR does not change
vllm/multimodal/hasher.py.

Compatibility

• Block hashes change for every block that has LoRA, cache_salt, multimodal or
  prompt-embeds extra keys. KV offloaded or persisted by an earlier version under those
  hashes will no longer match; it misses rather than being reused.
• KV cache events are unchanged: BlockStored.extra_keys still carries the untagged
  values (bare LoRA name and cache_salt, (identifier, offset) pairs, bare
  prompt-embeds digests), because external consumers such as KV-aware routers parse
  those shapes and can fail to decode the tagged tuples. Consumers that rebuild block
  keys from events can tell the values apart using lora_name and the fixed order
  (LoRA, multimodal, cache_salt, prompt embeds). BlockStored.block_hashes carries
  the new hashes.

Related PRs

• Supersedes fix(kv_cache): domain-tag LoRA and cache_salt prefix-cache keys to prevent collision (#44701) #44706 (now closed), which tagged only the LoRA and cache_salt keys.
• Uses the same ("lora", ...) tagging style as [V1][LoRA] Version prefix-cache keys for local runtime LoRA reloads #48352 (LoRA identity keys for
  same-name reloads), so the two combine cleanly.

Advisory: GHSA-rh6f-3x46-j33q

Track B: this is a public PR against vllm-project/vllm@main.

Test plan

pytest tests/v1/core/test_kv_cache_utils.py tests/v1/core/test_prefix_caching.py \
  tests/v1/core/prefix_cache tests/v1/simple_kv_offload \
  tests/distributed/test_kv_cache_events.py \
  tests/v1/kv_connector/unit/offloading_connector

• test_lora_name_and_cache_salt_block_hashes_do_not_collide checks that the first
  block hashes differ. It fails without the fix and passes with it.
• Existing assertions on hash inputs were updated to the tagged form, including tests
  added on main since this PR was opened.
• The existing KV event tests
  (test_cache_partial_block_event_keeps_every_mm_feature,
  test_eager_store_lora_metadata) are unchanged from main and still pass, so the
  event format is unchanged. Without the event conversion both fail.
• On 4x GB200: the command above gives 977 passed, 8 skipped, and 2 errors in
  test_hybrid_prefix_cache_hit_rate that also occur on main.
• A wider run (tests/v1/core, tests/v1/kv_connector/unit, tests/v1/kv_offload,
  tests/v1/simple_kv_offload, tests/distributed/test_kv_cache_events.py) failed
  exactly the same set of tests with and without this PR (gated models and
  environment-dependent engine startup), apart from one test whose fix is included here.

Maintainer update

@njhill rebased this onto current main, moved the new test into
tests/v1/core/test_kv_cache_utils.py, and updated the tests that asserted untagged key
shapes. The authors' source change is unchanged. A follow-up commit keeps the KV event
format unchanged. AI assistance (Claude) was used for the rebase, the test updates,
the event-format commit and the test runs.

Credit

• @KernelClint (Clinton Thomas) and @dhalf (Lucas Bourtoule)
• Patch the Planet (Trail of Bits + OpenAI collaboration); discovered using GPT-5.5-Cyber
• @Anai-Guo (Tai An), who reported [Bug]: V1 prefix-cache extra-key domain collision between LoRA name and cache_salt #44701 and proposed the LoRA/salt tagging in fix(kv_cache): domain-tag LoRA and cache_salt prefix-cache keys to prevent collision (#44701) #44706

All commits include DCO Signed-off-by trailers.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[DarkLight1337]

According to #22825 (comment) , this code path for ImageID is only possible to hit in offline inference

[KernelClint]

You are correct about the EXIF ImageID path. From #22825 it seems like that only triggers when the EXIF value is already a uuid.UUID (an engine-level caller passing a prepared PIL object). The online media-connector loads it as str/bytes, so it's not remotely reachable. I've dropped that part.

The core of this PR is separate and is remotely reachable: untyped prefix-cache extra keys let a LoRA name and a cache_salt collide. cache_salt is accepted on chat/completions/responses/pooling, and model="<lora-name>" selects a LoRA — both get appended as bare strings to the same extra-key tuple in kv_cache_utils, so for identical token blocks model="foo" and cache_salt="foo" alias. The typed-key change disambiguates them. I've re-scoped the PR to that. (It doesn't address request-supplied media uuid aliasing — that's a separate issue we're writing up independently.)

[KernelClint]

Gentle ping — ready for review. cc @heheda12345 @WoosukKwon

[njhill]

/ci run

[github-actions]

❌ This PR is 1 commit behind upstream main. Your branch must contain every commit currently on upstream main. No new CI build was started. Merge or rebase onto the latest main, then rerun /ci run. To test this branch at your own risk, use /ci run --allow-stale.

[njhill]

/ci run

[github-actions]

✅ Triggered Buildkite CI #91959 for commit 166608ec098c.

[github-actions]

✅ @KernelClint, CI is now available for this PR.

• /ci run starts upstream CI; /amd-ci run starts AMD CI only.
• Your branch must contain every commit currently on its upstream target branch. Merge or rebase onto the latest target branch, then rerun the command. Append --allow-stale to a run command to test an outdated branch at your own risk.
• /ci retry retries failed jobs in the CI build for the current PR head. If the current head has no CI build, it starts a new CI build for the current head containing only jobs that failed in the latest earlier CI build for this PR.
• /amd-ci retry retries failed jobs in AMD CI for the current PR head. Use /amd-ci run when the current head has no AMD CI build.
• /ci cancel cancels scheduled or running CI builds for this PR branch; /amd-ci cancel does the same for AMD CI only.

[njhill]

Thanks @KernelClint, I made some updates to the tests.

[njhill]

Actually I think some additional changes are needed to avoid breaking kv-events backwards compatibility.

[njhill]

Now updated for kv-events backward compatibility

[njhill]

/ci run

[github-actions]

✅ Triggered Buildkite CI #91982 for commit 85aaf093afd6.