[Feat][KVConnector] Add bind_gpu_block_pool() to KVConnectorBase_V1

[ivanium]

Purpose

Promote bind_gpu_block_pool() from a SimpleCPUOffloadConnector-only method to a first-class API on KVConnectorBase_V1. This API is OPT-in and remains compatible with all existing connectors; it additionally allows any connector to access the GPU block pool for per-GPU-block status tracking (e.g., ref count management, prefix cache block iteration).

Test Plan

Existing CI tests.

Test Result

Passed.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

This pull request refactors the bind_gpu_block_pool method by moving it from the SimpleCPUOffloadConnector to the base KVConnectorBase class. This change allows for a more consistent interface across different connector implementations and simplifies the logic in the scheduler by removing the need for hasattr checks. Feedback was provided regarding the docstring in the base class, which should be updated to advise against direct reference count manipulation in favor of using the BlockPool's touch() and free_blocks() methods to ensure internal consistency.

[jonathanc-n]

Looks good to me. cc @orozery

[NickLucche]

I think this is fine, although I have to admit it's a bit specific.
Overall we're missing a "post-init" hook. At the same time, I am trying to consider what other alternatives we might have, like:

• broadening the semantics of this method to an actual "post-init", but we may as well just do this in the v2 interface
• postponing the connector allocation and allowing kwargs to be passed to the connector (for gpu blocks). This way we should be able to get the same result.

However I don't feel strongly that any of the above is better than what we have here.

[orozery]

I fed claude with my concerns about making BlockPool accessible to connectors.
He suggested this idea:

Define a narrow GPUBlockPoolView protocol that exposes only what connectors need:

  class GPUBlockPoolView(Protocol):
      def touch_blocks(self, block_ids: list[int]) -> None: ...
      def free_blocks(self, block_ids: list[int]) -> None: ...
      def iter_free_block_ids_with_hash(self) -> Iterator[tuple[int, bytes]]: ...

Then BlockPool (or a thin adapter) implements this, and the base class method becomes bind_gpu_block_pool(self, view: GPUBlockPoolView). This gives connectors the power they need for ref-count management and lazy-mode scanning without
exposing allocation, eviction, cache reset, or the internal KVCacheBlock objects.

====

Also, I agree with @NickLucche we should try to generalize into a single "scheduler-side initialization" hook.
See here for a worker-side example:
#37339 (comment)

[NickLucche]

For the sake of pushing this forward (given dsv4 roadmap) and as I also find myself needing a similar hook for TTL renewal P-side, I pushed a PR with the proposed "generically available API" that we discussed above (or at least my interpretation of it).
Let me know what you think @orozery @ivanium

[markmc]

IMO, what's being proposed here is to expose KVCacheManager.block_pool to out-of-tree connectors, and thereby commiting to maintaining backwards compatibility for the current semantics of whatever creative stuff a connector might do with that interface.

See also in the design doc

@njhill:

we need to think is there a slightly higher abstraction that makes sense for connectors to work with or should they have free rein on the pool so that they can hold/free non-contiguous blocks etc.

@ivanium:

Agreed to have a unified way to protect blocks and better abstractions for it.

In that vein, I much prefer @orozery's suggestion:

Define a narrow GPUBlockPoolView protocol that exposes only what connectors need:

And I'd view that as a post-merge cleanup of #37160

(Honestly, it feels like there is also a lurking, unresolved question about request-oriented vs block-oriented connectors, the potential for a redesign of the connectors API, and whether out-of-tree connectors are worth supporting ... but all that aside, I think we should be more thoughtful about how we expose the block pool to connectors)