[Bugfix][KV Offload] Bound the HIT_PENDING wait so a stalled write cannot defer requests indefinitely

docs/features/kv_offloading_usage.md

@@ -76,6 +76,7 @@ vllm serve <model> \
 | `max_tracker_size` | no | `64000` | single-tier | Max entries in the lookup tracker. |
 | `secondary_tiers` | no | `[]` | multi-tier | List of secondary tier configs (see below). |
 | `offload_prompt_only` | no | `true` | both | If `true`, only prompt (prefill) blocks are offloaded; decode blocks are skipped. |
+| `hit_pending_deadline_s` | no | `60` | both | Seconds a request may stay deferred because a candidate block is still write-in-flight in the primary tier. Past this, that block is treated as a miss for the rest of the request and the candidate prefix is truncated there, so the request recomputes locally rather than deferring until the client times out. The deadline is timed per blocking block: when the block holding a request up changes, the new one starts a deadline of its own instead of inheriting the elapsed wait. The default sits above the P2P worst case for a live promotion (`_LOAD_TIMEOUT_S` 30s + `_ABORT_ACK_TIMEOUT_S` 10s), so on that tier it fires only for writes that have genuinely leaked. **The CPU, filesystem and object-store backends have no equivalent bound on transfer time**, so there a healthy but slow write can cross the deadline too; the only consequence is a cache miss and local recomputation, never incorrect output. Expirations are counted by `vllm:kv_offload_hit_pending_deadline_expired`; a non-zero value is worth investigating on P2P but may simply reflect slow promotions on the other backends. Set `0` to disable and defer indefinitely. |
 | `self_describing_kv_events` | no | `false` | both | Opt-in. When `true` *and* KV cache events are enabled (`--kv-events-config` with `enable_kv_cache_events`), the connector emits self-describing block-granular `BlockStored`/`BlockRemoved` payloads (constituent block hashes, whole-chunk `token_ids`, per-block `block_size`, parent hash, LoRA + group/cache-spec metadata) instead of the placeholder fallback, so external KV-event consumers can index offloaded blocks. Inert unless events are enabled. With `TieringOffloadingSpec`, a CPU promotion is self-describing when a local request observes its primary-tier `HIT` before event translation; otherwise its stored event may retain the placeholder, while a later `HIT` can backfill metadata for removal. Pending-removal/re-promotion races and externally initiated promotions may also produce placeholders, and consumers must ignore removals for unknown hashes. Full-attention groups only; sliding-window/SSM groups keep the placeholder fallback. In chunk mode (`block_size` > GPU block size, or `blocks_per_chunk` > 1), overlapping chunks re-announce shared per-block hashes, so consumers must reference-count (deduplicate) repeated store/remove announcements. |
 | `spec_module_path` | no | — | both | Python import path for a custom `OffloadingSpec` not in the built-in registry. Required only when `spec_name` is not built-in (advanced). |