[V1] [P/D] Add Support for KV Load Failure Recovery#19330

Merged

simon-mo merged 18 commits intovllm-project:mainfrom

sdavidbd:feature/kv-load-failure-recovery

Sep 30, 2025

Commits on Sep 30, 2025

Add KV load failure recovery by rescheduling affected requests
David Ben-David
committed
Add KV load failure recovery unit tests
David Ben-David
committed
Add KV load failure recovery example
David Ben-David
committed
Add failure recovery support for async KV load
David Ben-David
committed
Fix unit tests after rebase
David Ben-David
committed
Refactor scheduler changes
David Ben-David
committed
Fix and refactor rollback handling of invalid sampled tokens
David Ben-David
committed
Refactor: always return a set of failed block IDs from connectors
David Ben-David
committed
Implement get_block_ids_with_load_errors in MultiConnector
David Ben-David
committed
Emphasize that KV load failure recovery does not currently support HMA
David Ben-David
committed
Fix handling of async KV load failures
David Ben-David
committed
Fix handling of progressive async KV load failures
David Ben-David
committed
Refactor total_affected_tokens calculation
David Ben-David
committed
Add note on reporting only loadable KV prefix in get_num_new_matched_tokens
David Ben-David
committed
Rename recovered_req_ids to affected_req_ids for clarity
David Ben-David
committed
Refactor KV connector output emptiness check
David Ben-David
committed
Fix PR comments
David Ben-David
committed
Fix rollback of invalid output tokens and generator state
David Ben-David
committed