[None][fix] Revert "Add PyTorch reset_prefix_cache API (#14970)"

[xxi-nv]

Description

Reverts #14970 ([None][feat] Add PyTorch reset_prefix_cache API, merge commit ae9226e2852d39f93beb824bbd156a58d4445f31).

Why

PR #14970 added a reset_prefix_cache() method to BaseWorker in tensorrt_llm/executor/base_worker.py. Since RayGPUWorker(RpcWorkerMixin, BaseWorker) inherits from BaseWorker, the method is now present on the worker class.

The Ray worker-extension injection path (RayGPUWorker._inject_worker_extension, tensorrt_llm/executor/ray_gpu_worker.py:196-203) rejects any extension whose attribute name already exists on the worker class:

for attr in dir(extension_cls):
    if attr.startswith("__"):
        continue
    if hasattr(worker_class, attr):
        raise ValueError(
            f"Worker class {worker_class.__name__} already defines '{attr}', "
            f"which conflicts with extension {extension_cls.__name__}.")

RL / verl WorkerExtension also defines reset_prefix_cache, so the names now collide and worker creation raises:

ValueError: Worker class RayGPUWorker already defines 'reset_prefix_cache',
which conflicts with extension WorkerExtension.

This kills RayGPUWorker during initialization, so the entire H100_PCIe-PyTorch-Ray-1 stage fails — every test_llm_update_weights*, test_llm_partial_update_weights*, and async_llm test dies before reaching its own logic (surfaced as RuntimeError: RayGPUWorker died during initialization).

How it reached main

PR #14970 was merged with /bot skip --comment "Known flaky tests" even though the H100_PCIe-PyTorch-Ray-1 stage was already failing in its own final CI run (L0_MergeRequest_PR #42818, commit dd8936e). The failure was not flaky — it is this collision — so the regression landed on main and broke post-merge CI (e.g. L0_Test-x86_64-Single-GPU #3385).

Follow-up

The reset_prefix_cache feature can be re-landed by either choosing a non-colliding method name on BaseWorker, or making the extension-injection check tolerate an intentional override instead of hard-failing on any name match.

Test Coverage

This is a pure revert (211 deletions across the 7 files originally touched by #14970, net diff is the exact inverse). Unblocks the H100_PCIe-PyTorch-Ray-1 Ray orchestrator update-weights tests that #14970 broke.

PR Checklist

• PR description clearly explains what and why.

Summary by CodeRabbit

• Breaking Changes

  • Removed the reset_prefix_cache() public API method from the LLM class and corresponding OpenAI server endpoint.

• New Features

  • Added /kv_cache_events endpoint to the OpenAI server for KV cache event management.

[xxi-nv]

/bot run

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 90879966-f437-4188-9a2c-b2078824ba04

📥 Commits

Reviewing files that changed from the base of the PR and between 82ca2c5 and 62e6f80.

📒 Files selected for processing (7)

• tensorrt_llm/_torch/pyexecutor/py_executor.py
• tensorrt_llm/executor/base_worker.py
• tensorrt_llm/llmapi/llm.py
• tensorrt_llm/serve/openai_server.py
• tests/unittest/_torch/executor/test_py_executor.py
• tests/unittest/api_stability/references/llm.yaml
• tests/unittest/llmapi/test_llm.py

💤 Files with no reviewable changes (7)

• tests/unittest/_torch/executor/test_py_executor.py
• tests/unittest/api_stability/references/llm.yaml
• tensorrt_llm/_torch/pyexecutor/py_executor.py
• tests/unittest/llmapi/test_llm.py
• tensorrt_llm/executor/base_worker.py
• tensorrt_llm/llmapi/llm.py
• tensorrt_llm/serve/openai_server.py

---

📝 Walkthrough

Walkthrough

This PR removes the reset_prefix_cache() API feature from the executor stack, public LLM interface, OpenAI server, and all corresponding tests. The removal spans from low-level PyExecutor implementation through high-level HTTP endpoints and API contracts.

Changes

Removal of reset_prefix_cache API

Layer / File(s)	Summary
Remove reset_prefix_cache from executor implementations tensorrt_llm/_torch/pyexecutor/py_executor.py, tensorrt_llm/executor/base_worker.py	PyExecutor removes the precondition check that rejected requests with active or queued items; BaseWorker deletes the entire reset_prefix_cache() method, eliminating the PyTorch-only prefix-cache invalidation API.
Remove reset_prefix_cache endpoint and handler from OpenAI server tensorrt_llm/serve/openai_server.py	register_routes stops registering the /reset_prefix_cache POST endpoint and handler method; minor cleanup to get_kv_cache_events adds explicit pass to the empty-queue exception path.
Remove reset_prefix_cache test coverage and API stability reference tests/unittest/_torch/executor/test_py_executor.py, tests/unittest/llmapi/test_llm.py, tests/unittest/api_stability/references/llm.yaml	Deletes all PyExecutor and LLM unit tests validating reset_prefix_cache behavior, removes fake executor helper test classes, and removes reset_prefix_cache from the stable API method schema.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

• NVIDIA/TensorRT-LLM#14970: Adds reset_prefix_cache API across BaseWorker, _TorchLLM (llm.py), and the OpenAI server route/handler; this PR removes the same API and its test coverage.

Suggested reviewers

• suyoggupta
• hchings
• DomBrown
• achartier

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately describes the main change: reverting PR #14970 which added a reset_prefix_cache API. It is clear, specific, and directly related to the changeset.
Description check	✅ Passed	The description is comprehensive and follows the template structure. It clearly explains what was reverted (the reset_prefix_cache API), why (name collision with Ray worker extension), how it reached main, test coverage, and includes the PR checklist.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[tensorrt-cicd]

PR_Github #53865 [ run ] triggered by Bot. Commit: 62e6f80 Link to invocation

[shuyixiong]

LGTM

[xxi-nv]

/bot skip -m "skip to unblock the CI, just revert PR"

[github-actions]

GitHub Bot Help

/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...

Provide a user friendly way for developers to interact with a Jenkins server.

Run /bot [-h|--help] to print this help message.

See details below for each supported subcommand.

Details

run [--reuse-test (optional)pipeline-id --disable-fail-fast --skip-test --stage-list "A10-PyTorch-1, xxx" --gpu-type "A30, H100_PCIe" --test-backend "pytorch, cpp" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" --detailed-log --debug(experimental) --high-priority]

Launch build/test pipelines. All previously running jobs will be killed.

--reuse-test (optional)pipeline-id (OPTIONAL) : Allow the new pipeline to reuse build artifacts and skip successful test stages from a specified pipeline or the last pipeline if no pipeline-id is indicated. If the Git commit ID has changed, this option will be always ignored. The DEFAULT behavior of the bot is to reuse build artifacts and successful test results from the last pipeline.

--disable-reuse-test (OPTIONAL) : Explicitly prevent the pipeline from reusing build artifacts and skipping successful test stages from a previous pipeline. Ensure that all builds and tests are run regardless of previous successes.

--disable-fail-fast (OPTIONAL) : Disable fail fast on build/tests/infra failures.

--skip-test (OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.

--stage-list "A10-PyTorch-1, xxx" (OPTIONAL) : Only run the specified test stages. Supports wildcard * for pattern matching (e.g., "*PerfSanity*" matches all stages containing PerfSanity). Examples: "A10-PyTorch-1, xxx", "PerfSanity". Note: Does NOT update GitHub check status.

--gpu-type "A30, H100_PCIe" (OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.

--test-backend "pytorch, cpp" (OPTIONAL) : Skip test stages which don't match the specified backends. Only support [pytorch, cpp, tensorrt, triton]. Examples: "pytorch, cpp" (does not run test stages with tensorrt or triton backend). Note: Does NOT update GitHub pipeline status.

--only-multi-gpu-test (OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.

--disable-multi-gpu-test (OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.

--add-multi-gpu-test (OPTIONAL) : Force run the multi-GPU tests in addition to running L0 pre-merge pipeline.

--post-merge (OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.

--extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" (OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Supports wildcard * for pattern matching. Examples: --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx", --extra-stage "Post-Merge".

--detailed-log (OPTIONAL) : Enable flushing out all logs to the Jenkins console. This will significantly increase the log volume and may slow down the job.

--debug (OPTIONAL) : Experimental feature. Enable access to the CI container for debugging purpose. Note: Specify exactly one stage in the stage-list parameter to access the appropriate container environment. Note: Does NOT update GitHub check status.

--high-priority (OPTIONAL) : Run the pipeline with high priority. This option is restricted to authorized users only and will route the job to a high-priority queue.

kill

kill

Kill all running builds associated with pull request.

skip

skip --comment COMMENT

Skip testing for latest commit on pull request. --comment "Reason for skipping build/test" is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

reuse-pipeline

reuse-pipeline

Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

[xxi-nv]

/bot skip --comment "skip to unblock the CI, just revert PR"

[tensorrt-cicd]

PR_Github #53870 [ skip ] triggered by Bot. Commit: 62e6f80 Link to invocation

[tensorrt-cicd]

PR_Github #53865 [ run ] completed with state ABORTED. Commit: 62e6f80

Link to invocation

[tensorrt-cicd]

PR_Github #53870 [ skip ] completed with state SUCCESS. Commit: 62e6f80
Skipping testing for commit 62e6f80

Link to invocation