[https://nvbugs/5911594][fix] Restrict HTTP cluster storage to loopback by yibinl-nvidia · Pull Request #14161 · NVIDIA/TensorRT-LLM

yibinl-nvidia · 2026-05-15T00:48:00Z

Summary:

Add a loopback-host validator for HTTP(S) cluster storage.
Reject disaggregated server startup when HTTP cluster storage is configured outside loopback-only serving.
Leave etcd cluster storage unchanged.

Summary by CodeRabbit

Bug Fixes
- Enhanced validation for cluster storage configuration to ensure HTTP/HTTPS URIs comply with loopback-only requirements in disaggregated serving setups, preventing misconfiguration errors.

coderabbitai · 2026-05-15T00:51:21Z

📝 Walkthrough

Walkthrough

This pull request adds HTTP cluster storage scope validation to prevent insecure disaggregated serving configurations. It introduces loopback host detection and a validator that ensures HTTP/HTTPS cluster URIs are only used when both the cluster URI hostname and server hostname are loopback addresses, then integrates this validation into the OpenAI disaggregated server initialization.

Changes

HTTP Cluster Storage Validation

Layer / File(s)	Summary
Loopback and HTTP cluster storage scope validators `tensorrt_llm/serve/cluster_storage.py`	Adds `ipaddress` import, then defines `is_loopback_host` to detect loopback from hostname or IP string, and `validate_http_cluster_storage_scope` to enforce that HTTP(S) cluster URIs are only used in loopback-only disaggregated serving configurations.
Server initialization with cluster storage validation `tensorrt_llm/serve/openai_disagg_server.py`	Imports the validator and refactors `OpenAIDisaggServer.__init__` to validate cluster storage URI/hostname scope before creating the cluster storage server instance, raising `ValueError` if scope validation fails.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check	❓ Inconclusive	The PR description is concise but incomplete. It lacks required sections like detailed explanation of the issue, test coverage details, and PR checklist items.	Expand the description to include: (1) the specific issue being addressed, (2) detailed test coverage information, and (3) completion of the PR checklist with appropriate confirmations.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The PR title clearly and concisely summarizes the main change: restricting HTTP cluster storage usage to loopback connections, which aligns with the primary objective of adding validation to prevent disaggregated server startup with HTTP cluster storage outside loopback-only serving.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tensorrt_llm/serve/cluster_storage.py`:
- Line 5: Add the required NVIDIA Apache 2.0 license header at the very top of
this modified source file (above the existing import ipaddress), using the
NVIDIA copyright header with the latest modification year (2026) and the Apache
License, Version 2.0 text; ensure the header matches other TensorRT-LLM files
and appears before any imports or code (so functions/classes that reference this
file like cluster storage utilities will have the mandated license header).

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 63e3bf74-5158-426e-853b-2ba0e099e2ff

📥 Commits

Reviewing files that changed from the base of the PR and between d75df19 and 86f8c45.

📒 Files selected for processing (2)

tensorrt_llm/serve/cluster_storage.py
tensorrt_llm/serve/openai_disagg_server.py

yibinl-nvidia · 2026-05-16T01:44:54Z

/bot run

tensorrt-cicd · 2026-05-16T01:51:03Z

PR_Github #48657 [ run ] triggered by Bot. Commit: 86f8c45 Link to invocation

yibinl-nvidia · 2026-05-16T07:36:56Z

/bot kill

tensorrt-cicd · 2026-05-16T07:42:03Z

PR_Github #48677 [ kill ] triggered by Bot. Commit: c97683f Link to invocation

tensorrt-cicd · 2026-05-16T07:42:06Z

PR_Github #48657 [ run ] completed with state ABORTED. Commit: 86f8c45
/LLM/main/L0_MergeRequest_PR pipeline #38437 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

tensorrt-cicd · 2026-05-16T07:42:15Z

PR_Github #48677 [ kill ] completed with state SUCCESS. Commit: c97683f
Successfully killed previous jobs for commit c97683f

Link to invocation

yibinl-nvidia · 2026-05-16T07:43:50Z

/bot run

tensorrt-cicd · 2026-05-16T07:49:47Z

PR_Github #48678 [ run ] triggered by Bot. Commit: c97683f Link to invocation

tensorrt-cicd · 2026-05-16T11:13:53Z

PR_Github #48678 [ run ] completed with state SUCCESS. Commit: c97683f
/LLM/main/L0_MergeRequest_PR pipeline #38456 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

yibinl-nvidia · 2026-05-16T11:19:06Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-05-16T11:24:10Z

PR_Github #48684 [ run ] triggered by Bot. Commit: c97683f Link to invocation

tensorrt-cicd · 2026-05-16T16:28:43Z

PR_Github #48684 [ run ] completed with state SUCCESS. Commit: c97683f
/LLM/main/L0_MergeRequest_PR pipeline #38462 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

yibinl-nvidia · 2026-05-16T16:29:11Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-05-16T16:36:01Z

PR_Github #48698 [ run ] triggered by Bot. Commit: c97683f Link to invocation

yibinl-nvidia · 2026-05-16T16:45:50Z

/bot kill

tensorrt-cicd · 2026-05-16T16:51:39Z

PR_Github #48700 [ kill ] triggered by Bot. Commit: c97683f Link to invocation

tensorrt-cicd · 2026-05-16T16:55:02Z

PR_Github #48698 [ run ] completed with state ABORTED. Commit: c97683f

Link to invocation

tensorrt-cicd · 2026-05-16T16:55:12Z

PR_Github #48700 [ kill ] completed with state SUCCESS. Commit: c97683f
Successfully killed previous jobs for commit c97683f

Link to invocation

yibinl-nvidia · 2026-05-16T16:57:43Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-05-16T17:02:51Z

PR_Github #48703 [ run ] triggered by Bot. Commit: c97683f Link to invocation

yibinl-nvidia · 2026-05-27T20:48:40Z

/bot run

tensorrt-cicd · 2026-05-27T20:56:26Z

PR_Github #50615 [ run ] triggered by Bot. Commit: d60a68c Link to invocation

tensorrt-cicd · 2026-05-27T21:36:28Z

PR_Github #50615 [ run ] completed with state FAILURE. Commit: d60a68c
/LLM/main/L0_MergeRequest_PR pipeline #40110 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

yibinl-nvidia · 2026-05-27T22:20:14Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-05-27T22:26:32Z

PR_Github #50622 [ run ] triggered by Bot. Commit: d60a68c Link to invocation

yibinl-nvidia · 2026-05-28T05:09:42Z

/bot kill

tensorrt-cicd · 2026-05-28T05:15:43Z

PR_Github #50698 [ kill ] triggered by Bot. Commit: d60a68c Link to invocation

Signed-off-by: Yibin Li <109242046+yibinl-nvidia@users.noreply.github.com>

tensorrt-cicd · 2026-05-28T05:21:58Z

PR_Github #50698 [ kill ] completed with state SUCCESS. Commit: d60a68c
Successfully killed previous jobs for commit d60a68c

Link to invocation

yibinl-nvidia · 2026-05-28T05:22:50Z

/bot run

tensorrt-cicd · 2026-05-28T05:30:35Z

PR_Github #50701 [ run ] triggered by Bot. Commit: 9dbb436 Link to invocation

tensorrt-cicd · 2026-05-28T05:48:57Z

PR_Github #50701 [ run ] completed with state FAILURE. Commit: 9dbb436
/LLM/main/L0_MergeRequest_PR pipeline #40185 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

yibinl-nvidia · 2026-05-28T05:50:31Z

/bot run

tensorrt-cicd · 2026-05-28T05:56:07Z

PR_Github #50704 [ run ] triggered by Bot. Commit: 9dbb436 Link to invocation

yibinl-nvidia · 2026-05-28T11:48:05Z

/bot kill

tensorrt-cicd · 2026-05-28T11:55:02Z

PR_Github #50767 [ kill ] triggered by Bot. Commit: 9dbb436 Link to invocation

tensorrt-cicd · 2026-05-28T11:55:08Z

PR_Github #50704 [ run ] completed with state ABORTED. Commit: 9dbb436
/LLM/main/L0_MergeRequest_PR pipeline #40188 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

tensorrt-cicd · 2026-05-28T11:55:15Z

PR_Github #50767 [ kill ] completed with state SUCCESS. Commit: 9dbb436
Successfully killed previous jobs for commit 9dbb436

Link to invocation

yibinl-nvidia · 2026-05-28T11:56:17Z

/bot run

tensorrt-cicd · 2026-05-28T12:01:55Z

PR_Github #50769 [ run ] triggered by Bot. Commit: 9dbb436 Link to invocation

tensorrt-cicd · 2026-05-28T16:11:58Z

PR_Github #50769 [ run ] completed with state SUCCESS. Commit: 9dbb436
/LLM/main/L0_MergeRequest_PR pipeline #40246 completed with status: 'SUCCESS'

CI Report

Link to invocation

yibinl-nvidia requested review from a team as code owners May 15, 2026 00:48

yibinl-nvidia requested review from Superjomn, Tabrizian and bo-nv May 15, 2026 00:48

github-actions Bot assigned yibinl-nvidia May 15, 2026

coderabbitai Bot reviewed May 15, 2026

View reviewed changes

Comment thread tensorrt_llm/serve/cluster_storage.py

yibinl-nvidia changed the title ~~fix(serve): restrict HTTP cluster storage to loopback~~ [https://nvbugs/5911594][fix] Restrict HTTP cluster storage to loopback May 15, 2026

yibinl-nvidia force-pushed the fix/5911594-cluster-storage-loopback branch from 572af5c to c97683f Compare May 16, 2026 07:40

yibinl-nvidia force-pushed the fix/5911594-cluster-storage-loopback branch from d3e58dd to d60a68c Compare May 27, 2026 20:48

yibinl-nvidia added 6 commits May 27, 2026 22:20

fix(serve): restrict HTTP cluster storage to loopback

fa55cf3

Signed-off-by: Yibin Li <109242046+yibinl-nvidia@users.noreply.github.com>

test(disaggregated): cover loopback cluster storage validation

fed26b7

Signed-off-by: Yibin Li <109242046+yibinl-nvidia@users.noreply.github.com>

test(disaggregated): use loopback host for HTTP storage

a40c2dd

Signed-off-by: Yibin Li <109242046+yibinl-nvidia@users.noreply.github.com>

test(disaggregated): address cluster storage review

f9ee20f

Signed-off-by: Yibin Li <109242046+yibinl-nvidia@users.noreply.github.com>

style: apply pre-commit fixes

fbae51e

Signed-off-by: Yibin Li <109242046+yibinl-nvidia@users.noreply.github.com>

test: remove stale service discovery waive

9dbb436

Signed-off-by: Yibin Li <109242046+yibinl-nvidia@users.noreply.github.com>

yibinl-nvidia force-pushed the fix/5911594-cluster-storage-loopback branch from d60a68c to 9dbb436 Compare May 28, 2026 05:21

yibinl-nvidia merged commit e8a42a1 into NVIDIA:main May 28, 2026
7 checks passed

yibinl-nvidia deleted the fix/5911594-cluster-storage-loopback branch May 28, 2026 19:59

Uh oh!

Conversation

yibinl-nvidia commented May 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented May 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning, 1 inconclusive)

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

yibinl-nvidia commented May 16, 2026

Uh oh!

tensorrt-cicd commented May 16, 2026

Uh oh!

yibinl-nvidia commented May 16, 2026

Uh oh!

tensorrt-cicd commented May 16, 2026

Uh oh!

tensorrt-cicd commented May 16, 2026

Uh oh!

tensorrt-cicd commented May 16, 2026

Uh oh!

yibinl-nvidia commented May 16, 2026

Uh oh!

tensorrt-cicd commented May 16, 2026

Uh oh!

tensorrt-cicd commented May 16, 2026

Uh oh!

yibinl-nvidia commented May 16, 2026

Uh oh!

tensorrt-cicd commented May 16, 2026

Uh oh!

tensorrt-cicd commented May 16, 2026

Uh oh!

yibinl-nvidia commented May 16, 2026

Uh oh!

tensorrt-cicd commented May 16, 2026

Uh oh!

yibinl-nvidia commented May 16, 2026

Uh oh!

tensorrt-cicd commented May 16, 2026

Uh oh!

tensorrt-cicd commented May 16, 2026

Uh oh!

tensorrt-cicd commented May 16, 2026

Uh oh!

yibinl-nvidia commented May 16, 2026

Uh oh!

tensorrt-cicd commented May 16, 2026

Uh oh!

yibinl-nvidia commented May 27, 2026

Uh oh!

tensorrt-cicd commented May 27, 2026

Uh oh!

tensorrt-cicd commented May 27, 2026

Uh oh!

yibinl-nvidia commented May 27, 2026

Uh oh!

tensorrt-cicd commented May 27, 2026

Uh oh!

yibinl-nvidia commented May 28, 2026

Uh oh!

tensorrt-cicd commented May 28, 2026

Uh oh!

tensorrt-cicd commented May 28, 2026

Uh oh!

yibinl-nvidia commented May 28, 2026

Uh oh!

tensorrt-cicd commented May 28, 2026

Uh oh!

tensorrt-cicd commented May 28, 2026

Uh oh!

yibinl-nvidia commented May 28, 2026

yibinl-nvidia commented May 15, 2026 •

edited

Loading

coderabbitai Bot commented May 15, 2026 •

edited

Loading