fix(trtllm): populate disagg_request_id for PYTHON transceiver

[yifjiang]

Summary

• Fix response shuffling and crashes in disaggregated serving when using the PYTHON cache transceiver (transceiver_runtime=PYTHON) with NIXL
• Populate disagg_request_id on DisaggregatedParams in the prefill path using TRT-LLM's get_global_disagg_request_id() snowflake ID generator

Problem

When dynamo creates DisaggregatedParams(request_type="context_only") in the prefill worker, it never sets disagg_request_id. TRT-LLM's Python transceiver uses get_unique_rid() which prefers disagg_request_id over request_id. When disagg_request_id is None:

1. AssertionError crash (rc9): transfer.py:140 asserts params.disagg_request_id is not None — 100% failure rate, every single request crashes
2. Response shuffling (older TRT-LLM): All concurrent requests map to the same dictionary key (None) in the transceiver's session tracking

TRT-LLM's native serving path (openai_disagg_service.py) calls get_global_disagg_request_id() to generate unique IDs, but dynamo bypasses that path.

Proof: HEAD crashes without this fix

Built from dynamo HEAD (f849e1a69) + TRT-LLM 1.3.0rc9, deployed 1P1D disagg with transceiver_runtime: PYTHON on GB200. Every request fails:

File "tensorrt_llm/_torch/disaggregation/native/transfer.py", line 140, in __init__
    assert params.disagg_request_id is not None
AssertionError

Image: nvcr.io/goirlvsxnepa/llm_nim/dynamo-trtllm:head-no-fix (local only, not pushed)

Fix

Call get_global_disagg_request_id(self.disagg_machine_id) when creating DisaggregatedParams in the prefill path. This generates a snowflake ID (timestamp + machine_id + counter) that is globally unique and persists across the prefill→decode handoff via the params_dict transport.

Machine ID derivation: endpoint.connection_id() % 1021 — uses the worker's etcd lease ID (dynamo's per-worker unique identifier) mod the largest prime < 1024. This matches the 10-bit machine_id space used by TRT-LLM's snowflake generator (ref) and provides per-worker uniqueness even when multiple workers run on the same host.

Also ensures disagg_request_id is set when using ep_disaggregated_params from the encode worker (EPD flow) if it comes in as None.

Test Plan

• 6 unit tests added to test_trtllm_handler_base.py:
  • ID populated in prefill mode
  • IDs unique across calls
  • ID set on ep_params with None
  • Existing non-None ID preserved
  • Machine ID taken from config
  • Different machine IDs produce different snowflake IDs
• E2E validated on GB200 1P1D disagg serving with transceiver_runtime: PYTHON
  • Image pulled fresh from nvcr.io (TRT-LLM 1.3.0rc9, digest sha256:65cda0da...)
  • Model: Qwen3-Coder-30B-A3B + EAGLE3
  • 5 concurrent keyword requests (ALPHA/BRAVO/CHARLIE/DELTA/ECHO): zero response shuffling
  • Nodes: gb-nvl-053-compute[07,09], Job 693513
• Proved HEAD without fix crashes 100% (head-no-fix image, same setup)
• Linters pass (black, ruff, isort)
• CI

Related

• Companion TRT-LLM fix for gen-side timeout: NVIDIA/TensorRT-LLM#12471
• tool_choice=none fix: ai-dynamo/dynamo#7391

Signed-off-by: Yifan Jiang 19356972+yifjiang@users.noreply.github.com

[github-actions]

👋 Hi yifjiang! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

[coderabbitai]

Walkthrough

Modified the disaggregated request handling in PREFILL mode to ensure disagg_request_id is always populated. Added import for get_global_disagg_request_id and implemented a guard clause that sets the ID when creating or selecting disaggregated parameters.

Changes

Cohort / File(s)	Summary
Disaggregation Request ID Initialization components/src/dynamo/trtllm/request_handlers/handler_base.py	Added get_global_disagg_request_id import and modified _setup_disaggregated_params_for_mode to ensure disagg_request_id is always set for PREFILL mode, with a post-creation guard that populates the ID if missing or None, regardless of whether parameters come from encoder or are newly constructed.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: populating disagg_request_id for PYTHON transceiver in TRT-LLM integration, which is the core objective of this pull request.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Description check	✅ Passed	The pull request description is comprehensive and well-structured, containing all required template sections: overview (Summary), details (Problem, Fix, Test Plan), reviewer guidance (Where to start), and related issues (Related).

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@components/src/dynamo/trtllm/request_handlers/handler_base.py`:
- Around line 474-476: The highlighted assignment to
disaggregated_params.disagg_request_id using get_global_disagg_request_id(0) is
causing CI formatting failures; update the file by running the project's
formatting and lint hooks (e.g., ruff format, ruff check --fix or pre-commit,
and isort --profile=black) so the line and surrounding code are rewritten to the
project's style, then re-stage the formatted changes — specifically ensure the
line with disaggregated_params.disagg_request_id = (
get_global_disagg_request_id(0) ) is reformatted to match black/isort
expectations.
- Around line 470-476: Replace the defensive getattr usage and access
disagg_request_id directly on disaggregated_params so missing attributes
fail-fast; change the check to use disaggregated_params.disagg_request_id is
None and, if so, assign disaggregated_params.disagg_request_id =
get_global_disagg_request_id(0). This keeps the type contract enforced for
disaggregated_params and allows AttributeError to surface when the attribute is
missing.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 9b420290-5bd9-4c07-b9b2-963a349018be

📥 Commits

Reviewing files that changed from the base of the PR and between 149bf5a and bbd13fe.

📒 Files selected for processing (1)

• components/src/dynamo/trtllm/request_handlers/handler_base.py