feat(trtllm): engine-owned conversation-aware ADP routing

[nv-yna]

DRAFT — blocked on a TensorRT-LLM dependency bump. Opening now for approach review (per @PeaBrane's guidance in DM). Not for merge until the runtime carries the API (see "Dependency gate" below).

What this PR does

Adds opt-in support for TensorRT-LLM's engine-owned conversation-affinity ADP routing. When the engine's attention_dp_config.kv_cache_routing_conversation_affinity is enabled, the TRT-LLM worker:

1. derives the conversation id from the frontend-forwarded agent_context.session_id,
2. passes it as ConversationParams to generate_async, and
3. stops forcing attention_dp_rank, so the engine's ConversationAwareADPRouter pins the conversation to a DP rank (sticky, load-balanced first turn).

When the engine policy is off, Dynamo-owned strict DP-rank forcing is byte-for-byte unchanged.

Why this shape (design notes for review)

This is the main-appropriate re-implementation of the DSV4 feature branch's conversation-aware routing (321567ee83, #10517), following @PeaBrane's review guidance ("avoid duplicate routing plumbing; derive the id from session identity in the engine handlers"):

• No Rust changes. On main the session identity already reaches the worker via the serialized PreprocessedRequest.agent_context field (lib/llm/src/protocols/common/preprocessor.rs:272). The feature branch's approach added a second copy (RoutingHints.conversation_id) — dropped here entirely.
• agent_context.session_id, not the old session_control — main diverged; session_control doesn't exist here. Uses session_id (the active reasoning/tool chain), not parent_session_id.
• Gated on the resolved engine config, not a DYN_* env var — a Dynamo-side switch could disagree with the engine (e.g. suppress the rank when no engine router exists). The engine config is the single source of truth.
• ConversationParams, not disaggregated_params.conversation_id — the feature branch mutated disaggregation metadata (specific to a custom wheel). Upstream shipped a dedicated ConversationParams argument (generate_async(..., conversation_params=...)), which also covers ordinary aggregated requests (the old path only worked for P/D).
• No exp-H/gap #4 instrumentation — all experimental one-shot logging stripped for production.

Dependency gate ⛔ (why this is a draft)

ConversationParams, kv_cache_routing_conversation_affinity, and ConversationAwareADPRouter require a TensorRT-LLM release newer than 1.3.0rc20. main currently pins 1.3.0rc19 (pyproject.toml, container/context.yaml), which has none of them — the API only exists on TRT-LLM main today (not in any release tag: rc20 is latest). So this cannot be enabled or e2e-tested here yet.

This PR is therefore safe-by-default and CI-clean: the ConversationParams import is guarded, the gate resolves False on rc19, and behavior is identical to today. Enabling it on an incompatible wheel fails fast at startup. The wheel/image bump (and the GPU integration + full handler test matrix) is a prerequisite follow-up to be landed first.

Files

• components/src/dynamo/trtllm/conversation_affinity.py — new helper: session-id extraction, engine-config gate, ConversationParams builder (guarded import).
• components/src/dynamo/trtllm/llm_engine.py — unified path: resolve gate at init; suppress rank + pass ConversationParams when enabled.
• components/src/dynamo/trtllm/request_handlers/handler_base.py — legacy path: same, gate resolved lazily.
• components/src/dynamo/trtllm/tests/test_conversation_affinity.py — wheel-independent unit tests for the helpers.

Testing

• ruff check / ruff format --check clean; helper unit tests pass (6 passed, 1 skipped where the API is absent).
• No Rust diff → no cargo exposure.
• Deferred to the dependency-bump follow-up: GPU integration (sticky rank across turns, no-id fallback, aggregated + P/D) and the mocked-generate_async handler matrix.

Open questions for review

1. Which released TRT-LLM version/image will carry the post-rc20 API? (sole hard blocker)
2. Frontend per-rank accounting vs engine-owned placement (push_router.rs still stamps the frontend rank) — needs an e2e accounting pass; must not reintroduce Rust conversation plumbing.
3. Fail-startup on incompatible-wheel-with-affinity-enabled — currently yes; confirm desired.

[datadog-official]

⚠️ Warnings

🚦 2 Pipeline jobs failed

PR | backend-status-check     

PR | trtllm-runtime / Test cuda13.1, amd64     

ℹ️ Info

🎯 Code Coverage (details)
• Patch Coverage: 92.86%
• Overall Coverage: 28.61% (-17.36%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: c988b3c | Docs | Give us feedback!}