Skip to content

fix(router): disclose selected worker in non-KV modes for 1.3.0 - #11324

Merged
saturley-hall merged 1 commit into
ai-dynamo:release/1.3.0from
ishandhanani:idhanani/dyn-3353-worker-disclosure-release-1.3.0
Jul 7, 2026
Merged

fix(router): disclose selected worker in non-KV modes for 1.3.0#11324
saturley-hall merged 1 commit into
ai-dynamo:release/1.3.0from
ishandhanani:idhanani/dyn-3353-worker-disclosure-release-1.3.0

Conversation

@ishandhanani

Copy link
Copy Markdown
Contributor

Backports #11245 to release/1.3.0 using the release branch's existing session-affinity API. It discloses the resolved worker for non-KV routing only after dispatch succeeds, preventing failed attempts from poisoning retry metadata.

CLOSES: DYN-3353

Summary

  • Add prepared-dispatch paths that return the worker selected after transport resolution.
  • Record worker attribution for standard, session-affinity, direct, and LoRA routing only after a response stream is created successfully.
  • Preserve occupancy accounting when transport fallback selects a different worker.

How This Was Implemented

Walkthrough

Mental model

sequenceDiagram
    participant Router
    participant PushRouter
    participant Backend
    participant Tracker
    Router->>PushRouter: Select and prepare request
    PushRouter->>Backend: Dispatch to resolved worker
    Backend-->>PushRouter: Response stream created
    PushRouter->>Tracker: Record resolved worker and rank
Loading

State and ordering

The tracker is removed from the prepared request and returned as dispatch metadata. Worker attribution is committed only after dispatch returns a stream; preparation or dispatch failures leave the tracker unchanged so retry and fallback paths can record the worker that actually succeeds.

Boundaries and limitations

This backport does not import the distributed-affinity work or its later revert. The public release callback remains (worker_id, dp_rank), and existing release routing behavior remains unchanged.

Validation

  • cargo test -p dynamo-runtime -p dynamo-llm push_router::tests --lib (41 passed)
  • cargo clippy -p dynamo-runtime -p dynamo-llm --lib -- -D warnings
  • cargo fmt --all -- --check
  • git diff --check

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
(cherry picked from commit 59d6146)
@datadog-official

datadog-official Bot commented Jul 7, 2026

Copy link
Copy Markdown

Pipelines

⚠️ Warnings

🚦 2 Pipeline jobs failed

Docs link check | lychee   View in Datadog   GitHub Actions

PR Reminder Full CI Comment Bot | pr_reminder   View in Datadog   GitHub Actions

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: c554435 | Docs | Give us feedback!

@ishandhanani
ishandhanani marked this pull request as ready for review July 7, 2026 21:22
@ishandhanani
ishandhanani requested a review from a team July 7, 2026 21:22

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

@saturley-hall
saturley-hall merged commit b67832c into ai-dynamo:release/1.3.0 Jul 7, 2026
100 of 102 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants