[hhzhang16]

Overview

When multiple active-active PoolRelay Pods report on that same pool, this change keeps each Pod's reports separate, pairs serving load with KV load from the same Pod, and chooses one fresh view. Builds on #17524, which made Global Router use reported load when choosing a pool.

1. The frontend connects through the PoolRelay Service to one Relay Pod. A reconnect may reach a different Pod.
2. Serving-load and KV-load streams come over the same connection, Global Router/FE checks each report's protocol version and marker, and records which replica sent it.
3. Global Router keeps the replicas' reports separate for each pool.
4. It chooses one fresh replica view and uses KV load only from that replica when ranking pools.

Key Changes

• Track load by Relay replica and incarnation so active-active reports are not added together.
• Use serving and KV load from the same replica. If that replica has no fresh KV report, rank using its serving load.
• Stop the previous subscription before replacing it, so an old stream cannot restore stale reports.

Validation

• cargo test -p dynamo-global-frontend -p dynamo-global-router --lib --jobs 8: 31 passed.
• The frontend test uses a real TCP/gRPC connection to a mock Relay and verifies that both load streams use the same connection.

Issues and Ongoing Work

• Closes #XXXX

Pull Request Checklist

• I used Dynamo's /pr-checklist skill.

Summary by CodeRabbit

• Bug Fixes
  • Pool selection now pairs load and KV usage data from the same Relay, reducing mismatches between reported capacity and usage.
  • Outdated Relay reports expire, allowing selection to fall back to another active Relay.
  • Relay updates now refresh subscriptions when the Relay URL or model changes. Incomplete or invalid updates are not recorded.

[coderabbitai]

No actionable comments were generated in the recent review.

Recent review info

Run configuration

• Source mode: Auto
• Repository: ai-dynamo/dynamo
• Branch: hannahz/llm-217-plumb-serving-load-through-poolrelay
• File: .coderabbit.yaml
• Commit: 4ed2b56
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Enterprise
• Run ID: ef0856bf-c56a-4ed0-b856-246ef36cfb41

Commits

Reviewing files that changed from the base of the PR and between 91a64d9 and 4ed2b56.

Files selected for processing: 4

• lib/global-frontend/src/relay.rs
• lib/global-router/src/affinity.rs
• lib/global-router/src/lib.rs
• lib/global-router/src/signals.rs

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

Walkthrough

The frontend now follows serving and KV load streams through one Relay connection and records validated reports with Relay runtime identity. The router stores reports by source, expires stale entries, and selects occupancy from one source.

Changes

Relay Load Signal Collection

Layer / File(s)	Summary
Source-aware signal storage and selection lib/global-router/src/signals.rs, lib/global-router/src/lib.rs, lib/global-router/src/affinity.rs	PoolSignals stores load and KV reports by Relay runtime ID and incarnation. Occupancy uses the newest eligible load report and KV data from the same source. Tests cover replica selection, expiry, and incarnation changes.
Relay subscription lifecycle lib/global-frontend/src/relay.rs	follow_relays tracks each subscription's Relay URL and model. It cancels and awaits tasks for changed subscriptions before forgetting pool signals, then starts one subscription task for each wanted pool.
Combined stream and validated updates lib/global-frontend/src/relay.rs	stream_pool subscribes to serving and KV load over one connection. Update recorders validate the protocol envelope, require Relay identity, and skip incomplete reports. Tests cover envelope validation, identity matching, and the shared connection.

Priority: Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Merge Risk: Minimal · up to 4ed2b

No actionable issue is established for the documented deployment. Normal validation can proceed before merge.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 55.56% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta…	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Title check		The title "feat(global-router): keep pool load tied to a Relay replica" matches the main change. The PR keeps load reports separate per Relay replica (RelaySource with runtime ID and incarnation), pai…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Description check		The description follows the repository template with Overview, Key Changes, Validation, Issues and Ongoing Work, and the checklist. It explains the goal of keeping each active-active PoolRelay replica…

Full details: Title check

Explanation

The title "feat(global-router): keep pool load tied to a Relay replica" matches the main change. The PR keeps load reports separate per Relay replica (RelaySource with runtime ID and incarnation), pairs serving load with KV load from the same replica, and selects one fresh replica view. The title names the Global Router scope and the core idea. It is concise, specific, and uses the conventional commit format.

Full details: Description check

Explanation

The description follows the repository template with Overview, Key Changes, Validation, Issues and Ongoing Work, and the checklist. It explains the goal of keeping each active-active PoolRelay replica's reports separate, pairing serving load with KV load from the same replica, and choosing one fresh replica view. Key Changes list the replica tracking, same-replica load usage, and stale-subscription handling. Validation gives a concrete test command with 31 passing tests and describes the real TCP/gRPC frontend test. The main gap is the unfilled issue link: 'Closes #XXXX' is a placeholder. The template marks this link check as advisory today, so the gap does not block the description from being mostly complete.

Full details: Docstring Coverage

Explanation

Docstring coverage is 55.56% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 9 functions across 4 files.

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches: 💡 1

🛠️ Fix failing CI checks: 💡

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[devin-ai-integration]

✅ Dynamo PR CI passed — run 41079838370 (attempt 1) on 9b21e7efcc

Gate checks: ✅ backend-status-check · ✅ deploy-status-check · ✅ dynamo-status-check

_{Posted automatically by Devin for run 41079838370. Updated on every full-CI run of this PR.}