fix(thunderagent): support topology-only wrapped backends

[cpakkamisaac-sae]

Overview:

This PR addresses the session-final GPU-forwarding issue described in #12016 for ThunderAgent Kubernetes deployments.

ThunderAgent already short-circuits final-session requests when they reach dynamo.thunderagent_router, but the K8s demo can bypass ThunderAgent when the wrapped backend advertises the same public chat/completions model surface. This PR adds an explicit topology-only endpoint mode so wrapped backends can stay discoverable for routing/readiness without being selected directly by the frontend.

Details:

• Add --endpoint-types none support, parsed as ModelType.Empty, for topology-only backend registration.
• Reject mixed endpoint declarations such as chat,none or none,completions.
• Update shared runtime CLI help to document none for workers fronted by another Dynamo service.
• Document using --endpoint-types none for ThunderAgent-wrapped workers.
• Add parser coverage for none, invalid mixes, invalid endpoint names, and empty input.

Validation performed:

• python3 -m py_compile components/src/dynamo/common/utils/endpoint_types.py components/src/dynamo/common/configuration/groups/runtime_args.py components/src/dynamo/common/utils/tests/test_endpoint_types.py
• git diff --check
• uv run --no-project --with ruff ruff check components/src/dynamo/common/utils/endpoint_types.py components/src/dynamo/common/utils/tests/test_endpoint_types.py
• Parser smoke test with a stubbed ModelType for chat, completions, chat,completions, none, invalid none combinations, and empty inputs.
• Validated on AKS with Dynamo Platform v1.3.0 (dynamo-platform-1.3.0, app version 1.3.0) using the existing aks-dynamo-thunderagent deployment and Qwen/Qwen3-0.6B on Standard_NC4as_T4_v3.
  • Baseline v1.3.0 behavior: 1 of 6 x-dynamo-session-final: true requests generated tokens; vLLM logs showed the final request hit component=backend, endpoint=generate.
  • Patched release deployment with an in-memory equivalent of this change plus --endpoint-types none: backend DynamoWorkerMetadata advertised model_type: "", while ThunderAgent advertised model_type: Chat | Completions.
  • After patch: DGD Ready, /v1/models returned Qwen/Qwen3-0.6B, a normal chat request returned 200 with generated tokens, and 8 of 8 x-dynamo-session-final: true requests returned the empty ThunderAgent short-circuit response without backend generation.

Where should the reviewer start?

Start with components/src/dynamo/common/utils/endpoint_types.py for the new parser behavior, then components/src/dynamo/common/utils/tests/test_endpoint_types.py for coverage. The ThunderAgent deployment guidance is in components/src/dynamo/thunderagent_router/README.md.

Related Issues

🔗 This PR is linked to an issue:

• Relates to [CONTRIBUTION]: ThunderAgent Improvements #12016

This specifically addresses the session-final GPU-forwarding/bypass item in #12016. It does not close the full tracking issue, which also includes observability and DGDR items.

---

Summary by CodeRabbit

• New Features

  • Added support for the none endpoint type for topology-only workers.
  • Prevented none from being combined with chat or completions endpoints.

• Documentation

  • Updated command-line help and ThunderAgent setup guidance.
  • Clarified routing behavior when backend services also expose public endpoints.

• Tests

  • Added coverage for valid topology-only configuration, invalid combinations, empty input, and unsupported endpoint types.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

🌿 Fern Docs Preview: https://nvidia-preview-019fb930-7947-76e3-856c-23a1a99fb663.docs.buildwithfern.com/dynamo/dev

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

[coderabbitai]

Walkthrough

Changes

Endpoint type support

Layer / File(s)	Summary
Parser validation and coverage components/src/dynamo/common/utils/endpoint_types.py, components/src/dynamo/common/utils/tests/test_endpoint_types.py	none maps to ModelType.Empty, cannot be combined with chat or completions, and is covered by validation tests.
Topology-only worker guidance components/src/dynamo/common/configuration/groups/runtime_args.py, components/src/dynamo/thunderagent_router/README.md	CLI help and ThunderAgent instructions document --endpoint-types none for topology-only workers.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes the main change: adding topology-only backend support for ThunderAgent.
Description check	✅ Passed	The description follows the template with Overview, Details, reviewer start, and a related issue link.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@components/src/dynamo/common/configuration/groups/runtime_args.py`:
- Around line 256-259: Update the argument help text for the runtime
endpoint-type option to explicitly state that “none” must be used by itself and
cannot be combined with “chat” or “completions”; keep the existing option
descriptions unchanged.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 00e9f79c-e8ff-4b91-b371-c4c721ab246f

📥 Commits

Reviewing files that changed from the base of the PR and between 769ee07 and 0a51fd6.

📒 Files selected for processing (4)

• components/src/dynamo/common/configuration/groups/runtime_args.py
• components/src/dynamo/common/utils/endpoint_types.py
• components/src/dynamo/common/utils/tests/test_endpoint_types.py
• components/src/dynamo/thunderagent_router/README.md

[cpakkamisaac-sae]

Hi @ishandhanani, this PR addresses the session-final GPU-forwarding/bypass issue described in #12016.

The root cause is that the ThunderAgent-wrapped vLLM backend can advertise the same public chat/completions model surface as ThunderAgent itself. In Kubernetes discovery, that allows the frontend to sometimes route x-dynamo-session-final: true requests directly to the backend, bypassing ThunderAgent’s short-circuit handler and wasting GPU work.

This PR adds explicit --endpoint-types none support so wrapped backends can remain discoverable for topology/readiness while ThunderAgent remains the only public chat/completions surface.

I validated the behavior on AKS with Dynamo Platform v1.3.0:

• Baseline: 1 of 6 x-dynamo-session-final: true requests generated tokens, and vLLM backend logs showed the final request reached backend.generate.
• With this change applied and the worker launched with --endpoint-types none: backend metadata advertised model_type: "", ThunderAgent advertised Chat | Completions, normal chat still worked, and 8 of 8 x-dynamo-session-final: true requests returned the expected empty ThunderAgent short-circuit response without backend generation.

So this does not close all of #12016, but it specifically fixes the session-final GPU-forwarding issue from that report.

[ishandhanani]

/ok-to-test fd5fcf7