[cquil11]

Summary

RFC: Per-role inference engines and container images (#516).

Adds vLLM-prefill / TileRT-decode disaggregation. I don’t have write permissions here to set up stacked PRs, so the implementation is split into two commits for review:

1. Per-role engines and images: dispatch placement and worker launch through each role's backend. Recipes use either a shared engine or explicit engines on every role, never both. Unsupported frontend/engine combinations fail before submission.
2. TileRT backend and router: launch decode workers and TileRT's P/D router with allocated addresses/ports, worker readiness, and frontend-aware metrics discovery. Includes NIXL and Mooncake examples. Currently requires vLLM prefill with TileRTConnector.

The adapter follows TileRT 0.1.6's decode server and router. The public model name comes from vLLM prefill. Native workers no longer inherit Dynamo's launch defaults. Benchmark API limitations are documented in docs/tilert.md.

Validation

Cluster / transport	InferenceX run	Result
B200 / NIXL	36508942342	1k1k and 8k1k passed; 16/16 requests each. 8k1k power validated across both nodes.
MI355X / Mooncake	36512786185	3,600-second AgentX profile passed; 271 successful requests, zero errors. Prefill metrics validated.

Hardware runs used fork revision b421420b, before rebasing onto current upstream. No new cluster runs or accuracy evaluations were performed for the rebase or review fixes. Locally: both TileRT dry-runs and the shared-engine vLLM dry-run, 548 focused tests, Ruff, Linux-target type checking, and schema checks passed. The broader suite has 21 macOS baseline failures, reproduced on unchanged upstream 92fbe04c (including MLPerf snapshot path ordering).

InferenceX ports

PR	Scope
#3551	Shared TileRT integration
#3553	GLM-5.1 B200 fixed-sequence
#3552	GLM-5.3 MI355X AgentX

[codecov-commenter]

Codecov Report

❌ Patch coverage is 92.56595% with 31 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@d700e38). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
src/srtctl/frontends/tilert_router.py	86.30%	10 Missing ⚠️
src/srtctl/backends/tilert.py	92.00%	6 Missing ⚠️
src/srtctl/core/roles.py	89.47%	6 Missing ⚠️
src/srtctl/cli/do_sweep.py	50.00%	2 Missing ⚠️
src/srtctl/cli/submit.py	80.00%	2 Missing ⚠️
src/srtctl/core/schema.py	96.96%	2 Missing ⚠️
src/srtctl/cli/mixins/benchmark_stage.py	91.66%	1 Missing ⚠️
src/srtctl/cli/mixins/worker_stage.py	98.30%	1 Missing ⚠️
src/srtctl/core/worker_backends.py	96.55%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #529   +/-   ##
=======================================
  Coverage        ?   83.52%           
=======================================
  Files           ?      157           
  Lines           ?    22471           
  Branches        ?        0           
=======================================
  Hits            ?    18769           
  Misses          ?     3702           
  Partials        ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.