Repository navigation
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
f75240a to
4dd6f26
Compare
4dd6f26 to
6c7b0da
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 6c7b0da. Configure here.
| result: NamedProcesses = {} | ||
| for role, backend in self.config.active_role_backends(): | ||
| processes = [p for p in self.backend_processes if p.endpoint_mode == role] | ||
| result.update(self._start_workers(backend, processes)) |
There was a problem hiding this comment.
Colocated workers skip sequential start
Medium Severity
start_all_workers now launches each role through a separate _start_workers call. sequential_node_start therefore groups endpoints by node within one role only, so colocated prefill and decode on the same node are no longer batched and health-gated together. Concurrent model loads on shared GPUs can contend or OOM when a role engine uses per-endpoint launch.
Reviewed by Cursor Bugbot for commit 6c7b0da. Configure here.
|
|
||
| configured_backend = getattr(getattr(config, "backend", None), "type", self.required_backend) | ||
| if configured_backend != self.required_backend: | ||
| if self.required_backend is not None and configured_backend != self.required_backend: |
There was a problem hiding this comment.
Router uses serving engine only
Medium Severity
The static router now allows required_backend to be unset so mixed engines can start, but it still reads config.backend for worker scheme, bootstrap ports, and the type check. That object is a copy of the serving role only, so prefill workers can be advertised with decode-engine URL rules.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 6c7b0da. Configure here.
|
Closing because per-role engines and images were merged upstream in NVIDIA#529. |
1 participant
Allow independent engines and worker images per role, using existing backends with shared node/GPU placement and port allocation.
Recipes use either a shared top-level engine or an explicit engine on every role. Mixing the forms fails preflight, even when the engine types match. Role engines do not inherit options from each other.
Validation runs during configuration loading, before Slurm submission:
Configuration acceptance does not verify the connector installed inside an image or prove cross-engine KV compatibility.
Stack 1/2. TileRT support builds on this in #29.
Validation: 423 focused tests passed independently, without TileRT installed or registered. The latest update documents the existing validation boundary.