fix(planner): resolve aggregate workers by component type - #11578
Conversation
|
👋 Hi waynehacking8! Thank you for contributing to ai-dynamo/dynamo. Just a reminder: The 🚀 |
WalkthroughChangesAggregate worker resolution
Estimated code review effort: 3 (Moderate) | ~20 minutes 🚥 Pre-merge checks | ✅ 2 | ❌ 3❌ Failed checks (3 warnings)
✅ Passed checks (2 passed)
Comment |
|
Applied in 77f8bba. GPU-count resolution now gets the same explicit names (dynamo-review-agent's catch), and resolve_worker_info forwards decode_component_name to the model-name fallback (Devin's first point -- in practice that path only fires when the DGD args carry no model at all, since get_worker_info's arg-parsing fallback already used the name, but forwarding it makes the residual case fail with the truthful error). Devin's second point actually cut the other way once I checked the recipes: the repo's own vllm agg examples are split between VllmWorker (agg_failover, agg_gms) and VllmDecodeWorker (agg, agg_router, agg_kvbm), so a single static name would regress one shape -- resolution now takes ordered candidates (agg name, then decode name), which also covers the sglang/trtllm agg recipes via their decode names. |
77f8bba to
cda929e
Compare
|
Rebased over #10767, which moved deployment bootstrapping out of core/base.py into the environment facade -- the candidate derivation now lives in PlannerEnvironmentImpl._decode_component_name (environment/base.py) and the connector-side threading is unchanged. Re-ran the planner unit suites against current main (identical pass/fail set to main in my environment) and updated the PR body to match the new structure. |
|
@waynehacking8 It seems like your PR overlaps with mine #9848. I'm shortening the vllm renames but have a problem with making it backwards compatible. Maybe we can try to collab on getting both of these in together? |
|
Happy to collab -- I think the two compose cleanly. This PR makes the planner's decode-role resolution try ordered name candidates read from the backend defaults, so your rename flows through automatically once the defaults say prefill/decode/worker, and the candidate list is also a natural home for the legacy fallback sttts asked about (try "worker" then "VllmWorker", so pre-rename DGDs keep resolving on the planner side -- it does not help the operator/override side of your change). If yours lands first I'll rebase mine over it. |
|
I think my PR is blocked on having backwards compatibility so yours would probably be a better candidate to merge first. I think we shouldn't rely on hard coded name lookups and instead have the selector choose based on the component type. That way, if we decide to change the name again, we'll need to maintain Do you want to take this? Or could I PR into your branch or something? |
|
@waynehacking8 Also are you on the CNCF Slack? Could ping you there too in case we need to chat/brainstorm some more. |
Aggregate v1beta1 DGDs expose one generic type: worker component, so exact backend-specific names make planner startup and scaling depend on recipe naming. Treat the sole generic worker as the decode component while keeping explicit-name disambiguation for legacy disaggregated deployments with multiple generic workers. Fixes ai-dynamo#11491 Signed-off-by: WEI CHENG CHIU <waynehacking8@gmail.com>
cda929e to
4229039
Compare
|
@Jont828 Yes, I took this. I replaced the component name candidate plumbing with a type based fallback: a single generic |
|
@waynehacking8 Thanks for doing this! Btw, @sttts left a comment on my PR for renaming "VllmPrefillWorker" -> "prefill".
Does this component based matching work for overrides as well? Meaning if someone forgot to update the |
|
No. This PR only changes planner lookup after the DGD exists. DGDR overrides are merged earlier by exact |
|
/ok to test 4229039 |
|
/ok to test 4229039 |
Summary
type: workercomponent as the planner's decode component, regardless of its name.This fixes #11491 and lets #9848 rename vLLM workers without carrying planner-side aliases for every historical name.
Validation
KubernetesConnector.validate_deployment: an aggregate DGD with an arbitrary worker name fails before this change and passes after it.test_kubernetes_connector.py: 75 passed.Live-cluster startup and scaling were not run because they require a real aggregate deployment.