refactor(arguments): align with slime + revert spurious dist.gather_object patch - #233
refactor(arguments): align with slime + revert spurious dist.gather_object patch#233aoshen02 wants to merge 1 commit into
Conversation
…rt spurious dist.gather_object patch arguments.py (415→361 lines): - Trim module docstring and inline comments throughout - Slim SKIPPED_DESTS to names-only (remove per-entry rationale) - Add mutual-exclusion assertions in validate_args mirroring slime (vllm_config / prefill_num_servers / rollout_external conflict checks) - Note: _make_add_argument_wrapper stays public (5 unit tests reference it by name); validate_args intentionally omits vllm_tp_size (pinned by test — global TP broke heterogeneous per-group engines, per-engine resolver owns it) reloadable_process_group.py: - Drop dist.gather_object monkey-patch (added by PR #22, never in slime; unused in vime — callers explicitly use all_gather_object to stay on the patched path) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request refactors docstrings and comments in vime/backends/vllm_utils/arguments.py, introduces mutual exclusion validation checks for prefill_num_servers, vllm_config, and rollout_external, and removes the patching of dist.gather_object in vime/utils/reloadable_process_group.py. The reviewer feedback suggests converting SKIPPED_DESTS from a list to a frozenset for more efficient O(1) lookups, and replacing the runtime assert statements with explicit ValueError exceptions to ensure validation is not bypassed under Python optimization flags.
| SKIPPED_DESTS = [ | ||
| # model identity: hf_checkpoint owns this | ||
| "model", | ||
| "served_model_name", | ||
| "config", | ||
| # tokenizer: vime uses its own | ||
| "tokenizer", | ||
| "tokenizer_mode", | ||
| "tokenizer_revision", | ||
| # security toggle: always-on for vime's curated checkpoints | ||
| "trust_remote_code", | ||
| # seed: vime computes args.seed + rank | ||
| "seed", | ||
| # dtype: vime --fp16 / training config owns this | ||
| "dtype", | ||
| # tp_size is fully owned by the orchestrator (rollout_num_gpus_per_engine | ||
| # // pp_size) — see validate_args. pipeline_parallel_size and | ||
| # data_parallel_size remain user-controllable and auto-forward to the vllm | ||
| # subprocess when set. | ||
| # TP is orchestrator-owned; PP/DP remain user-controllable and auto-forward. | ||
| "tensor_parallel_size", | ||
| # multi-node logical engine (orchestrator-owned) | ||
| "nnodes", | ||
| "node_rank", | ||
| "master_addr", | ||
| "master_port", | ||
| "data_parallel_backend", | ||
| "distributed_executor_backend", | ||
| # network: engine launcher decides per-engine port/host | ||
| "port", | ||
| "host", | ||
| # vime decides this based on training algo, not user CLI | ||
| "enable_return_routed_experts", | ||
| ] |
There was a problem hiding this comment.
Using a frozenset instead of a list for SKIPPED_DESTS improves lookup efficiency from canonical in SKIPPED_DESTS). Since this collection is only used for membership tests, a set-based structure is more idiomatic and performant.
SKIPPED_DESTS = frozenset({
"model",
"served_model_name",
"config",
"tokenizer",
"tokenizer_mode",
"tokenizer_revision",
"trust_remote_code",
"seed",
"dtype",
# TP is orchestrator-owned; PP/DP remain user-controllable and auto-forward.
"tensor_parallel_size",
"nnodes",
"node_rank",
"master_addr",
"master_port",
"data_parallel_backend",
"distributed_executor_backend",
"port",
"host",
"enable_return_routed_experts",
})| assert not ( | ||
| getattr(args, "prefill_num_servers", None) is not None and getattr(args, "rollout_external", False) | ||
| ), "prefill_num_servers cannot be set with --rollout-external-engine-addrs." | ||
|
|
||
| assert not ( | ||
| getattr(args, "vllm_config", None) is not None and getattr(args, "rollout_external", False) | ||
| ), "vllm_config cannot be set with --rollout-external-engine-addrs." | ||
|
|
||
| assert not ( | ||
| getattr(args, "vllm_config", None) is not None and getattr(args, "prefill_num_servers", None) is not None | ||
| ), "vllm_config and prefill_num_servers are mutually exclusive. Use server_groups in the YAML config instead." |
There was a problem hiding this comment.
Avoid using assert statements for runtime configuration validation. In Python, assertions can be globally disabled when the interpreter is run with optimization flags (e.g., python -O), which would bypass these mutual-exclusion checks entirely. Use explicit if conditions and raise ValueError instead.
| assert not ( | |
| getattr(args, "prefill_num_servers", None) is not None and getattr(args, "rollout_external", False) | |
| ), "prefill_num_servers cannot be set with --rollout-external-engine-addrs." | |
| assert not ( | |
| getattr(args, "vllm_config", None) is not None and getattr(args, "rollout_external", False) | |
| ), "vllm_config cannot be set with --rollout-external-engine-addrs." | |
| assert not ( | |
| getattr(args, "vllm_config", None) is not None and getattr(args, "prefill_num_servers", None) is not None | |
| ), "vllm_config and prefill_num_servers are mutually exclusive. Use server_groups in the YAML config instead." | |
| if getattr(args, "prefill_num_servers", None) is not None and getattr(args, "rollout_external", False): | |
| raise ValueError("prefill_num_servers cannot be set with --rollout-external-engine-addrs.") | |
| if getattr(args, "vllm_config", None) is not None and getattr(args, "rollout_external", False): | |
| raise ValueError("vllm_config cannot be set with --rollout-external-engine-addrs.") | |
| if getattr(args, "vllm_config", None) is not None and getattr(args, "prefill_num_servers", None) is not None: | |
| raise ValueError("vllm_config and prefill_num_servers are mutually exclusive. Use server_groups in the YAML config instead.") |
|
Merging into PR #232 instead. |
Summary
vime/backends/vllm_utils/arguments.py(415→361 lines): trim verbose docstrings/comments, slimSKIPPED_DESTSto names-only, add mutual-exclusion assertions invalidate_argsmirroring slime (vllm_config/prefill_num_servers/rollout_externalconflict checks)._make_add_argument_wrapperstays public (5 unit tests reference it by name);validate_argsintentionally omitsvllm_tp_size(global TP broke heterogeneous per-group engines — per-engine resolver owns it, pinned by test).vime/utils/reloadable_process_group.py: dropdist.gather_objectmonkey-patch (added by PR [feat] Support Colocated Weight Sync via CUDA IPC for vime #22, never in slime; unused in vime — callers explicitly useall_gather_objectto stay on the patched path).Test plan
pytest tests/unit/backends/vllm_utils/test_arguments.py -m unit— all argument contract tests passpytest tests/ -m unit— full unit suite🤖 Generated with Claude Code