Skip to content

refactor(arguments): align with slime + revert spurious dist.gather_object patch - #233

Closed
aoshen02 wants to merge 1 commit into
mainfrom
refactor/arguments-revert-gather-object
Closed

refactor(arguments): align with slime + revert spurious dist.gather_object patch#233
aoshen02 wants to merge 1 commit into
mainfrom
refactor/arguments-revert-gather-object

Conversation

@aoshen02

Copy link
Copy Markdown
Collaborator

Summary

  • vime/backends/vllm_utils/arguments.py (415→361 lines): trim verbose docstrings/comments, slim SKIPPED_DESTS to names-only, add mutual-exclusion assertions in validate_args mirroring slime (vllm_config / prefill_num_servers / rollout_external conflict checks). _make_add_argument_wrapper stays public (5 unit tests reference it by name); validate_args intentionally omits vllm_tp_size (global TP broke heterogeneous per-group engines — per-engine resolver owns it, pinned by test).

  • vime/utils/reloadable_process_group.py: drop dist.gather_object monkey-patch (added by PR [feat] Support Colocated Weight Sync via CUDA IPC for vime #22, never in slime; unused in vime — callers explicitly use all_gather_object to stay on the patched path).

Test plan

  • pytest tests/unit/backends/vllm_utils/test_arguments.py -m unit — all argument contract tests pass
  • pytest tests/ -m unit — full unit suite

🤖 Generated with Claude Code

…rt spurious dist.gather_object patch

arguments.py (415→361 lines):
- Trim module docstring and inline comments throughout
- Slim SKIPPED_DESTS to names-only (remove per-entry rationale)
- Add mutual-exclusion assertions in validate_args mirroring slime
  (vllm_config / prefill_num_servers / rollout_external conflict checks)
- Note: _make_add_argument_wrapper stays public (5 unit tests reference it
  by name); validate_args intentionally omits vllm_tp_size (pinned by test —
  global TP broke heterogeneous per-group engines, per-engine resolver owns it)

reloadable_process_group.py:
- Drop dist.gather_object monkey-patch (added by PR #22, never in slime;
  unused in vime — callers explicitly use all_gather_object to stay on the
  patched path)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors docstrings and comments in vime/backends/vllm_utils/arguments.py, introduces mutual exclusion validation checks for prefill_num_servers, vllm_config, and rollout_external, and removes the patching of dist.gather_object in vime/utils/reloadable_process_group.py. The reviewer feedback suggests converting SKIPPED_DESTS from a list to a frozenset for more efficient O(1) lookups, and replacing the runtime assert statements with explicit ValueError exceptions to ensure validation is not bypassed under Python optimization flags.

Comment on lines 70 to 91
SKIPPED_DESTS = [
# model identity: hf_checkpoint owns this
"model",
"served_model_name",
"config",
# tokenizer: vime uses its own
"tokenizer",
"tokenizer_mode",
"tokenizer_revision",
# security toggle: always-on for vime's curated checkpoints
"trust_remote_code",
# seed: vime computes args.seed + rank
"seed",
# dtype: vime --fp16 / training config owns this
"dtype",
# tp_size is fully owned by the orchestrator (rollout_num_gpus_per_engine
# // pp_size) — see validate_args. pipeline_parallel_size and
# data_parallel_size remain user-controllable and auto-forward to the vllm
# subprocess when set.
# TP is orchestrator-owned; PP/DP remain user-controllable and auto-forward.
"tensor_parallel_size",
# multi-node logical engine (orchestrator-owned)
"nnodes",
"node_rank",
"master_addr",
"master_port",
"data_parallel_backend",
"distributed_executor_backend",
# network: engine launcher decides per-engine port/host
"port",
"host",
# vime decides this based on training algo, not user CLI
"enable_return_routed_experts",
]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using a frozenset instead of a list for SKIPPED_DESTS improves lookup efficiency from $O(N)$ to $O(1)$ when performing membership checks (e.g., canonical in SKIPPED_DESTS). Since this collection is only used for membership tests, a set-based structure is more idiomatic and performant.

SKIPPED_DESTS = frozenset({
    "model",
    "served_model_name",
    "config",
    "tokenizer",
    "tokenizer_mode",
    "tokenizer_revision",
    "trust_remote_code",
    "seed",
    "dtype",
    # TP is orchestrator-owned; PP/DP remain user-controllable and auto-forward.
    "tensor_parallel_size",
    "nnodes",
    "node_rank",
    "master_addr",
    "master_port",
    "data_parallel_backend",
    "distributed_executor_backend",
    "port",
    "host",
    "enable_return_routed_experts",
})

Comment on lines +268 to +278
assert not (
getattr(args, "prefill_num_servers", None) is not None and getattr(args, "rollout_external", False)
), "prefill_num_servers cannot be set with --rollout-external-engine-addrs."

assert not (
getattr(args, "vllm_config", None) is not None and getattr(args, "rollout_external", False)
), "vllm_config cannot be set with --rollout-external-engine-addrs."

assert not (
getattr(args, "vllm_config", None) is not None and getattr(args, "prefill_num_servers", None) is not None
), "vllm_config and prefill_num_servers are mutually exclusive. Use server_groups in the YAML config instead."

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Avoid using assert statements for runtime configuration validation. In Python, assertions can be globally disabled when the interpreter is run with optimization flags (e.g., python -O), which would bypass these mutual-exclusion checks entirely. Use explicit if conditions and raise ValueError instead.

Suggested change
assert not (
getattr(args, "prefill_num_servers", None) is not None and getattr(args, "rollout_external", False)
), "prefill_num_servers cannot be set with --rollout-external-engine-addrs."
assert not (
getattr(args, "vllm_config", None) is not None and getattr(args, "rollout_external", False)
), "vllm_config cannot be set with --rollout-external-engine-addrs."
assert not (
getattr(args, "vllm_config", None) is not None and getattr(args, "prefill_num_servers", None) is not None
), "vllm_config and prefill_num_servers are mutually exclusive. Use server_groups in the YAML config instead."
if getattr(args, "prefill_num_servers", None) is not None and getattr(args, "rollout_external", False):
raise ValueError("prefill_num_servers cannot be set with --rollout-external-engine-addrs.")
if getattr(args, "vllm_config", None) is not None and getattr(args, "rollout_external", False):
raise ValueError("vllm_config cannot be set with --rollout-external-engine-addrs.")
if getattr(args, "vllm_config", None) is not None and getattr(args, "prefill_num_servers", None) is not None:
raise ValueError("vllm_config and prefill_num_servers are mutually exclusive. Use server_groups in the YAML config instead.")

@aoshen02

Copy link
Copy Markdown
Collaborator Author

Merging into PR #232 instead.

@aoshen02 aoshen02 closed this Jun 10, 2026
@aoshen02
aoshen02 deleted the refactor/arguments-revert-gather-object branch June 10, 2026 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant