Skip to content

test(vllm): drop tests of APIs removed by PR #22 refactor - #43

Closed
aoshen02 wants to merge 1 commit into
update_weights_tensorfrom
aoshen/pr22-fix-test-drift
Closed

test(vllm): drop tests of APIs removed by PR #22 refactor#43
aoshen02 wants to merge 1 commit into
update_weights_tensorfrom
aoshen/pr22-fix-test-drift

Conversation

@aoshen02

Copy link
Copy Markdown
Collaborator

Summary

Follow-up to @knlnguyen1802's request on #22 (comment) to send a small PR against update_weights_tensor rather than a manual edit.

Same kind of test/production drift as the one cleaned up in commit 2ee402b "Fix test", but for the other 8 unit tests that were also broken by the 132066f "Fix colocated mode weight sync" refactor.

On the current update_weights_tensor HEAD, pytest tests/unit/backends/vllm_utils/test_vllm_engine.py returns 8 failed, 18 passed. After this PR: 20 passed.

What broke

PR #22 commit 132066f removed / inlined three things in slime/backends/vllm_utils/vllm_engine.py:

Removed API Replacement
VLLMEngine._weight_transfer_http_timeout() Inlined float(os.environ.get("SLIME_VLLM_WEIGHT_TRANSFER_HTTP_TIMEOUT_SEC", "900")) in start_weight_update / finish_weight_update; _post_vllm_update_weights_http uses a slightly different env-var precedence
Module-level _response_json_or_fallback(response) Inlined try: return response.json() except Exception: return {"ok": True, "raw": response.text}
_http_base() init-guard (RuntimeError when server_host unset) Removed; now bare attribute access

But PR #21 had added tests against those exact APIs in tests/unit/backends/vllm_utils/test_vllm_engine.py, and commit 2ee402b only cleaned up the two _skipped_if_not_leader tests, leaving the other 8 drifting.

What this PR does

Deletes 6 tests of fully-removed APIs:

  • test_weight_transfer_http_timeout_reads_env
  • test_weight_transfer_http_timeout_fallback_to_legacy_env
  • test_response_json_or_fallback_parses_dict
  • test_response_json_or_fallback_non_dict_wrapped
  • test_response_json_or_fallback_invalid_json
  • test_http_base_requires_init (the happy path is still covered by test_http_base_ipv6_host)

Trims the dead _weight_transfer_http_timeout() reference from 2 tests but keeps their endpoint/payload assertions, which still pin real production behavior:

  • test_start_weight_update_posts_four_phase_endpoint
  • test_finish_weight_update_posts_empty_body

Verification

$ pytest tests/unit/backends/vllm_utils/test_vllm_engine.py -q
....................                                                     [100%]
20 passed in 2.24s

$ pytest tests/unit/ --ignore=tests/unit/backends/megatron_utils -q
53 passed, 2 warnings in 29.38s

$ pre-commit run --all-files
... all hooks Passed

Diff is +3/-52 in a single file.

PR #22 commit 132066f ("Fix colocated mode weight sync") removed/inlined
several private helpers on ``VLLMEngine`` and the module-level
``_response_json_or_fallback``, but the unit tests for them (added in
PR #21) were not updated. They fail on the PR #22 HEAD with
``AttributeError``.

Apply the same cleanup as commit 2ee402b ("Fix test", which removed the
``_skipped_if_not_leader`` tests) to the remaining drift:

- Delete the tests for ``_weight_transfer_http_timeout`` (inlined into
  ``start_weight_update`` / ``finish_weight_update`` /
  ``_post_vllm_update_weights_http``).
- Delete the tests for ``_response_json_or_fallback`` (the helper was
  removed; the inline replacement uses a simpler ``try: response.json()``
  fallback that doesn't reproduce the wrapping behavior the tests asserted).
- Delete ``test_http_base_requires_init`` (the explicit RuntimeError guard
  was removed; the test ``test_http_base_ipv6_host`` already covers the
  happy path).
- In ``test_start_weight_update_posts_four_phase_endpoint`` and
  ``test_finish_weight_update_posts_empty_body``, drop the dead reference
  to ``_weight_transfer_http_timeout`` while keeping the endpoint/payload
  assertions, which still test real production behavior.

Result: ``pytest tests/unit/backends/vllm_utils/test_vllm_engine.py``
goes from 8 failed / 18 passed to 20 passed.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request removes several unit tests and assertions related to weight transfer HTTP timeouts and JSON response fallback handling in test_vllm_engine.py. The reviewer suggests that instead of completely removing the timeout assertions, the tests should verify that the default timeout of 900.0 seconds is correctly passed.

@@ -83,7 +83,6 @@ def fake_post(endpoint: str, payload: dict, timeout: float):
assert len(calls) == 1
assert calls[0][0] == "start_weight_update"
assert calls[0][1] == {"is_checkpoint_format": True}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Instead of completely removing the timeout assertion, we should verify that the default timeout of 900.0 seconds is correctly passed to the _post_json call.

Suggested change
assert calls[0][1] == {"is_checkpoint_format": True}
assert calls[0][1] == {"is_checkpoint_format": True}
assert calls[0][2] == 900.0

Comment on lines +101 to +103
assert len(calls) == 1
assert calls[0][0] == "finish_weight_update"
assert calls[0][1] == {}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Instead of dropping the timeout assertion, we should verify that the default timeout of 900.0 seconds is correctly passed to the _post_json call.

Suggested change
assert len(calls) == 1
assert calls[0][0] == "finish_weight_update"
assert calls[0][1] == {}
assert len(calls) == 1
assert calls[0][0] == "finish_weight_update"
assert calls[0][1] == {}
assert calls[0][2] == 900.0

@knlnguyen1802

Copy link
Copy Markdown
Collaborator

@aoshen02 Let's fix in on main branch

@aoshen02

Copy link
Copy Markdown
Collaborator Author

PR #22 is now merged into main; reopening against main with broader scope to also restore the production behavior that the test drift was hiding (timeout-env precedence and response-parsing semantics) — see follow-up PR.

@aoshen02 aoshen02 closed this May 27, 2026
@aoshen02
aoshen02 deleted the aoshen/pr22-fix-test-drift branch June 8, 2026 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants