Skip to content

fix(vllm): tolerate empty 200 body in _response_json (/sleep, /wake_up) #80

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
aoshen02 wants to merge 2 commits into from
Closed

fix(vllm): tolerate empty 200 body in _response_json (/sleep, /wake_up) #80

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9b60e68
fix(vllm): tolerate empty 200 body in _response_json (/sleep, /wake_up)
aoshen02 May 30, 2026
4bb005f
test(vllm): cover empty control-plane responses
aoshen02 May 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions slime/backends/vllm_utils/vllm_engine.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,13 @@ def _response_json(response: requests.Response) -> dict:
except requests.exceptions.HTTPError as e:
e.add_note(f"{response.text=}")
raise
# Some vLLM control-plane endpoints (e.g. POST /sleep, /wake_up) return an
# HTTP 200 with an EMPTY body. response.json() would then raise
# JSONDecodeError("Expecting value: line 1 column 1 (char 0)") and break
# colocate memory offload (release_memory_occupation / resume_memory_occupation).
# Treat an empty body as an empty result.
if not response.content or not response.text.strip():
return {}
Comment on lines +46 to +47

@gemini-code-assist gemini-code-assist Bot May 30, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using response.text can be inefficient because it triggers automatic character encoding detection (which can be slow and CPU-intensive if charset_normalizer or chardet is invoked) and decodes the entire response body into a string.\n\nSince response.content is a bytes object, and bytes supports the .strip() method in Python, we can check if the body is empty or contains only whitespace directly on the raw bytes. This is much more efficient and avoids any encoding detection overhead.

    if not response.content.strip():\n        return {}

return response.json()


Expand Down
25 changes: 23 additions & 2 deletions tests/unit/backends/vllm_utils/test_vllm_engine.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,25 @@


class _MockResponse:
def __init__(self, *, json_data: dict | None = None, text: str = "", status_code: int = 200):
def __init__(
self,
*,
json_data: dict | None = None,
text: str = "",
status_code: int = 200,
content: bytes | None = None,
):
self._json_data = json_data
self.text = text
self.text = text if text or json_data is None else "{}"
self.status_code = status_code
if content is not None:
self.content = content
elif text:
self.content = text.encode()
elif json_data is not None:
self.content = self.text.encode()
else:
self.content = b""

def raise_for_status(self) -> None:
if self.status_code >= 400:
Expand Down Expand Up @@ -277,6 +292,12 @@ def test_response_json_parses_dict():
assert mod._response_json(response) == {"status": "ready"}


@pytest.mark.unit
def test_response_json_empty_body_returns_empty_dict():
response = _MockResponse()
assert mod._response_json(response) == {}


@pytest.mark.unit
def test_response_json_invalid_json_raises():
response = _MockResponse(text="not-json")
Expand Down
Loading