Skip to content

Added regression test for openai/harmony/issues/78 #29830

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jacobthebanana wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Added regression test for openai/harmony/issues/78 #29830

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f44dda6
Added regression test for openai/harmony/issues/78
jacobthebanana Dec 2, 2025
e19140f
Clarified "content" is supposed to be part of each message per OAI
jacobthebanana Dec 2, 2025
1eba6cf
Whitespace fixes
jacobthebanana Dec 2, 2025
a3ea500
ruff formatting fixes for est_response_api_with_harmony
jacobthebanana Dec 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions tests/entrypoints/openai/test_response_api_with_harmony.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -864,6 +864,20 @@ async def test_output_messages_enabled(client: OpenAI, model_name: str, server):
assert len(response.input_messages) > 0
assert len(response.output_messages) > 0

# Regression test for github.com/openai/harmony/issues/78 (empty content)
is_query_returned: bool = False
for _message in [*response.input_messages, *response.output_messages]:
for _item in _message["content"]:
assert isinstance(_item, dict), _message
assert len(_item) > 0, _message

# Ensure original input is returned
_item_text = _item.get("text")
if _item_text and "South Korea" in _item_text:
is_query_returned = True

assert is_query_returned


@pytest.mark.asyncio
@pytest.mark.parametrize("model_name", [MODEL_NAME])
Expand Down