Skip to content

[Tests] Replace flaky sleep with polling in test_background_cancel #32986

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
robertgshaw2-redhat merged 2 commits into from
Jan 24, 2026
Merged

[Tests] Replace flaky sleep with polling in test_background_cancel #32986

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
20e45e4
[Tests] Replace flaky sleep with polling in test_background_cancel
sjhddh Jan 23, 2026
2ca94c6
Address review: use monotonic clock for accurate timeout
sjhddh Jan 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 19 additions & 6 deletions tests/v1/entrypoints/openai/serving_responses/test_stateful.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,15 +70,28 @@ async def test_background_cancel(client: openai.AsyncOpenAI):
assert response.status == "queued"

# Cancel the response before it is completed.
# FIXME: This test can be flaky.
await asyncio.sleep(0.5)
# Poll until the response is no longer queued (started processing) or timeout
loop = asyncio.get_running_loop()
start_time = loop.time()
max_wait_seconds = 5.0
poll_interval = 0.1
while loop.time() - start_time < max_wait_seconds:
response = await client.responses.retrieve(response.id)
if response.status != "queued":
# Started processing or completed - try to cancel
break
Comment on lines +76 to +82
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The current polling loop uses elapsed += poll_interval to track time, but this doesn't account for the time spent in the client.responses.retrieve(response.id) call. This can cause the effective timeout to be significantly longer than max_wait_seconds if the API call is slow, potentially slowing down tests in CI.

Using a monotonic clock like asyncio.get_running_loop().time() provides a more accurate and robust timeout mechanism. I've also reordered the loop to check the condition before sleeping, which is slightly more efficient.

Suggested change
max_wait_seconds = 5.0
poll_interval = 0.1
elapsed = 0.0
while elapsed < max_wait_seconds:
await asyncio.sleep(poll_interval)
elapsed += poll_interval
response = await client.responses.retrieve(response.id)
if response.status != "queued":
# Started processing or completed - try to cancel
break
loop = asyncio.get_running_loop()
start_time = loop.time()
max_wait_seconds = 5.0
poll_interval = 0.1
while loop.time() - start_time < max_wait_seconds:
response = await client.responses.retrieve(response.id)
if response.status != "queued":
# Started processing or completed - try to cancel
break
await asyncio.sleep(poll_interval)

await asyncio.sleep(poll_interval)

response = await client.responses.cancel(response.id)
assert response.status == "cancelled"

# Make sure the response status remains unchanged.
await asyncio.sleep(5)
response = await client.responses.retrieve(response.id)
assert response.status == "cancelled"
# Make sure the response status remains unchanged after some time.
max_retries = 10
for _ in range(max_retries):
await asyncio.sleep(0.5)
response = await client.responses.retrieve(response.id)
# Verify status is still cancelled
assert response.status == "cancelled"


@pytest.mark.asyncio
Expand Down