Skip to content

Fix potentially flaky tests in multiple integrations#155108

Closed
frenck wants to merge 1 commit intodevfrom
claude/review-pr-comment-011CUSCzbovysnh4RF2ANkRd
Closed

Fix potentially flaky tests in multiple integrations#155108
frenck wants to merge 1 commit intodevfrom
claude/review-pr-comment-011CUSCzbovysnh4RF2ANkRd

Conversation

@frenck
Copy link
Copy Markdown
Member

@frenck frenck commented Oct 24, 2025

Proposed change

Replace multiple consecutive async_block_till_done() calls with single async_block_till_done(wait_background_tasks=True) calls after async_fire_time_changed() in test files.

This pattern is more reliable and predictable than calling async_block_till_done() multiple times, which doesn't guarantee proper handling of all async operations and can lead to flaky tests.

Affected integrations:

  • nest: 1 instance in test_media_source.py
  • openrgb: 3 instances in test_init.py (including one with 3 consecutive calls)
  • playstation_network: 1 instance in test_image.py
  • rflink: 2 instances in test_binary_sensor.py
  • sun: 1 instance in test_init.py
  • template: 4 instances (1 in test_sensor.py, 3 in test_trigger.py)

Total: 14 instances fixed across 7 integrations.

Follows the same fix pattern from PR #155095 which addressed similar issues in the openrgb light tests.

🤖 Generated with Claude Code

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:
  • Link to documentation pull request:
  • Link to developer documentation pull request:
  • Link to frontend pull request:

Checklist

  • I understand the code I am submitting and can explain how it works.
  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Ruff (ruff format homeassistant tests)
  • Tests have been added to verify that the new code works.
  • Any generated code has been carefully reviewed for correctness and compliance with project standards.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.

To help with the load of incoming pull requests:

Replace multiple consecutive async_block_till_done() calls with single
async_block_till_done(wait_background_tasks=True) calls after
async_fire_time_changed() in test files.

This pattern is more reliable and predictable than calling
async_block_till_done() multiple times, which doesn't guarantee proper
handling of all async operations and can lead to flaky tests.

Affected integrations:
- knx: 2 instances in test_init.py
- nest: 1 instance in test_media_source.py
- openrgb: 3 instances in test_init.py (including one with 3 consecutive calls)
- playstation_network: 1 instance in test_image.py
- rflink: 2 instances in test_binary_sensor.py
- sun: 1 instance in test_init.py
- template: 4 instances (1 in test_sensor.py, 3 in test_trigger.py)

Total: 14 instances fixed across 7 integrations.

Follows the same fix pattern from PR #155095 which addressed similar
issues in the openrgb light tests.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings October 24, 2025 18:51
@frenck frenck requested review from a team, allenporter, farmio and marvin-w as code owners October 24, 2025 18:51
Copy link
Copy Markdown
Contributor

@home-assistant home-assistant Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @claude

It seems you haven't yet signed a CLA. Please do so here.

Once you do that we will be able to review and accept this pull request.

Thanks!

@home-assistant
Copy link
Copy Markdown
Contributor

Please take a look at the requested changes, and use the Ready for review button when you are done, thanks 👍

Learn more about our pull request process.

@home-assistant home-assistant Bot marked this pull request as draft October 24, 2025 18:52
@frenck frenck added smash Indicator this PR is close to finish for merging or closing and removed Hacktoberfest core cla-needed small-pr PRs with less than 30 lines. has-tests code-quality labels Oct 24, 2025
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR improves test reliability by replacing multiple consecutive async_block_till_done() calls with single async_block_till_done(wait_background_tasks=True) calls after async_fire_time_changed() operations. This pattern is more robust and predictable, as it explicitly waits for all background tasks to complete rather than requiring multiple iterations.

Key changes:

  • Consolidated 14 instances of double/triple async_block_till_done() calls into single calls with wait_background_tasks=True
  • Applied changes across 7 integrations (nest, openrgb, playstation_network, rflink, sun, template, knx)
  • Follows the established pattern from PR #155095

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/components/template/test_trigger.py Replaced 3 instances of double async_block_till_done() calls with single calls using wait_background_tasks=True parameter
tests/components/template/test_sensor.py Replaced 1 instance of double async_block_till_done() calls with single call using wait_background_tasks=True parameter
tests/components/sun/test_init.py Replaced 1 instance of double async_block_till_done() calls with single call using wait_background_tasks=True parameter
tests/components/rflink/test_binary_sensor.py Replaced 2 instances of double async_block_till_done() calls with single calls using wait_background_tasks=True parameter
tests/components/playstation_network/test_image.py Replaced 1 instance of double async_block_till_done() calls with single call using wait_background_tasks=True parameter
tests/components/openrgb/test_init.py Replaced 3 instances of multiple async_block_till_done() calls (including one with 3 consecutive calls) with single calls using wait_background_tasks=True parameter
tests/components/nest/test_media_source.py Replaced 1 instance of double async_block_till_done() calls with single call using wait_background_tasks=True parameter
tests/components/knx/test_init.py Replaced 2 instances of double async_block_till_done() calls with single calls using wait_background_tasks=True parameter

Copy link
Copy Markdown
Contributor

@home-assistant home-assistant Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @claude

It seems you haven't yet signed a CLA. Please do so here.

Once you do that we will be able to review and accept this pull request.

Thanks!

@frenck frenck changed the title Review GitHub PR Comment Suggestion Fix potentially flaky tests in multiple integrations Oct 24, 2025
@frenck frenck marked this pull request as ready for review October 24, 2025 18:55
@home-assistant home-assistant Bot dismissed stale reviews from themself October 24, 2025 18:55

Stale

Comment on lines +293 to +298
await hass.async_block_till_done(wait_background_tasks=True)
await knx.assert_no_telegram()

freezer.tick(timedelta(minutes=1)) # 60 minutes passed
async_fire_time_changed(hass)
await hass.async_block_till_done()
await hass.async_block_till_done()
await hass.async_block_till_done(wait_background_tasks=True)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't waiting for a HomeAssistant background task, but an xknx internal background task 😬

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I removed this one... not sure why it is still in the branch. I swear I removed it.

@frenck frenck marked this pull request as draft October 24, 2025 19:15
Copy link
Copy Markdown
Contributor

@allenporter allenporter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nest change looks good.

@github-actions
Copy link
Copy Markdown

There hasn't been any activity on this pull request recently. This pull request has been automatically marked as stale because of that and will be closed if no further activity occurs within 7 days.
If you are the author of this PR, please leave a comment if you want to keep it open. Also, please rebase your PR onto the latest dev branch to ensure that it's up to date with the latest changes.
Thank you for your contribution!

@github-actions github-actions Bot added the stale label Jan 14, 2026
@github-actions github-actions Bot closed this Jan 21, 2026
@github-actions github-actions Bot locked and limited conversation to collaborators Jan 22, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

cla-signed code-quality core has-tests small-pr PRs with less than 30 lines. smash Indicator this PR is close to finish for merging or closing stale

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants