Fix shelly tests for Python 3.14.3#166683
Conversation
justanotherariel
requested review from
bdraco,
bieniu,
chemelli74 and
thecode
as code owners
home-assistant
Bot
added
cla-signed
code-quality
has-tests
integration: shelly
small-pr
|
Hey there @bieniu, @thecode, @chemelli74, @bdraco, mind taking a look at this pull request as it has been labeled with an integration ( Code owner commandsCode owners of
|
There was a problem hiding this comment.
Pull request overview
Stabilizes Shelly integration tests under Python 3.14.3 by making “sleeping/offline” device scenarios deterministic and preventing teardown failures in config flow/device-trigger tests.
Changes:
- Make “sleeping device” tests deterministic by forcing
initialize()to raiseDeviceConnectionErroruntil the device is brought online. - Prevent unintended competing Bluetooth config flows in config flow tests by injecting BLE discovery while patching
flow.async_init. - Patch Shelly unload during mocked setup to avoid teardown errors when
entry.runtime_datais intentionally absent.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/components/shelly/conftest.py | Patch async_unload_entry alongside async_setup_entry in mock_setup_entry to avoid teardown errors in flow tests. |
| tests/components/shelly/test_config_flow.py | Add _async_inject_ble_discovery helper and update BLE discovery injections to avoid duplicate/competing flows; make sleeping-device paths deterministic. |
| tests/components/shelly/test_device_trigger.py | Patch Shelly unload in a runtime_data-negative scenario to avoid teardown failures. |
| tests/components/shelly/test_binary_sensor.py | Make sleeping RPC binary sensor tests deterministic via DeviceConnectionError on initialize(). |
| tests/components/shelly/test_button.py | Make sleeping RPC smoke mute button test deterministic by controlling initialize() behavior. |
| tests/components/shelly/test_update.py | Make sleeping RPC update entity test deterministic by controlling initialize() behavior. |
| tests/components/shelly/test_coordinator.py | Make late setup of sleeping RPC device deterministic by controlling initialize() behavior. |
| tests/components/shelly/test_climate.py | Ensure correct target temperature block selection in specific climate error-path tests via missing-attribute setup. |
| ) -> None: | ||
| """Test RPC online sleeping binary sensor.""" | ||
| entity_id = f"{BINARY_SENSOR_DOMAIN}.test_name_cloud" | ||
| mock_rpc_device.initialize.side_effect = DeviceConnectionError |
There was a problem hiding this comment.
This is incorrect and changes the role of this test
There was a problem hiding this comment.
Do you think this test failing is thus a bug in the integration itself and requires a seperate PR?
There was a problem hiding this comment.
It seems the core issue is a race condition in
The tests are using await init_integration with sleep_period=xyz, and changing them to use sleep_period=None fixes most of the issues.
However, this most likely highlights a deeper issue in the integration that should be investigated by the code owners.
There was a problem hiding this comment.
I will take a look, the problem is that I don't have an environment to reproduce it, if we change the tests so they pass but they don't really validate what they need to than we better disable them with a note that they need to be fixed.
I suggest to finish #169293 first and get back to this one
justanotherariel
force-pushed
the
3.14.3-fix-shelly
branch
from
b844b46 to
53b8aab
Compare
|
Is it possible to split this PR into multiple smaller PRs, each addressing a specific issue? |
|
Yes, I actually just wrote with @thecode and we decided to split this into 3 PRs: one PR for |
justanotherariel
force-pushed
the
3.14.3-fix-shelly
branch
from
9d0b123 to
696f552
Compare
|
As discussed, split the PR into 3:
Adjusted the PR description accordingly |
|
@justanotherariel I think this has been resolved with #169301 / #169304 / #169305 |
|
Yes indeed, thank you @epenet for taking care of it :) |
Breaking change
Proposed change
Python 3.14.3 changes how tasks are scheduled which suddenly makes many tests flaky. This PR (in addition to #166851 and #166850) addresses the flakyness of the shelly tests which is caused by a number of different things - all of them being bugs within the tests even before the python update, but were masked because of how tasks were scheduled. I will try to outline each failure case and the fix:
test_climate.py: Here we add twomonkeypatch.delattrcalls to the two tests that were missing it in the climate tests. Without them, the integration picks up the wrong 'block' as its temperature sensor (a valve block) and when it tries to read the temperature from it, it returns a mock instead of a int/float.test_device_trigger.py: We patch unload to make sure that the integration doesn't error on teardown, as unload tries to accessentry.runtime_dataotherwise when the device is connected (coordinator.py: 845 inis_rpc_ble_scanner_supported). I'm not sure if this is a bug and we should also/instead guard against in the shutdown method or if this can never happen in the real world.DeviceConnectionErrorwhich properly tests the functionality and makes them deterministic. There are more sleeping tests which don't require the same fix because it goes through different paths within the integration.Type of change
Additional information
Checklist
ruff format homeassistant tests)If user exposed functionality or configuration variables are added/changed:
If the code communicates with devices, web services, or third-party tools:
Updated and included derived files by running:
python3 -m script.hassfest.requirements_all.txt.Updated by running
python3 -m script.gen_requirements_all.To help with the load of incoming pull requests: