Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow MQTT device based auto discovery #118757

Draft
wants to merge 2 commits into
base: dev
Choose a base branch
from
Draft

Conversation

jbouwh
Copy link
Contributor

@jbouwh jbouwh commented Jun 3, 2024

Proposed change

Rework #109030 which introduces a new device based MQTT discovery schema.

The original PR was reverted during the beta of HA Core 2024.6.0. With PR and branch we aim to test and optimize large MQTT setups that run via MQTT discovery.

Context

The current MQTT discovery model only allows to setup per component, if a device has multiple entities to published, the device context, availability and origin information needs to be duplicated.

This PR reduces IO overhead in the paho client on processing discovery packages: The baseline was 10000 packets, with device discovery it 6250 packet reads so a 37.5% reduction in I/O. Processing less I/O reduced the paho client run time by ~18%. The grouped discovery reduced the run time in the HA client by ~12%.

This PR adds a way to discover all components for a device with one discovery. The device, availability and origin mapping is only submitted once. Also the following options are allowed in the shared context:

  • state_topic
  • command_topic
  • encoding
  • qos

Except for device and origin options, supported shared options can be overridden in the component config.

Discovery updates and removal is fully supported.

The device based discovery coexists with the component based discovery, so no deprecation is planned for discovery with the single component schema to guarantee older devices keep working.

Note that some code and schemas that are related to device, availability and origin validation was moved to a new module schemas.py.

Example

The example below is of a device based auto discovery that supplies a sensor and binary_sensor.

Discovery topic:

homeassistant/device/test123/config

Payload:

{
  "dev": {
    "identifiers": [
      "mysensors123"
    ]
  },
  "avty": {
    "topic": "avty-topic"
  },
  "cmp": {
    "bins1": {
      "platform": "binary_sensor",
      "name": "Beer",
      "state_topic": "test-topic"
    },
    "sens1": {
      "platform": "sensor",
      "name": "Milk",
      "state_topic": "test-topic"
    }
  },
  "o": {
    "name": "My org",
    "sw": "bla2mqtt"
  },
  "qos": 0
}

Note that a required platform option is added to identify the component platform. The keys under the components (abbreviated cmp), are treated as object_id and are used to create a unique device discovery_id. In this case the discover_id's are test123 bins1 and test123 sens1.

In case a node_id is specified (e.g. node123) in the device discovery topic ...

homeassistant/device/node123/test123/config

... this will become part of the object_id.
The discovery_id 's become test123 node123 bins1 and test123 node123 sens1.

Migration from single topic to device based discovery

To allow a smooth migration from single topic discovery to device based discovery, the discovery_id for an mqtt item must be the same. Migration is supported of the single topic id has a node_id and a object_id. After migration the object id moves inside the discovery payload and the previous node_id becomes the new object_id of the device discovery topic.

The config to migrate to must have a migrate_discovery config option set to True. This will allow clearing the migrated discovery topics without removing them. So "migrate_discovery": true needs to be added to the JSON discovery payload(s) to migrate to. Migration is supported in both directions.

Example (device automation):

Discovery topic single: homeassistant/device_automation/0AFFD2/bla/config
Discovery id: 0AFFD2 bla
Discovery payload single:

{
  "automation_type": "trigger",
  "device": {
    "identifiers": [
      "0AFFD2"
    ]
  },
  "o": {
    "name": "foobar"
  },
  "payload": "short_press",
  "topic": "foobar/triggers/button1",
  "type": "button_short_press",
  "subtype": "button_1"
}

Migrated:

Discovery topic device: homeassistant/device/0AFFD2/config
Discovery id: 0AFFD2 bla
Discovery payload device:

{
  "device": {
    "identifiers": [
      "0AFFD2"
    ]
  },
  "o": {
    "name": "foobar"
  },
  "cmp": {
    "bla": {
      "automation_type": "trigger",
      "payload": "short_press",
      "topic": "foobar/triggers/button1",
      "type": "button_short_press",
      "subtype": "button_1",
      "platform": "device_automation"
    }
  },
  "migrate_discovery": true
}

If the new device discovery payload has the same discovery_id and comes after the single discovery payload. Home Assistant will publish None (retained) to the single discovery payload to remove it.

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Ruff (ruff format homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
  • Untested files have been added to .coveragerc.

To help with the load of incoming pull requests:

@home-assistant
Copy link

home-assistant bot commented Jun 3, 2024

Hey there @emontnemery, @bdraco, mind taking a look at this pull request as it has been labeled with an integration (mqtt) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of mqtt can trigger bot actions by commenting:

  • @home-assistant close Closes the pull request.
  • @home-assistant rename Awesome new title Renames the pull request.
  • @home-assistant reopen Reopen the pull request.
  • @home-assistant unassign mqtt Removes the current integration label and assignees on the pull request, add the integration domain after the command.
  • @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the pull request.
  • @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the pull request.

@corporategoth
Copy link

Since this is difficult to read for someone not involved. Does this still support specifying topics device-wide (ie. same for all entities), and then overriding them on a per-entity basis if required?

@jbouwh
Copy link
Contributor Author

jbouwh commented Jun 6, 2024

Since this is difficult to read for someone not involved. Does this still support specifying topics device-wide (ie. same for all entities), and then overriding them on a per-entity basis if required?

Effectively the PR is the same is the previous PR, but we love to seem some test results. We will try to update this branch as good as possible. A documentation PR is linked.

@jbouwh jbouwh force-pushed the mqtt-device-based-discovery branch from 9d516dd to 317bb4b Compare June 6, 2024 21:10
@jbouwh jbouwh force-pushed the mqtt-device-based-discovery branch 2 times, most recently from bdb0bf7 to 85d2546 Compare June 25, 2024 14:19
@@ -138,7 +138,9 @@ def remove_trigger_discovery_data(
hass: HomeAssistant, discovery_hash: tuple[str, str]
) -> None:
"""Remove discovery data."""
del hass.data[DATA_MQTT].debug_info_triggers[discovery_hash]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to #120430

discovery_data: DiscoveryInfoType


def clear_discovery_hash(hass: HomeAssistant, discovery_hash: tuple[str, str]) -> None:
"""Clear entry from already discovered list."""
hass.data[DATA_MQTT].discovery_already_discovered.remove(discovery_hash)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to #120430

@@ -180,5 +181,6 @@ async def async_tear_down(self) -> None:
self._sub_state = subscription.async_unsubscribe_topics(
self.hass, self._sub_state
)
if self.device_id:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to #120430

@jbouwh jbouwh force-pushed the mqtt-device-based-discovery branch from ffd153b to 51fda9a Compare June 27, 2024 18:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants