Include all non-numeric sensor events in logbook

[emontnemery]

Proposed change

Include all non-numeric sensor events in logbook

• Numeric sensors are assumed to be sensors which have a state class or a unit of measurement

This PR makes the following changes:

• Consider sensors with a state class numeric / continuous, and thus excluded from the log book
• Don't group non-numeric sensor or proximity events by 15-minute intervals

Type of change

• Dependency upgrade
• Bugfix (non-breaking change which fixes an issue)
• New integration (thank you!)
• New feature (which adds functionality to an existing integration)
• Breaking change (fix/feature causing existing functionality to break)
• Code quality improvements to existing code or addition of tests

Additional information

• This PR fixes or closes issue: fixes #
• This PR is related to issue:
• Link to documentation pull request:

Checklist

• The code change is tested and works locally.
• Local tests pass. Your PR cannot be merged unless tests pass
• There is no commented out code in this PR.
• I have followed the development checklist
• The code has been formatted using Black (black --fast homeassistant tests)
• Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

• Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

• The manifest file has all fields filled out correctly.
  Updated and included derived files by running: python3 -m script.hassfest.
• New or updated dependencies have been added to requirements_all.txt.
  Updated by running python3 -m script.gen_requirements_all.
• For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
• Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

• No score or internal
• 🥈 Silver
• 🥇 Gold
• 🏆 Platinum

To help with the load of incoming pull requests:

• I have reviewed two other open pull requests in this repository.

[probot-home-assistant]

Hey there @home-assistant/core, mind taking a look at this pull request as it has been labeled with an integration (logbook) you are listed as a code owner for? Thanks!
_{^{(message by CodeOwnersMention)}}

[emontnemery]

The SQL query generated by _get_events already filters out all entities from the sensor and proximity domains which have a unit of measurement; hence the grouping here filled no purpose and is removed in this PR.

[emontnemery]

As an alternative to changing this, we could set a unit on the state.

[emontnemery]

As an alternative to changing this, we could set a unit on the state.

[emontnemery]

This was not a good test, humanify should not be tested on its own.
In a follow-up we should rename humanify to _humanify to make it clear it's not to be used on its own.

[bdraco]

I’ll profile and test this tomorrow when I get back home

[bdraco]

I did a profile with

@@ -308,7 +309,11 @@ class LogbookView(HomeAssistantView):
 
         def json_events() -> web.Response:
             """Fetch events and generate JSON."""
-            return self.json(
+            import cProfile
+
+            pr = cProfile.Profile()
+            pr.enable()
+            ret = self.json(
                 _get_events(
                     hass,
                     start_day,
@@ -320,6 +325,10 @@ class LogbookView(HomeAssistantView):
                     context_id,
                 )
             )
+            pr.disable()
+            pr.create_stats()
+            pr.dump_stats(f"logbook.{time.time()}.cprof")
+            return ret
 
         return cast(
             web.Response, await get_instance(hass).async_add_executor_job(json_events)

Using some test data I had with MANY sensor updates.

The new code looks good. The bottleneck for non-sqlalchemy code was split_entity_id.

[bdraco]

Something like this removes the split_entity_id bdraco@b039933

[emontnemery]

FTR: #71345 adds an LRU-cache to split_entity_id and is more efficient than #71331 (comment)