Implement time tracking in templates

[bdraco]

Completes home-assistant/architecture#206

• Retest adjusted frontend change
• Wait for Improve template ratelimit performance #41741
• Frontend Add support for displaying time listeners frontend#7220

Breaking change

It is no longer necessary to reference sensor.time, sensor.date, or manually update template entities that do not listen for state changes when now() or utcnow() is present in the template.

The template will automatically be updated when:

• A referenced entity changes state.
• At the start of each minute when now() or utcnow() is present in the template.

Type of change

• Dependency upgrade
• Bugfix (non-breaking change which fixes an issue)
• New integration (thank you!)
• New feature (which adds functionality to an existing integration)
• Breaking change (fix/feature causing existing functionality to break)
• Code quality improvements to existing code or addition of tests

Example entry for configuration.yaml:

# Example configuration.yaml

Additional information

• This PR fixes or closes issue: fixes Binary sensor using now() never updates #40639
• This PR is related to issue:
• Link to documentation pull request: Implement track_time_pattern in templates home-assistant.io#14854 (outdated)

Checklist

• The code change is tested and works locally.
• Local tests pass. Your PR cannot be merged unless tests pass
• There is no commented out code in this PR.
• I have followed the development checklist
• The code has been formatted using Black (black --fast homeassistant tests)
• Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

• Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

• The manifest file has all fields filled out correctly.
  Updated and included derived files by running: python3 -m script.hassfest.
• New or updated dependencies have been added to requirements_all.txt.
  Updated by running python3 -m script.gen_requirements_all.
• Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

• No score or internal
• 🥈 Silver
• 🥇 Gold
• 🏆 Platinum

To help with the load of incoming pull requests:

• I have reviewed two other open pull requests in this repository.

[probot-home-assistant]

Hey there @home-assistant/core, mind taking a look at this pull request as its been labeled with an integration (homeassistant) you are listed as a codeowner for? Thanks!
_{^{(message by CodeOwnersMention)}}

[bdraco]

Looks like we need tests for

• a template group that has some using state tracking and some using time pattern tracking
• a time pattern that changes
• returning the time_patterns in the listeners

[frenck]

This is functionality that should not be in the template engine. I feel like we are going down a rabbit hole we should not.

I think we should put a halt to this design we are heading towards, it makes things unneeded complex, for reasons I fail to see at this point.

[bdraco]

This is functionality that should not be in the template engine. I feel like we are going down a rabbit hole we should not.

I think we should put a halt to this design we are heading towards, it makes things unneeded complex, for reasons I fail to see at this point.

Without a way for users to take control of the update frequency, we leave cases like this #40621 (comment) without a solution.

Admittedly, it appears to be a rare case, and this does seem to be heading for feature overload so, I'll put this on ice until we can show that we have a larger set of affected users.

[bdraco]

Offline discussion: do away with the configurability and just call_later(60) after each update (cancelled on the next update).

That should have minimal performance impact and also avoid a CPU spike at xx:yy:00.

[amelchio]

I think it is more consistent (and thus easier to explain/understand) to always delay 60 seconds after the last update. I have tried to adjust to that and it actually simplified things: amelchio@e08a598 ... but you said this would not work so maybe I missed something?

[bdraco]

I think it is more consistent (and thus easier to explain/understand) to always delay 60 seconds after the last update. I have tried to adjust to that and it actually simplified things: amelchio@e08a598 ... but you said this would not work so maybe I missed something?

What you have will work.

There are two caveats we would need to be ok with:

• We don't cancel the time based refresh if there is state changed event (not sure if that is an issue).
  - If now() is no longer in scope in the template, the time based refreshes will continue.

[amelchio]

I don't think those caveats apply. The running async_call_later() is cancelled in both of these cases and a new one is installed iff now() is still in scope.

[bdraco]

I don't think those caveats apply. The running async_call_later() is cancelled in both of these cases and a new one is installed iff now() is still in scope.

I've stared at this and I can't see where the async_call_later is canceled? What am I missing?

[amelchio]

It's self._time_listeners.pop(template)() from your commit, so not a part of my diff.

[bdraco]

I misunderstood. So the intent is to always update every 60 seconds even if there is a state changed event in the time window (taking out the _skip_next_time_update)?

[amelchio]

No, my intent is to force an update if the template is idle for 60 seconds.

A state change at time 17 will schedule an update for time 77. Another state change at time 33 will cancel the first timer and instead schedule a new update for time 93 (always 60 seconds later).

With the same state change at time 33, your implementation might skip a time update at time 77 and then refresh at time 137 (104 seconds after the last state change).

[bdraco]

No, my intent is to force an update if the template is idle for 60 seconds.

I checked this out on my desktop instead of trying to look at it on github and parse the changes on top of changes in my head and I'll all on board now.

It looks good to me. 👍 Do you want to push it on top of the existing branch in this PR?

[amelchio]

Pushed it now. I am obviously cool with this change but I'll refrain from marking as approved since I think we should get feedback from at least @frenck as well so we can avoid any more emergency brakes :)

[bdraco]

• I will adjust the frontend PR
• retest later today

[amelchio]

Using sensor.time to align the update with the change of a minute will race with the automatic update. Thus, some minutes will have two updates right after each other. I wonder if we should change the delay to 61 or 65 seconds to avoid the redundant refresh in this (probably common) case? 🤔

[bdraco]

Using sensor.time to align the update with the change of a minute will race with the automatic update. Thus, some minutes will have two updates right after each other. I wonder if we should change the delay to 61 or 65 seconds to avoid the redundant refresh in this (probably common) case? 🤔

Good call. Since we have microsecond precision, how about 60.45? We could probably even go lower.

[amelchio]

Whatever you think is fine, I'm in ...

[balloob]

🎉

[amelchio]

Grammar question to Breaking change: ... when an referenced entity changes state or at the start of each minute ...

Should that "or" be "and"? As it is, I understand that one will not get updates from state changes if now() is used. But it could be my reading comprehension that is the problem.

[bdraco]

It is a bit unwieldy. I'll rework the phrasing.

[bdraco]

I adjusted the phrasing. Hopefully that makes more sense.