Prepare for recorder purge to be active by default

[amelchio]

Breaking change note

From version 0.64, Home Assistant will by default purge recorded state history that is older than 10 days. If you want to keep your recorded data for longer than that, you must configure the number of days to retain:

recorder:
  purge_keep_days: 30

If you want to keep the previous default of never deleting history, use this configuration:

recorder:
  purge_interval: 0

Description:

As discussed elsewhere, this sets the stage for changing the recorder purge_keep_days default from None to 10.

Because purging is surprisingly cheap and vacuuming is surprisingly expensive, the purge_interval now defaults to 1 and vacuuming is only done with the first purge after startup. This is to lower the performance impact on new adopters.

The vacuum will be fully removed one release after purge is active by default.

Pull request in home-assistant.github.io with documentation (if applicable): home-assistant/home-assistant.io#4534

Example entry for configuration.yaml (if applicable):

recorder:
  purge_keep_days: 7

Checklist:

• The code change is tested and works locally.

If user exposed functionality or configuration variables are added/changed:

• Documentation added/updated in home-assistant.github.io

If the code does not interact with devices:

• Local tests with tox run successfully.
• Tests have been added to verify that the new code works.

[pvizeli]

Maybe we can add vacuum to service purge call as option?

[OttoWinter]

I think it would be good if the warning would also tell you how to disable the new default behavior - i.e. how to disable purging altogether.

For example, I use Home Assistant's recorder to store all sensor data so that I can analyze and optimize my systems later. I know this is probably terribly inefficient and my databases are getting humongous, but it's just much simpler than having to keep track of all data yourself. Anyway, as a user I would like to know how to disable purging before all precious data is lost in 0.64.x.

Of course this then also means that we'd need to settle on a new configuration schema for this. One (terribly ugly, in my view) way of doing this would be to allow 0 (or inf) for purge_keep_days indicating that we should never purge. Another possibility might be to add a new option like purging: enabled or something like that.

Another aspect to this is that we should already include the new schema in 0.63.x. If you're like me and have a cron-job pulling the latest docker image for home assistant every night and want to disable purging, you'd want to have a configuration that a) disables purging for 0.64.x but b) doesn't break your configuration.yaml with 0.63.x.

(I didn't know whether I should open a new issue instead - in that case, please tell me so that I can do it correctly in the future)

[amelchio]

How do you feel about simply setting purge_keep_days: 99999 to effectively keep data forever?

[OttoWinter]

I mean yes that's possible - not that I'd live for 274 years but it's also kind of ugly... I believe at least the warning/docs should be updated.

[pvizeli]

We should allow to set CONF_PURGE_INTERVAL to 0 and disable this process

[amelchio]

I kind of like it to be ugly, to signal that this is a configuration that has some problems. I won't fight a 0 interval, though.

Agreed on the documentation, I will put a blurb at the top of this PR before the release.

[amelchio]

I proposed another PR for the purge_interval: 0 configuration to disable purging and also added a "breaking change" notice with configuration hints at the top of this PR.