Store JSON in database in compact format

[frenck]

Proposed change

A great observation from @slovdahl reported on Discord:

MariaDB [home_assistant]> SELECT event_type, event_data FROM events WHERE event_type = 'knx_event' LIMIT 1;
+------------+---------------------------------------------------------------------------------------------------------------------------+
| event_type | event_data                                                                                                                |
+------------+---------------------------------------------------------------------------------------------------------------------------+
| knx_event  | {"data": [7, 78], "destination": "1/3/0", "direction": "Incoming", "source": "1.1.34", "telegramtype": "GroupValueWrite"} |
+------------+---------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.001 sec)

This PR compacts the JSON a bit, thanks @slovdahl ❤️

Result:

Type of change

• Dependency upgrade
• Bugfix (non-breaking change which fixes an issue)
• New integration (thank you!)
• New feature (which adds functionality to an existing integration)
• Breaking change (fix/feature causing existing functionality to break)
• Code quality improvements to existing code or addition of tests

Additional information

• This PR fixes or closes issue: fixes #
• This PR is related to issue:
• Link to documentation pull request:

Checklist

• The code change is tested and works locally.
• Local tests pass. Your PR cannot be merged unless tests pass
• There is no commented out code in this PR.
• I have followed the development checklist
• The code has been formatted using Black (black --fast homeassistant tests)
• Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

• Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

• The manifest file has all fields filled out correctly.
  Updated and included derived files by running: python3 -m script.hassfest.
• New or updated dependencies have been added to requirements_all.txt.
  Updated by running python3 -m script.gen_requirements_all.
• Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

• No score or internal
• 🥈 Silver
• 🥇 Gold
• 🏆 Platinum

To help with the load of incoming pull requests:

• I have reviewed two other open pull requests in this repository.

[bdraco]

Nice.

[raman325]

this will only impact new events and states that get recorded right? Is it worth doing a migration as part of this? Perhaps not since the old data should age out over time

[bdraco]

this will only impact new events and states that get recorded right? Is it worth doing a migration as part of this? Perhaps not since the old data should age out over time

I think its best not to re-write history if we can avoid it since you are only saving 3-7% and having to pay a large I/O & cpu cost. Probably best to let it expire over time

[bdraco]

This will break some of the logbook optimizations (we jump through some hoops to avoid de-serializing JSON since we could end up with 200-millions of json deserialize calls per logbook api call) but should be trivial to fix.

diff --git a/homeassistant/components/logbook/__init__.py b/homeassistant/components/logbook/__init__.py
index 0a36790746..e3bb8fcd77 100644
--- a/homeassistant/components/logbook/__init__.py
+++ b/homeassistant/components/logbook/__init__.py
@@ -48,10 +48,10 @@ from homeassistant.helpers.integration_platform import (
 from homeassistant.loader import bind_hass
 import homeassistant.util.dt as dt_util
 
-ENTITY_ID_JSON_TEMPLATE = '"entity_id": "{}"'
-ENTITY_ID_JSON_EXTRACT = re.compile('"entity_id": "([^"]+)"')
-DOMAIN_JSON_EXTRACT = re.compile('"domain": "([^"]+)"')
-ICON_JSON_EXTRACT = re.compile('"icon": "([^"]+)"')
+ENTITY_ID_JSON_TEMPLATE = '"entity_id": ?"{}"'
+ENTITY_ID_JSON_EXTRACT = re.compile('"entity_id": ?"([^"]+)"')
+DOMAIN_JSON_EXTRACT = re.compile('"domain": ?"([^"]+)"')
+ICON_JSON_EXTRACT = re.compile('"icon": ?"([^"]+)"')
 
 ATTR_MESSAGE = "message"
 

[frenck]

WUT?! haha
omg

[bdraco]

WUT?! haha
omg

Yeah..... I'd rather we could normalize more of data in the database instead of JSON, but thats a whole rabbit hole of doom.