Enable compression when sending json to client#11165
Conversation
Make server compress json content when transmitting to client. Json is quite verbose and compresses well. A real world example is history_graph requested data for in my case 4 temperature sensors updating every half a second for a graph over 10 days lead to 6MB json which compressed to 200KB using deflate compression.
|
Note. This was a quick proof of concept from github gui, that was tested locally with just web tests. |
|
This needs on-device testing. Specifically how cpu-time compares between transmitting uncompressed data vs compressing+transmitting compressed data. |
| msg = json.dumps( | ||
| result, sort_keys=True, cls=rem.JSONEncoder).encode('UTF-8') | ||
| return web.Response( | ||
| j = web.Response( |
There was a problem hiding this comment.
Preference:
Please name this response instead of j.
There was a problem hiding this comment.
@elupus FYI: you renamed it to request instead of response, as requested.
|
Even the most underpowered CPUs should be able to handle deflate without breaking a sweat. I don't have any benchmarks on hand, but I did some testing on this a year or 2 ago when sending big JSON payloads from a raspberry pi and even over a local connection the gzipped content loaded significantly faster than the ungzipped content. |
|
Damn. Will fix.
…On Jan 3, 2018 01:05, "Matt N." ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In homeassistant/components/http/__init__.py
<#11165 (comment)>
:
> @@ -362,9 +362,11 @@ def json(self, result, status_code=200, headers=None):
"""Return a JSON response."""
msg = json.dumps(
result, sort_keys=True, cls=rem.JSONEncoder).encode('UTF-8')
- return web.Response(
+ j = web.Response(
@elupus <https://github.com/elupus> FYI: you renamed it to request
instead of response, as requested.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#11165 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAdeoIcTWcQwxL7SqeRZD7EFZ5ThdHmmks5tGsO2gaJpZM4REVAj>
.
|
|
Kind ping :) |
|
I think this is fine, a client which cannot handle gzip compression should not send a header indicating it wants to accept such content. If that's the case, |
|
Benchmarks would be cool, but I don't think it's worth blocking a merge on it. We know that the JSON we're serving will dramatically benefit even from just gzip compression, and gzip is really lightweight to decompress. It's up to clients to signal their preferred compression formats. |
|
My concern was about server CPU, not client CPU. |
|
I think that networkIO is worse than a couple of CPU-cycles, but it should be benchmarked. AND this should be in mind https://en.wikipedia.org/wiki/BREACH |
|
BREACH isn't an issue as we aren't compressing data from an attacker. Basically, if an attacker could modify data being sent to craft a chosen-plaintext attack, they have already owned the system. And about the benchmarks, would it be easier to merge this behind a flag, then after it is out in the wild ask for benchmarks from various users to determine if it should be enabled by default? That will hit more users, use cases, and hardware than we would ever be able to test before releasing it, and it would give any users impacted the ability to undo this change quickly in their config. |
|
@andrey-git Ah, that does make more sense. I'd still be inclined to turn this on as a more sane default, and ask for benchmarks to justify disabling it for specific use cases though. |
|
Let's turn it on and see how it goes. |
* Enable compression when sending json to client Make server compress json content when transmitting to client. Json is quite verbose and compresses well. A real world example is history_graph requested data for in my case 4 temperature sensors updating every half a second for a graph over 10 days lead to 6MB json which compressed to 200KB using deflate compression. * Rename variable to request * Name the variable response instead of request
10 participants
Make server compress json content when transmitting to client. Json is quite verbose and compresses well.
A real world example is history_graph requested data for in my case 4 temperature sensors updating every half a second for a graph over 10 days lead to 6MB json which compressed to 200KB using deflate compression.