Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New: Add rule to check if resources are served compressed #731

Closed
wants to merge 3 commits into from
Closed

New: Add rule to check if resources are served compressed #731

wants to merge 3 commits into from

Conversation

alrra
Copy link
Contributor

@alrra alrra commented Jan 3, 2018

Pull request checklist

Make sure you:

For non-trivial changes, please make sure you also:

  • Added/Updated related documentation.
  • Added/Updated related tests.

Short description of the change(s)

Fix #1
Fix #12

@@ -645,6 +645,7 @@ const rule: IRuleBuilder = {
category: Category.performance,
description: 'Require resources to be served compressed'
},
ignoredConnectors: ['chrome'], // Headless Chrome on Travis CI fails miserably. :(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hopefully this is temporary, all the test pass even for the chrome connector on my machine. :(

@alrra
Copy link
Contributor Author

alrra commented Jan 4, 2018

Note: I've merged some of the fixes from this PR into master.

Copy link
Member

@molant molant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just reviewed the documentation. Need to take a look at the code but just in case you want to start doing changes or disagree with me


`gzip` is the most used encoding nowadays as it strikes a good
balance between compression ratio (as high as 70-90% especially
for larger files) and encoding time, and is supported pretty much
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have a link for the 70-90% claim?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

`gzip` is the most used encoding nowadays as it strikes a good
balance between compression ratio (as high as 70-90% especially
for larger files) and encoding time, and is supported pretty much
eveywere.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

everywhere

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

eveywere.

Better savings can be achieved using [`Zopfli`][zopfli] which
can reduce the size on average with [3–8% more then `gzip`][[zopfli
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on average 3-8% more than gzip

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, there is another issue here: double [[.

can reduce the size on average with [3–8% more then `gzip`][[zopfli
blog post]. Since `Zopfli` output (for the `gzip` option) is valid
`gzip` content, `Zopfli` works eveywere `gzip` works. The only
downsize is that encoding takes way more time than with `gzip`,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

encoding takes way more time

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

`gzip` content, `Zopfli` works eveywere `gzip` works. The only
downsize is that encoding takes way more time than with `gzip`,
thus, making `Zopfli` more suitable for static content (i.e. encoding
resources with is as part of build script, not on the fly).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with is as part of a build script

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.


When resources are served compressed, they should be served with
the `Vary` header containing the `Accept-Encoding` value (or with
something such as `Cache-Control: private` that prevents caching
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that prevents caching in proxy caches and such altogether

The final user can cache the response so we are not preventing caching altogether:

Indicates that all or part of the response message is intended for a single user and MUST NOT be cached by a shared cache, such as a proxy server.

source

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

* Resources should be served compressed only when requested as such,
appropriately encoded, and without relying on user agent sniffing.

What the `Accept-Encoding` request header specified should be
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What The Accept-Encoding

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

appropriately encoded, and without relying on user agent sniffing.

What the `Accept-Encoding` request header specified should be
respected. Sending a the content encoded with a different encoding
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sending a the content

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.


What the `Accept-Encoding` request header specified should be
respected. Sending a the content encoded with a different encoding
then one of the ones accepted can lead to problems.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

than one

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.


## Can the rule be configured?

You can overwrite the defaults by specifying what type of compression
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

override

@@ -0,0 +1,672 @@
/**
* @fileoverview Check is resources are served compressed.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checks if resources

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

*
* 2. Zopfli
*
* On thing that Zopfli does is that it sets FLG and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing


/*
* TODO: Remove the following once connectors
* support Brotli compression.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you open an issue for this?
Basically what we need is requester to auto decompress Brotli.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you open an issue for this?

Done.

};

const generateCompressionMessage = (encoding: string, notRequired?: boolean, suffix?: string) => {
return `Should${notRequired ? ' not' : ''} be served compressed${encoding ? ` with ${encoding}` : ''}${suffix ? ` ${suffix}` : ''}.`;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should make the message clearer.
Right now if a resources is not compressed the user is going to get 2 messages:

Should be served compressed with Zopfli
Should be served compressed with Brotli

which can be confusing. A way could be to say the protocol as you proposed the other day:

Should be served compressed with Zopfli for HTTP
Should be served compressed with Brotli for HTTPS

Copy link
Contributor Author

@alrra alrra Jan 4, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

(Note: Only changed the message for Brotli since Zopfli applies to both)

@alrra
Copy link
Contributor Author

alrra commented Jan 4, 2018

@molant I addressed all your feedback!

*/

if (isTextMediaType(mediaType) ||
['image/x-icon',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should have these kind of arrays of the method as a constant.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

alrra and others added 3 commits January 8, 2018 11:46
Fix several issues found when analyzing real websites with edge cases:

* 204 status code with a body.
* Missing headers.
* Better error reporting.
@alrra alrra closed this in b1c173c Jan 8, 2018
@alrra alrra deleted the compression branch January 8, 2018 22:47
@molant
Copy link
Member

molant commented Jan 8, 2018

🎉

alrra pushed a commit that referenced this pull request Jan 9, 2018
Fix several issues found when analyzing real websites with edge cases:

* 204 status code with a body.
* Missing headers.
* Better error reporting.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Close #731
alrra pushed a commit that referenced this pull request Jan 9, 2018
Fix several issues found when analyzing real websites with edge cases:

* 204 status code with a body.
* Missing headers.
* Better error reporting.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Close #731
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants