Use pre-gzipped content files? #1584

AnderssonPeter · 2016-06-21T09:21:23Z

is there any support for pre-gzipped files if there where we could compress css, html and javascript files with zopfli to make then as small as possible.

the files could be saved as
[filename]_[hash].gzip or some like that.

If there is support how would one go about to enable it?
If there isn't do you have any tips on how to create it?

The text was updated successfully, but these errors were encountered:

neyromant · 2016-06-21T14:21:34Z

I use gulp-gzip package for make .gz files.
And small middleware for handle requests for static files:

public class CompressedStaticFileMiddleware
    {
        private IHostingEnvironment _hostingEnv;
        private StaticFileMiddleware _base;
        public CompressedStaticFileMiddleware(RequestDelegate next, IHostingEnvironment hostingEnv, IOptions<StaticFileOptions> options, ILoggerFactory loggerFactory) 
        {
            _hostingEnv = hostingEnv;
            var contentTypeProvider = options.Value.ContentTypeProvider ?? new FileExtensionContentTypeProvider();
_base = new StaticFileMiddleware(next, hostingEnv, new StaticFileOptions()
            {
                ContentTypeProvider = contentTypeProvider,
                FileProvider = options.Value.FileProvider ?? hostingEnv.WebRootFileProvider,
                OnPrepareResponse = ctx =>
                {
                    const string ext = ".gz";
                    if (ctx.File.Name.EndsWith(ext))
                    {
                        string contentType = null;
                        if(contentTypeProvider.TryGetContentType(ctx.File.PhysicalPath.Remove(ctx.File.PhysicalPath.Length - ext.Length, ext.Length), out contentType))
                            ctx.Context.Response.ContentType = contentType;
                        ctx.Context.Response.Headers.Add("Content-Encoding", new[] { "gzip" });
                    }
                }
            }, loggerFactory);
        }

        public Task Invoke(HttpContext context)
        {
            if(context.Request.Path.HasValue)
            {
                string acceptEncoding = context.Request.Headers["Accept-Encoding"];
                if (
                    !string.IsNullOrEmpty(acceptEncoding) && 
                    (acceptEncoding.Contains("gzip") && 
                    System.IO.File.Exists(_hostingEnv.MapPath(context.Request.Path.Value.StartsWith("/") ? context.Request.Path.Value.Remove(0, 1) : context.Request.Path.Value) + ".gz"))
                )
                {
                    context.Request.Path = new PathString(context.Request.Path.Value + ".gz");
                    return _base.Invoke(context);
                }
            }
            return _base.Invoke(context);
        }
    }

AnderssonPeter · 2016-07-04T18:45:56Z

the constructor for StaticFileMiddleware takes a IOptions<StaticFileOptions> not a StaticFileOptions so i cant compile this?
And IHostingEnvironment does not seem to contain MapPath?

neyromant · 2016-07-08T09:04:16Z

@AnderssonPeter, yes, it was code for RC1.
For RC2 , you can try this:

public class CompressedStaticFileMiddleware
    {
        private IHostingEnvironment _hostingEnv;
        private StaticFileMiddleware _base;
        public CompressedStaticFileMiddleware(RequestDelegate next, IHostingEnvironment hostingEnv, IOptions<StaticFileOptions> options, ILoggerFactory loggerFactory)
        {
            _hostingEnv = hostingEnv;
            var contentTypeProvider = options.Value.ContentTypeProvider ?? new FileExtensionContentTypeProvider();
            options.Value.ContentTypeProvider = contentTypeProvider;
            options.Value.FileProvider = options.Value.FileProvider ?? hostingEnv.WebRootFileProvider;
            options.Value.OnPrepareResponse = ctx =>
            {
                const string ext = ".gz";
                if (ctx.File.Name.EndsWith(ext))
                {
                    string contentType = null;
                    if (contentTypeProvider.TryGetContentType(ctx.File.PhysicalPath.Remove(ctx.File.PhysicalPath.Length - ext.Length, ext.Length), out contentType))
                        ctx.Context.Response.ContentType = contentType;
                    ctx.Context.Response.Headers.Add("Content-Encoding", new[] { "gzip" });
                }
            };

            _base = new StaticFileMiddleware(next, hostingEnv, options, loggerFactory);
        }

        public Task Invoke(HttpContext context)
        {
            if (context.Request.Path.HasValue)
            {
                string acceptEncoding = context.Request.Headers["Accept-Encoding"];
                if (
                    !string.IsNullOrEmpty(acceptEncoding) &&
                    (
                        acceptEncoding.Contains("gzip") &&
                        System.IO.File.Exists(
                            System.IO.Path.Combine(
                                _hostingEnv.WebRootPath, context.Request.Path.Value.StartsWith("/") 
                                ? context.Request.Path.Value.Remove(0, 1) 
                                : context.Request.Path.Value
                            ) + ".gz"
                        )
                    )
                )
                {
                    context.Request.Path = new PathString(context.Request.Path.Value + ".gz");
                    return _base.Invoke(context);
                }
            }
            return _base.Invoke(context);
        }
    }

AnderssonPeter · 2016-07-08T09:12:16Z

Ill try it when i get home, but im using RTM not RC2 ill get back to you if this works in the RTM.

Thanks @neyromant !!

neyromant · 2016-07-08T09:17:44Z

You are welcome.
I think that in the RTM will not have problems, because these classes have not changed.

AnderssonPeter · 2016-07-11T19:15:40Z

Im confused does asp.net core gzip content on the fly?
If i don`t add the "CompressedStaticFileMiddleware" i still get GZipped content, and it does not matter if the .gz files are there or not so i guess they aren't used.

Also it seems like the Invoke method is only called once so it never has a chance to replace the file paths with .gz

neyromant · 2016-07-12T09:26:59Z

I found small mistake in code:

System.IO.File.Exists(
System.IO.Path.Combine(
_hostingEnv.WebRootPath, context.Request.Path.Value.StartsWith("/")
? context.Request.Path.Value.Remove(0, 1)
: context.Request.Path.Value
) + ".gz"
)

AnderssonPeter · 2016-07-12T10:14:30Z

Well some of my confusion has been lifted, the default project from visual studio runs IIS Express as a reverse proxy infront of Kestrel and IIS Express does automatic GZip compression on content.

So that was the reason for files being gzipped even when there was no .gz file.

Ill try your updated code when i get home and see if it works better!

Thanks @neyromant

neyromant · 2016-07-12T10:15:51Z

@AnderssonPeter, You are welcome.

AnderssonPeter · 2016-07-13T18:51:56Z

I made some improvements to support brotli compression, and if the browser supports both it picks the smallest file.

Now i haven't been able to test brotli for some reason its only used on https (Chrome and FireFox) and im to new to asp.net core and Kestrel to configure local https!

public class CompressedStaticFileMiddleware
{
    private static Dictionary<string, string> compressionTypes = new Dictionary<string, string>()
    {{"gzip", ".gz" },
        {"br", ".br" }};

    private IHostingEnvironment _hostingEnv;
    private StaticFileMiddleware _base;
    public CompressedStaticFileMiddleware(RequestDelegate next, IHostingEnvironment hostingEnv, IOptions<StaticFileOptions> options, ILoggerFactory loggerFactory)
    {
        _hostingEnv = hostingEnv;
        var contentTypeProvider = options.Value.ContentTypeProvider ?? new FileExtensionContentTypeProvider();
        options.Value.ContentTypeProvider = contentTypeProvider;
        options.Value.FileProvider = options.Value.FileProvider ?? hostingEnv.WebRootFileProvider;
        options.Value.OnPrepareResponse = ctx =>
        {
            foreach(var compressionType in compressionTypes.Keys)
            {
                var fileExtension = compressionTypes[compressionType];
                if (ctx.File.Name.EndsWith(fileExtension, StringComparison.OrdinalIgnoreCase))
                {
                    string contentType = null;
                    if (contentTypeProvider.TryGetContentType(ctx.File.PhysicalPath.Remove(ctx.File.PhysicalPath.Length - fileExtension.Length, fileExtension.Length), out contentType))
                        ctx.Context.Response.ContentType = contentType;
                    ctx.Context.Response.Headers.Add("Content-Encoding", new[] { compressionType });
                }
            }
        };

        _base = new StaticFileMiddleware(next, hostingEnv, options, loggerFactory);
    }

    public Task Invoke(HttpContext context)
    {
        if (context.Request.Path.HasValue)
        {
            string acceptEncoding = context.Request.Headers["Accept-Encoding"];
            FileInfo matchedFile = null;
            string[] browserSupportedCompressionTypes = context.Request.Headers["Accept-Encoding"].ToString().Split(new[] {',', ' '}, StringSplitOptions.RemoveEmptyEntries);
            foreach (var compressionType in compressionTypes.Keys)
            {
                if (browserSupportedCompressionTypes.Contains(compressionType, StringComparer.OrdinalIgnoreCase))
                {
                    var fileExtension = compressionTypes[compressionType];
                    var filePath = System.IO.Path.Combine(
                            _hostingEnv.WebRootPath, context.Request.Path.Value.StartsWith("/")
                            ? context.Request.Path.Value.Remove(0, 1)
                            : context.Request.Path.Value
                        ) + fileExtension;
                    var file = new FileInfo(filePath);
                    if (file.Exists)
                    {
                        if (matchedFile == null)
                            matchedFile = file;
                        else if (matchedFile.Length > file.Length)
                            matchedFile = file;
                    }
                }
            }
            if (matchedFile != null)
            {
                context.Request.Path = new PathString(context.Request.Path.Value + matchedFile.Extension);
                return _base.Invoke(context);
            }
        }
        return _base.Invoke(context);
    }
}

gulp-brotli can be used to compress the files with brotli it seems to beat gzip even when gzipped with zopfli.

AnderssonPeter · 2016-07-15T17:40:32Z

@neyromant is it ok if i create a github page with this code? and if any license you would prefer?
And if i figure out how ill create a nuget package.

neyromant · 2016-07-15T18:01:33Z

@AnderssonPeter, Of course, you can create a page and nuget package.
I will be happy if you are somewhere in the comments write my nickname :)

AnderssonPeter · 2016-07-15T22:59:14Z

Github page is up for it: https://github.com/AnderssonPeter/CompressedStaticFiles haven't managed to add nuget yet tho.

AnderssonPeter · 2016-07-20T19:15:20Z

And its up on nuget https://www.nuget.org/packages/CompressedStaticFiles/

Ponant · 2016-08-19T12:40:18Z

@AnderssonPeter
I reported a seemingly small issue on Microsoft Azure Forum as we have a production website that does not send compressed svg files. A moderator there pointed to this thread and I am trying to implement your package but I fail to make it work.

I installed your package on our AP.NET Core RTM 1.0.0
added app.UseCompressedStaticFiles(); before app.UseStaticFiles();
Removing .UseIISIntegration() in Program.cs makes the website unreachable both on localhost and on azure, so I left it in...
Uploading to Azure and checking out in the browser that the SVG files of 6kB and 13kB are not compressed.

Any clue?
Thanks

AnderssonPeter · 2016-08-19T16:29:47Z

i don't know any thing about Azure but i guess it uses IIS?
if that's true then im sorry the package wont work it only works when you use Kestrel.

Ponant · 2016-08-19T17:33:48Z

I am using IIS indeed, thanks for the reply, I also realized this won't work.

PeterDefendo · 2016-09-02T12:01:10Z

@Ponant i merged AnderssonPeter/CompressedStaticFiles#4 sometime ago so it might work now but im not 100% sure.

Ponant · 2016-09-04T15:09:16Z

Thanks @PeterDefendo , but where are the instructions that I try out your merge? Is it here-->
https://github.com/AnderssonPeter/CompressedStaticFiles ?

AnderssonPeter · 2016-09-05T06:21:29Z

@Ponant sorry i happend to write with my work github account :P sorry about the confusion, its in the official nuget package

ilya-git · 2016-11-13T19:55:16Z

Thanks for the nice plugin, I have advertised it a bit here :) But I do really hope MS includes in standard package in this issue. Maybe you can help them by making a pull request :)

aspnet-hello · 2017-12-30T23:55:44Z

This issue is being closed because it has not been updated in 3 months.

We apologize if this causes any inconvenience. We ask that if you are still encountering this issue, please log a new issue with updated information and we will investigate.

tkrotoff · 2017-12-31T03:30:45Z

@aspnet-hello fuck stupid bots

neyromant mentioned this issue Jul 30, 2016

Support serving of pre-GZIP encoded files aspnet/StaticFiles#7

Closed

aspnet-hello closed this as completed Dec 30, 2017

aspnet-hello mentioned this issue Jan 1, 2018

Support serving of pre-GZIP encoded files #2458

Closed

speige mentioned this issue Feb 17, 2018

Response Compression allows re-compressing of already compressed responses aspnet/BasicMiddleware#305

Closed

natemcmaster pushed a commit that referenced this issue Nov 14, 2018

Remove Kestrel.Core dependency from Transport.Libuv (#1584).

6a66323

ghost locked as resolved and limited conversation to collaborators Dec 4, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use pre-gzipped content files? #1584

Use pre-gzipped content files? #1584

AnderssonPeter commented Jun 21, 2016 •

edited

Loading

neyromant commented Jun 21, 2016 •

edited

Loading

AnderssonPeter commented Jul 4, 2016 •

edited

Loading

neyromant commented Jul 8, 2016 •

edited

Loading

AnderssonPeter commented Jul 8, 2016

neyromant commented Jul 8, 2016

AnderssonPeter commented Jul 11, 2016

neyromant commented Jul 12, 2016

AnderssonPeter commented Jul 12, 2016

neyromant commented Jul 12, 2016

AnderssonPeter commented Jul 13, 2016 •

edited

Loading

AnderssonPeter commented Jul 15, 2016

neyromant commented Jul 15, 2016

AnderssonPeter commented Jul 15, 2016

AnderssonPeter commented Jul 20, 2016

Ponant commented Aug 19, 2016

AnderssonPeter commented Aug 19, 2016

Ponant commented Aug 19, 2016

PeterDefendo commented Sep 2, 2016

Ponant commented Sep 4, 2016

AnderssonPeter commented Sep 5, 2016

ilya-git commented Nov 13, 2016 •

edited

Loading

aspnet-hello commented Dec 30, 2017

tkrotoff commented Dec 31, 2017

Use pre-gzipped content files? #1584

Use pre-gzipped content files? #1584

Comments

AnderssonPeter commented Jun 21, 2016 • edited Loading

neyromant commented Jun 21, 2016 • edited Loading

AnderssonPeter commented Jul 4, 2016 • edited Loading

neyromant commented Jul 8, 2016 • edited Loading

AnderssonPeter commented Jul 8, 2016

neyromant commented Jul 8, 2016

AnderssonPeter commented Jul 11, 2016

neyromant commented Jul 12, 2016

AnderssonPeter commented Jul 12, 2016

neyromant commented Jul 12, 2016

AnderssonPeter commented Jul 13, 2016 • edited Loading

AnderssonPeter commented Jul 15, 2016

neyromant commented Jul 15, 2016

AnderssonPeter commented Jul 15, 2016

AnderssonPeter commented Jul 20, 2016

Ponant commented Aug 19, 2016

AnderssonPeter commented Aug 19, 2016

Ponant commented Aug 19, 2016

PeterDefendo commented Sep 2, 2016

Ponant commented Sep 4, 2016

AnderssonPeter commented Sep 5, 2016

ilya-git commented Nov 13, 2016 • edited Loading

aspnet-hello commented Dec 30, 2017

tkrotoff commented Dec 31, 2017

AnderssonPeter commented Jun 21, 2016 •

edited

Loading

neyromant commented Jun 21, 2016 •

edited

Loading

AnderssonPeter commented Jul 4, 2016 •

edited

Loading

neyromant commented Jul 8, 2016 •

edited

Loading

AnderssonPeter commented Jul 13, 2016 •

edited

Loading

ilya-git commented Nov 13, 2016 •

edited

Loading