-
Notifications
You must be signed in to change notification settings - Fork 10.5k
Add Request Decompression middleware #40279
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 26 commits
Commits
Show all changes
48 commits
Select commit
Hold shift + click to select a range
77923d6
Add Request Decompression middleware
david-acker d888b86
Fix API baselines
david-acker ca2b675
Minor fixes
david-acker d3d074a
Eagerly resolve decompression provider and stream
david-acker 4a4e902
Pass requests to endpoint instead of returning 415
david-acker 8ad1352
Revert request body to original value
david-acker f8d2604
Seal decompression providers
david-acker 12c4519
Use constructor injection for logger
david-acker e000add
Remove unused property
david-acker e2a9963
Seal more classes
david-acker 74622dd
Only create RequestDecompressionBody if a decompresison provider exists
david-acker b4ddfd3
Remove Content-Encoding header if decompressed
david-acker 1d43fef
Fix extensions namespaces
david-acker fa98756
Remove RequestDecompressionBody
david-acker e055698
Fix class name
david-acker 03e31c1
Directly instantiate default providers
david-acker dca2b17
Replace Single with cast
david-acker 202cd2d
Add tests
david-acker 087cf21
Store providers in dictionary
david-acker 8857a63
Fix API baselines
david-acker c453739
Add MaxRequestBodySize to RequestDecompressionOptions
david-acker 2d674b1
Clean up
david-acker 76fe411
Add, clean up tests
david-acker 7b297ca
Add IRequestSizeLimitMetadata
david-acker ef9f9cc
Use updated middleware design
david-acker be883d9
Merge branch 'main' into request-decompression
david-acker d0e3ea5
Address PR feedback
david-acker 1302dac
Address PR feedback
david-acker 7acf307
Update DefaultRequestDecompressionProvider.cs
pranavkm f6304d0
Update DefaultRequestDecompressionProvider.cs
pranavkm 71c3cb4
Update RequestDecompressionMiddleware.cs
pranavkm c96647a
Merge branch 'main' into request-decompression
pranavkm 245ac78
Update RequestDecompressionMiddleware.cs
pranavkm 9a2268e
Use default implementations for BeginRead, EndRead
david-acker e5376e5
Merge branch 'main' into request-decompression
sebastienros 3ffb5f9
Merge branch 'main' into request-decompression
sebastienros 38eb172
Make MaxRequestBodySize nullable
david-acker ea3a334
Implement IRequestSizeLimitMetadata
david-acker c293ade
Add security concern to remarks
david-acker 7beceb9
Add test for invalid compressed data
david-acker cd32124
Set IHttpMaxRequestBodySizeFeature using IRequestSizeLimitMetadata
david-acker b33b594
Merge branch 'main' into request-decompression
david-acker 519310c
Add benchmarks
david-acker e31ef58
Remove unneeded project references
david-acker 18632ef
Pass all write operations to inner stream
david-acker 3f39a4d
Replace async/await with AsTask
david-acker b670356
Fix variable name
david-acker 5cb34b6
Merge branch 'main' into request-decompression
sebastienros File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
15 changes: 15 additions & 0 deletions
15
src/Http/Http.Abstractions/src/Metadata/IRequestSizeLimitMetadata.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| // Licensed to the .NET Foundation under one or more agreements. | ||
| // The .NET Foundation licenses this file to you under the MIT license. | ||
|
|
||
| namespace Microsoft.AspNetCore.Http.Metadata; | ||
|
|
||
| /// <summary> | ||
| /// Interface marking attributes that specify the maximum allowed size of the request body. | ||
| /// </summary> | ||
| public interface IRequestSizeLimitMetadata | ||
| { | ||
| /// <summary> | ||
| /// The maximum allowed size of the current request body in bytes. | ||
| /// </summary> | ||
| long MaxRequestBodySize { get; } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
15 changes: 15 additions & 0 deletions
15
src/Middleware/RequestDecompression/sample/CustomDecompressionProvider.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| // Licensed to the .NET Foundation under one or more agreements. | ||
| // The .NET Foundation licenses this file to you under the MIT license. | ||
|
|
||
| using Microsoft.AspNetCore.RequestDecompression; | ||
|
|
||
| namespace RequestDecompressionSample; | ||
|
|
||
| public class CustomDecompressionProvider : IDecompressionProvider | ||
| { | ||
| public Stream GetDecompressionStream(Stream stream) | ||
| { | ||
| // Create a custom decompression stream wrapper here. | ||
| return stream; | ||
| } | ||
| } |
27 changes: 27 additions & 0 deletions
27
src/Middleware/RequestDecompression/sample/Properties/launchsettings.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| { | ||
| "iisSettings": { | ||
| "windowsAuthentication": false, | ||
| "anonymousAuthentication": true, | ||
| "iisExpress": { | ||
| "applicationUrl": "http://localhost:6164/", | ||
| "sslPort": 0 | ||
| } | ||
| }, | ||
| "profiles": { | ||
| "RequestDecompressionSample": { | ||
| "commandName": "Project", | ||
| "launchBrowser": true, | ||
| "launchUrl": "http://localhost:5000/", | ||
| "environmentVariables": { | ||
| "ASPNETCORE_ENVIRONMENT": "Development" | ||
| } | ||
| }, | ||
| "IIS Express": { | ||
| "commandName": "IISExpress", | ||
| "launchBrowser": true, | ||
| "environmentVariables": { | ||
| "ASPNETCORE_ENVIRONMENT": "Development" | ||
| } | ||
| } | ||
| } | ||
| } |
13 changes: 13 additions & 0 deletions
13
src/Middleware/RequestDecompression/sample/RequestDecompressionSample.csproj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk.Web"> | ||
|
|
||
| <PropertyGroup> | ||
| <TargetFramework>$(DefaultNetCoreTargetFramework)</TargetFramework> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <Reference Include="Microsoft.AspNetCore.RequestDecompression" /> | ||
| <Reference Include="Microsoft.AspNetCore.Server.Kestrel" /> | ||
| <Reference Include="Microsoft.Extensions.Logging.Console" /> | ||
| </ItemGroup> | ||
|
|
||
| </Project> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| // Licensed to the .NET Foundation under one or more agreements. | ||
| // The .NET Foundation licenses this file to you under the MIT license. | ||
|
|
||
| using Microsoft.AspNetCore.RequestDecompression; | ||
|
|
||
| namespace RequestDecompressionSample; | ||
|
|
||
| public class Startup | ||
| { | ||
| public void ConfigureServices(IServiceCollection services) | ||
| { | ||
| services.AddRequestDecompression(options => | ||
| { | ||
| options.Providers.Add<GZipDecompressionProvider>(); | ||
| options.Providers.Add<CustomDecompressionProvider>(); | ||
| }); | ||
| } | ||
|
|
||
| public void Configure(IApplicationBuilder app) | ||
| { | ||
| app.UseRequestDecompression(); | ||
|
|
||
| app.Map("/test", testApp => | ||
| { | ||
| testApp.Run(async context => | ||
| { | ||
| using var reader = new StreamReader(context.Request.Body); | ||
| var decompressedBody = await reader.ReadToEndAsync(context.RequestAborted); | ||
|
|
||
| await context.Response.WriteAsync(decompressedBody, context.RequestAborted); | ||
| }); | ||
| }); | ||
| } | ||
|
|
||
| public static Task Main(string[] args) | ||
| { | ||
| var host = new HostBuilder() | ||
| .ConfigureWebHost(webHostBuilder => | ||
| { | ||
| webHostBuilder | ||
| .UseKestrel() | ||
| .ConfigureLogging(factory => | ||
| { | ||
| factory.AddConsole() | ||
| .SetMinimumLevel(LogLevel.Debug); | ||
| }) | ||
| .UseStartup<Startup>(); | ||
| }).Build(); | ||
|
|
||
| return host.RunAsync(); | ||
| } | ||
| } |
18 changes: 18 additions & 0 deletions
18
src/Middleware/RequestDecompression/src/BrotliDecompressionProvider.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| // Licensed to the .NET Foundation under one or more agreements. | ||
| // The .NET Foundation licenses this file to you under the MIT license. | ||
|
|
||
| using System.IO.Compression; | ||
|
|
||
| namespace Microsoft.AspNetCore.RequestDecompression; | ||
|
|
||
| /// <summary> | ||
| /// Brotli decompression provider. | ||
| /// </summary> | ||
| internal sealed class BrotliDecompressionProvider : IDecompressionProvider | ||
| { | ||
| /// <inheritdoc /> | ||
| public Stream GetDecompressionStream(Stream stream) | ||
| { | ||
| return new BrotliStream(stream, CompressionMode.Decompress); | ||
| } | ||
| } |
81 changes: 81 additions & 0 deletions
81
src/Middleware/RequestDecompression/src/DefaultRequestDecompressionProvider.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| // Licensed to the .NET Foundation under one or more agreements. | ||
| // The .NET Foundation licenses this file to you under the MIT license. | ||
|
|
||
| using Microsoft.AspNetCore.Http; | ||
| using Microsoft.Extensions.Logging; | ||
| using Microsoft.Extensions.Options; | ||
| using Microsoft.Extensions.Primitives; | ||
| using Microsoft.Net.Http.Headers; | ||
|
|
||
| namespace Microsoft.AspNetCore.RequestDecompression; | ||
|
|
||
| /// <inheritdoc /> | ||
| internal sealed partial class DefaultRequestDecompressionProvider : IRequestDecompressionProvider | ||
| { | ||
| private readonly ILogger _logger; | ||
| private readonly IDictionary<string, IDecompressionProvider> _providers; | ||
|
|
||
| public DefaultRequestDecompressionProvider( | ||
| ILogger<DefaultRequestDecompressionProvider> logger, | ||
| IOptions<RequestDecompressionOptions> options) | ||
| { | ||
| if (logger is null) | ||
| { | ||
| throw new ArgumentNullException(nameof(logger)); | ||
| } | ||
|
|
||
| if (options is null) | ||
| { | ||
| throw new ArgumentNullException(nameof(options)); | ||
| } | ||
|
|
||
| _logger = logger; | ||
| _providers = options.Value.DecompressionProviders; | ||
| } | ||
|
|
||
| /// <inheritdoc /> | ||
| public IDecompressionProvider? GetDecompressionProvider(HttpContext context) | ||
| { | ||
| var encodings = context.Request.Headers.ContentEncoding; | ||
|
|
||
| if (StringValues.IsNullOrEmpty(encodings)) | ||
| { | ||
| Log.NoContentEncoding(_logger); | ||
| return null; | ||
| } | ||
|
|
||
| if (encodings.Count > 1) | ||
| { | ||
| Log.MultipleContentEncodingsSpecified(_logger); | ||
| return null; | ||
| } | ||
|
|
||
| string encodingName = encodings!; | ||
|
|
||
| if (_providers.TryGetValue(encodingName, out var matchingProvider)) | ||
| { | ||
| context.Request.Headers.Remove(HeaderNames.ContentEncoding); | ||
|
|
||
| Log.DecompressingWith(_logger, encodingName.ToLowerInvariant()); | ||
| return matchingProvider; | ||
| } | ||
|
|
||
| Log.NoDecompressionProvider(_logger); | ||
| return null; | ||
| } | ||
|
|
||
| private static partial class Log | ||
| { | ||
| [LoggerMessage(1, LogLevel.Trace, "The Content-Encoding header is empty or not specified. Skipping request decompression.", EventName = "NoContentEncoding")] | ||
| public static partial void NoContentEncoding(ILogger logger); | ||
|
|
||
| [LoggerMessage(2, LogLevel.Debug, "Request decompression is not supported for multiple Content-Encodings.", EventName = "MultipleContentEncodingsSpecified")] | ||
| public static partial void MultipleContentEncodingsSpecified(ILogger logger); | ||
|
|
||
| [LoggerMessage(3, LogLevel.Debug, "No matching request decompression provider found.", EventName = "NoDecompressionProvider")] | ||
| public static partial void NoDecompressionProvider(ILogger logger); | ||
|
|
||
| [LoggerMessage(4, LogLevel.Debug, "The request will be decompressed with '{ContentEncoding}'.", EventName = "DecompressingWith")] | ||
| public static partial void DecompressingWith(ILogger logger, string contentEncoding); | ||
| } | ||
| } |
18 changes: 18 additions & 0 deletions
18
src/Middleware/RequestDecompression/src/DeflateDecompressionProvider.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| // Licensed to the .NET Foundation under one or more agreements. | ||
| // The .NET Foundation licenses this file to you under the MIT license. | ||
|
|
||
| using System.IO.Compression; | ||
|
|
||
| namespace Microsoft.AspNetCore.RequestDecompression; | ||
|
|
||
| /// <summary> | ||
| /// DEFLATE decompression provider. | ||
| /// </summary> | ||
| internal sealed class DeflateDecompressionProvider : IDecompressionProvider | ||
| { | ||
| /// <inheritdoc /> | ||
| public Stream GetDecompressionStream(Stream stream) | ||
| { | ||
| return new DeflateStream(stream, CompressionMode.Decompress); | ||
| } | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.