When sending MultipartFormDataContent from HttpClient, the MediaTypeHeaderValue.TryParse does not handle multipart boundary value enclosed with quotes correctly

[aDisplayName]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

TL;DR: When request is submitted by HttpClient via MultipartFormDataContent, the boundary value in multipart/formdata request header is enclosed with double quotes. In this case, when using MediaTypeHeaderValue.TryParse to parse the header, the MediaTypeHeaderValue will include those double quotes as boundary when parsing the header.

Let's say the header from MultipartFormDataContent is:

multipart/form-data; boundary="71e9da95-c5e7-4a9b-901a-ededd613dcd4"

The boundary value will be retrieved as "71e9da95-c5e7-4a9b-901a-ededd613dcd4" with the double quotes, instead of the 71e9da95-c5e7-4a9b-901a-ededd613dcd4 without the double quotes.

According to rfc2046 section 5.1.1 https://www.rfc-editor.org/rfc/rfc2046.html#section-5.1.1, the double quotes are used to avoid parsing error and should be removed when parsing the content body.

Long Story:

Using https://github.com/dotnet/AspNetCore.Docs/tree/main/aspnetcore/mvc/models/file-uploads/samples/5.x/LargeFilesSample as example, we want to test how to use C# to create a client uploading large files.
We changed the runtime from .net5 to .net6 so that it follows our production environment.

The following are the client side code snippet using .NET6 Console Application plus package:
<PackageReference Include="Microsoft.AspNetCore.WebUtilities" Version="2.2.0" />

    // Based on https://stackoverflow.com/a/28242716/3200008
    using var client = new HttpClient()
    {
        Timeout = TimeSpan.FromDays(1)
    };
    client.DefaultRequestHeaders.Add("User-Agent","dotnet-http-client");
    using var content= new MultipartFormDataContent();
    var path = @"C:\bin1234.dat";
    using var sc = new ByteArrayContent(new byte[]{1,2,3,4});
    sc.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
    sc.Headers.ContentDisposition = new ContentDispositionHeaderValue("form-data")
    {
        Name = @"""file""",
        FileName = @$"""{Path.GetFileName(path)}"""
    };
    
    content.Add(sc, "file", Path.GetFileName(path));

    
    var url = "http://localhost:54900/FileUpload/UploadLargeFile";
    using var response = await client.PostAsync(url, content);
    Console.WriteLine($"Status: {response.StatusCode}; Msg: {await response.Content.ReadAsStringAsync()}");

Instead, we always got exception from the controller:

Unexpected end of Stream, the content may have already been read by another component.

We couldn't figure out why. Using the PostMan, Curl in CLI, or javascript in browser, we can always get the correct response.

The exception was found to be thrown at https://github.com/dotnet/AspNetCore.Docs/blob/5e1401df3eb523da89fdb968dab35ff2610c8925/aspnetcore/mvc/models/file-uploads/samples/5.x/LargeFilesSample/Controllers/FileUploadController.cs#L50

    var section = await reader.ReadNextSectionAsync();

To figure out what has been run, we tried to dump all the payload: here is the payload:

--3bf2175e-bc64-44f2-8c59-d8b51b1ed334
Content-Type: application/octet-stream
Content-Disposition: form-data; name="file"; filename="bin1234.dat"

����
--3bf2175e-bc64-44f2-8c59-d8b51b1ed334--

And it seems nothing special.

We even tried to use the same payload to playback, and the request completed without a hitch.

    await using var fs = File.OpenRead("C:\\bin1234_bad.dmp");
    var boundary = "3bf2175e-bc64-44f2-8c59-d8b51b1ed334";
    var reader = new MultipartReader(boundary, fs);
    var section = await reader.ReadNextSectionAsync(CancellationToken.None);

After further debugging, we noticed that the when request are sent by HttpClient via MultipartFormDataContent, the boundary value is always a GUID, and in the header, it is enclosed with double quotes:

multipart/form-data; boundary="71e9da95-c5e7-4a9b-901a-ededd613dcd4"

But in the body, the boundary string are:

--3bf2175e-bc64-44f2-8c59-d8b51b1ed334

as beginning and

--3bf2175e-bc64-44f2-8c59-d8b51b1ed334--

as final.

We've traced into the call of https://github.com/dotnet/AspNetCore.Docs/blob/5e1401df3eb523da89fdb968dab35ff2610c8925/aspnetcore/mvc/models/file-uploads/samples/5.x/LargeFilesSample/Controllers/FileUploadController.cs#L43, and the return result of mediaTypeHeader have a boundary value of "3bf2175e-bc64-44f2-8c59-d8b51b1ed334" (with double quotes), and causing the following ReadNextSectionAsync failed with exception "Unexpected end of Stream, the content may have already been read by another component."

The request sent from browser javascript httpclient, or curl does not have double quotes in their boundary value in header, which will not causing the problem

Expected Behavior

HttpClient can upload large file using MultipartFormDataContent to an aspnet server side controller using https://github.com/dotnet/AspNetCore.Docs/tree/main/aspnetcore/mvc/models/file-uploads/samples/5.x/LargeFilesSample without getting exception.

Steps To Reproduce

1. Run server code from https://github.com/aDisplayName/bugsamplecode/tree/main/20220418/Server
2. Run client code from https://github.com/aDisplayName/bugsamplecode/tree/main/20220418/client

Exceptions (if any)

Status: InternalServerError; Msg: System.IO.IOException: Unexpected end of Stream, the content may have already been read by another component.
   at Microsoft.AspNetCore.WebUtilities.MultipartReaderStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.WebUtilities.StreamHelperExtensions.DrainAsync(Stream stream, ArrayPool`1 bytePool, Nullable`1 limit, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.WebUtilities.MultipartReader.ReadNextSectionAsync(CancellationToken cancellationToken)
   at LargeFilesSample.Controllers.FileUploadController.UploadLargeFile() in C:\Users\LMa2\source\dotnet\AspNetCore.Docs\aspnetcore\mvc\models\file-uploads\samples\5.x\LargeFilesSample\Controllers\FileUploadController.cs:line 49
   at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.TaskOfIActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask`1 actionResultValueTask)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

HEADERS
=======
Host: localhost:54900
User-Agent: dotnet-http-client
Content-Type: multipart/form-data; boundary="d9895de8-33e4-45e7-ba45-eab908473ce7"
Content-Length: 199

.NET Version

6.0.201

Anything else?

No response

[Tratcher]

I think the issue is here:
https://github.com/dotnet/AspNetCore.Docs/blob/5f362035992cc3b997903dda521a01ed59058dec/aspnetcore/mvc/models/file-uploads/samples/5.x/LargeFilesSample/Controllers/FileUploadController.cs#L49

The MediaTypeHeaderValue.Boundary is expected to be quoted, but MultipartReader requires the value to not be quoted. There should be an additional line here that trims the quotes.

We may also consider relaxing that restriction in MultipartReader and have it remove the quotes.

aspnetcore/src/Http/WebUtilities/src/MultipartBoundary.cs

Lines 26 to 35 in 7a1a1eb

	private void Initialize(string boundary, bool expectLeadingCrlf)
	{
	if (expectLeadingCrlf)
	{
	BoundaryBytes = Encoding.UTF8.GetBytes("\r\n--" + boundary);
	}
	else
	{
	BoundaryBytes = Encoding.UTF8.GetBytes("--" + boundary);
	}

[aDisplayName]

The current workaround I'm using is to remove the double quotes from the beginning and the end if they exist.

[Tratcher]

Here's an example from the built-in form reader:

aspnetcore/src/Http/Http/src/Features/FormFeature.cs

Lines 318 to 320 in 4eef6a1

	private static string GetBoundary(MediaTypeHeaderValue contentType, int lengthLimit)
	{
	var boundary = HeaderUtilities.RemoveQuotes(contentType.Boundary);

HeaderUtilities.RemoveQuotes is a publicly available API.

[adityamandaleeka]

Triage: we could make MultipartFormReader remove the quotes by calling HeaderUtilities.RemoveQuotes in the constructor.

Thanks for contacting us.

We're moving this issue to the .NET 7 Planning milestone for future evaluation / consideration. We would like to keep this around to collect more feedback, which can help us with prioritizing this work. We will re-evaluate this issue, during our next planning meeting(s).
If we later determine, that the issue has no community involvement, or it's very rare and low-impact issue, we will close it - so that the team can focus on more important and high impact issues.
To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.