BlockBlobClient throws System.IO.IOException when it is used for uploading a large file.

[shuk]

BlockBlobClient throws the exception when it is used for uploading a large file (1.2Gb) to Azure Gov Blob Storage. The issue is actual for large files only. It is interesting that BlobClient uploads the same file successfully. The details are below.

Library: 12.1.0 (https://www.nuget.org/packages/Azure.Storage.Blobs/12.1.0)

App: .NET Core 2.2, Console application

Exception:
"System.Threading.Tasks.TaskCanceledException: The operation was canceled. ---> System.Net.Http.HttpRequestException: Error while copying content to a stream. ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request. ---> System.Net.Sockets.SocketException: The I/O operation has been aborted because of either a thread exit or an application request
--- End of inner exception stack trace ---
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
at System.Net.Security.SslStreamInternal.g__CompleteAsync|36_1[TWriteAdapter](ValueTask writeTask, Byte[] bufferToReturn)
at System.Net.Security.SslStreamInternal.WriteAsyncChunked[TWriteAdapter](TWriteAdapter writeAdapter, ReadOnlyMemory1 buffer) at System.Net.Security.SslStreamInternal.<WriteAsyncInternal>g__ExitWriteAsync|35_0[TWriteAdapter](ValueTask task) at System.Net.Http.HttpConnection.WriteAsync(ReadOnlyMemory1 source)
at System.IO.Stream.CopyToAsyncInternal(Stream destination, Int32 bufferSize, CancellationToken cancellationToken)
at Azure.Core.RequestContent.StreamContent.WriteToAsync(Stream stream, CancellationToken cancellation)
at Azure.Core.Pipeline.HttpClientTransport.PipelineRequest.PipelineContentAdapter.SerializeToStreamAsync(Stream stream, TransportContext context)
at System.Net.Http.HttpContent.CopyToAsyncCore(ValueTask copyTask)
--- End of inner exception stack trace ---
at System.Net.Http.HttpContent.CopyToAsyncCore(ValueTask copyTask)
at System.Net.Http.HttpConnection.SendRequestContentAsync(HttpRequestMessage request, HttpContentWriteStream stream, CancellationToken cancellationToken)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) at Azure.Core.Pipeline.HttpClientTransport.ProcessAsync(HttpMessage message) at Azure.Core.Pipeline.RequestActivityPolicy.ProcessNextAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean isAsync)
at Azure.Core.Pipeline.RequestActivityPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean isAsync) at Azure.Core.Pipeline.BufferResponsePolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline)
at Azure.Core.Pipeline.LoggingPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async) at Azure.Core.Pipeline.LoggingPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline)
at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline)
at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async) at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async)
at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline)
at Azure.Storage.Blobs.BlobRestClient.BlockBlob.UploadAsync(ClientDiagnostics clientDiagnostics, HttpPipeline pipeline, Uri resourceUri, Stream body, Int64 contentLength, Nullable1 timeout, Byte[] transactionalContentHash, String blobContentType, String blobContentEncoding, String blobContentLanguage, Byte[] blobContentHash, String blobCacheControl, IDictionary2 metadata, String leaseId, String blobContentDisposition, String encryptionKey, String encryptionKeySha256, Nullable1 encryptionAlgorithm, Nullable1 tier, Nullable1 ifModifiedSince, Nullable1 ifUnmodifiedSince, Nullable1 ifMatch, Nullable1 ifNoneMatch, String requestId, Boolean async, String operationName, CancellationToken cancellationToken)
at Azure.Storage.Blobs.Specialized.BlockBlobClient.UploadInternal(Stream content, BlobHttpHeaders blobHttpHeaders, IDictionary2 metadata, BlobRequestConditions conditions, Nullable1 accessTier, IProgress1 progressHandler, Boolean async, CancellationToken cancellationToken) at Azure.Storage.Blobs.Specialized.BlockBlobClient.UploadAsync(Stream content, BlobHttpHeaders httpHeaders, IDictionary2 metadata, BlobRequestConditions conditions, Nullable1 accessTier, IProgress1 progressHandler, CancellationToken cancellationToken)"

Code:
`
var sharedKeyCredential = new StorageSharedKeyCredential("account-name", "account-key");
var blobServiceClient = new BlobServiceClient(
new Uri("https://miniostoragepoc.blob.core.usgovcloudapi.net"),
sharedKeyCredential);

BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient("test-container-for-large-files");

await containerClient.CreateIfNotExistsAsync();

using (FileStream uploadFileStream = File.OpenRead(@"c:\large_file.MP4"))
{
    BlockBlobClient blobClient = containerClient.GetBlockBlobClient("large_file.MP4");
    await blobClient.UploadAsync(uploadFileStream); // <-- The exception is here
}

`

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @xgithubtriage

[seanmcc-msft]

@tg-msft, could you take a look?

[seanmcc-msft]

It looks like BlockBlobClient.UploadInteral() doesn't support parallel and multi-part uploads:

https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/storage/Azure.Storage.Blobs/src/BlockBlobClient.cs#L492

But BlobClient.StagedUploadAsync() does:

https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/storage/Azure.Storage.Blobs/src/BlobClient.cs#L966

This is also a problem in AppendBlobClient.AppendBlockInternal() and PageBlobClient.UploadPagesAsync():

https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/storage/Azure.Storage.Blobs/src/AppendBlobClient.cs#L679

For AppendBlob, we can support multi-part appends, but not parallel appends.

https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/storage/Azure.Storage.Blobs/src/PageBlobClient.cs#L796

I think we can re-used BlobClient.StagedUploadAsync()'s logic for BlockBlobClient.UploadInteral(), but it would need to be re-written or generalized for AppendBlobClient.AppendBlockInteral() and PageBlobClient.UploadPagesInternal()