ClientModel: Change approach to buffering response.Content

[annelo-msft]

Problem Description

The APIs we have today in Azure.Core around Response.Content and Response.ContentStream and buffering are not transparent. It is not clear to the end-user which responses are buffered and which are not. The only way for users to discover this is to read the docs for a service method.

High-level proposal

Make a change to the current implementation of the PipelineResponse.Content property such that we no longer throw if the response has not been buffered (i.e. PipelineMessage.BufferResponse has been set to false by the client-author). This means that end-users can always access PipelineResponse.Content and if it is not buffered, it will buffer the response in the response.Content field. In the rare situation where response.Content is larger than what we can fit in memory, the user will either get an IOException or -- since we will have wrapped the live network stream in a ReadTimeoutStream -- the buffering will throw a TaskCanceledException due to the network timeout being applied.

Implementation proposal

(From @KrzysztofCwalina and under investigation per feasibility in #41693 and related PRs)

Could we do the following?

In the case of unbuffered responses:

1. Service method creates, composes, and sends an instance of HttpMessage.
2. The transport stashed HttpRequestMessage in the Response
3. Service method returns Response to the user.
4. If the user accesses Content first, we read the live stream, buffer into byte[], and return BinaryData wrapping the byte[]. If a subsequent call accesses ContentStream we either throw or return MemoryStream wrapping the byte[]
5. If the user accesses ContentStream first, any subsequent access to Content or ContentStream will throw InvalidOperationException

In the case of buffered responses:

1. Service method creates, composes, and sends an instance of HttpMessage.
2. The transport reads the live stream into a byte[], stashes the byte[] in Response, disposes the HttpRequestMessage
3. Service method returns Response to the user.
4. If the caller accesses Content, we return BinaryData wrapping the byte[]
5. If the caller accesses ContetnStream, we return MemoryStream wrapping the byte[]

Prior notes

See this comment thread for details: #41016 (comment)

This issue tracks the work to change how response buffering works for ClientModel responses. @KrzysztofCwalina would like to optimize for buffered content in the BinaryData response.Content property and not retain the MemoryStream we used to buffer the content in the ContentStream property. In a greenfield scenario, this would be great, but with a lot of code built on top of Azure.Core's response.ContentStream property and the idea that Content throws if ContentStream doesn't hold the buffered MemoryStream, we would likely need to go through breaking change review with the arch board to assess the feasibility of this change.

A proposed implementation that helps identify the CI tests that are affected by this change can be found here: #41047

[annelo-msft]

Initial notes from work on #41693 and annelo-msft#13:

• The proposal assumes we have moved buffering into the transport (tracked by ClientModel: Investigate feasibility of moving response buffering into the transport #41198)
• Buffering the network stream in the response.Content property means that we must do this synchronously. We may be able to do this with the cancellation token on the message if we store it as an internal property on the response.
• We may not be able to provide a new implementation for response.ContentStream in the base type PipelineResponse, because this is an abstract property in Azure.Core's Response, which means all existing implementations have overridden any implementation we might provide there if we changed this to virtual. So, a lazily-created stream need to be created in the derived type.
• It would be great if we could have the simplified story around disposing the streams -- if we've buffered the response (message.BufferResponse was true), the transport called Dispose on the network stream; if we didn't buffer the response -- we either Dispose the network stream when the user calls response.Content or it is the responsibility of the end-user to dispose response.ContentStream after reading from it (the streaming APIs where we set message.BufferRespose to false are uncommon and low-level/advanced). This is complicated by the public method ExtractResponseContent on HttpMessage and the assumption that that will be called if/when we pass the ContentStream to the end user in a Response from a service method. Today, if that is not called, the response is expected to dispose the ContentStream if it is not buffered in its own Dispose method.
  • It looks like ExtractResponseContent was added in this PR: Add CreateMessage method to the pipeline #7636

I am still working on an end-to-end solution to complete the feasibility assessment.