Skip to content

Azure.Core 2.0: Update HttpPipelinePolicy, RequestFailedException, and transport types#42294

Merged
annelo-msft merged 45 commits intoAzure:feature/azure.core-2.0from
annelo-msft:core2.0-transport
Mar 1, 2024
Merged

Azure.Core 2.0: Update HttpPipelinePolicy, RequestFailedException, and transport types#42294
annelo-msft merged 45 commits intoAzure:feature/azure.core-2.0from
annelo-msft:core2.0-transport

Conversation

@annelo-msft
Copy link
Copy Markdown
Member

@annelo-msft annelo-msft commented Feb 29, 2024

Description

Many of the basic Azure.Core types now have analogous types in the System.ClientModel library. In order to follow the DRY engineering principle, we plan to update Azure.Core to reuse functionality from these types, using either inheritance or adapters in most cases.

This PR makes the following changes:

  • Azure.Core HttpPipelinePolicy inherits from System.ClientModel PipelinePolicy
  • Azure.Core HttpPipelineTransport inherits from System.ClientModel PipelineTransport
  • Azure.Core HttpClientTransport adapts System.ClientModel HttpClientPipelineTransport
  • Uses response buffering implementation in the System.ClientModel PipelineTransport and so is able to remove internal ResponseBodyPolicy from Azure.Core. As part of this, moves NetworkTimeout value to be a property of HttpPipeline.
  • Azure.Core RequestFailedException inherits from System.ClientModel ClientRequestException

Design Motivation

This one is the most complex of the integration pieces. Because System.ClientModel HttpClientPipelineTransport request and response implementations are private, we have to call through to HttpClientPipelineTransport.CreateMessage to get an instance of the request. This means fields backing the Request properties are duplicated across the adapted PipelineRequest instance and the virtual properties on the base Request type.

Context

This PR is a smaller chunk of #41773 to make it easier to review. In addition, it this PR targets the feature/azure.core-2.0 feature branch because we cannot move the Azure.Core 2.0 integration into main until after System.ClientModel 1.1.0 is GA.

Debug.Assert(pipeline.IsEmpty);

await _transport.ProcessAsync(message).ConfigureAwait(false);
await _transport.ProcessAsync(message as PipelineMessage).ConfigureAwait(false);
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This calls through to the base PipelineTransport implementation that handles the response buffering now.

@annelo-msft annelo-msft marked this pull request as ready for review March 1, 2024 01:05
@annelo-msft annelo-msft requested review from a team, KrzysztofCwalina and tg-msft as code owners March 1, 2024 01:05
{
_responseMessage?.Dispose();
DisposeStreamIfNotBuffered(ref _contentStream);
if (response.ContentStream is MemoryStream &&
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the MemoryStream type necessary here, or are we including it just to avoid any behavioral changes?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's required because of the current behavior of HttpMessage.ExtractResponseContent -- it replaces ContentStream with a stream that throws if you call CanSeek on it. Checking MemoryStream limits the check to cases where either the transport response type did the buffering or a unit test replaced the ContentStream with something custom.

annelo-msft added a commit to annelo-msft/azure-sdk-for-net that referenced this pull request Mar 1, 2024
// Licensed under the MIT License.

using System;
using System.ClientModel;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be a lot of complex networking code here. I was expecting that all this code would be in client model and then Azure.Core would have a very simple call forwarding adapter. If we cannot get to that point, is there a good reason for all this inheritance or we should have two separate pipelines?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you referring to lines 126-224? This is for cookie, proxy, and cert handling that I had assumed were specific to Azure clients. If we feel they are general enough to move into ClientModel, we can do that. The cookie we have is named "Azure.Core.Pipeline.HttpClientTransport.EnableCookies" so we would probably want a different name for that one.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not any single part of the code. It's the overall feeling that the resulting Azure.Core is now even harder to understand than it was before. Possibly I am wrong about this, but I want to make sure we don't end up with a mess in Azure.Core.

/// Represent an extension point for the <see cref="HttpPipeline"/> that can mutate the <see cref="Request"/> and react to received <see cref="Response"/>.
/// </summary>
public abstract class HttpPipelinePolicy
public abstract class HttpPipelinePolicy : PipelinePolicy
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are doing adapters, is the inheritance needed?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ClientModel pipeline needs to be able to call ProcessNext on an Azure.Core policy from a System.ClientModel policy. We can look at another way of doing this, but inheritance allows us to do this rather simply.

@annelo-msft
Copy link
Copy Markdown
Member Author

I've added the inheritance for RequestFailedException to this PR since there is a Search client test that relies on the implementation of RFE that uses the new buffering logic from ClientModel. It will bring some DigitalTwins dependencies with it, as those have refdocs that rely on a property that has moved to ClientModel.

{
_response = response;
}
#region ISerializable implementation
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example, @KrzysztofCwalina, we could remove the serialization constructor from the base exception type in ClientModel if we wanted to, and add it back if/when we did the Azure.Core 2.0 integration.

@annelo-msft annelo-msft changed the title Azure.Core 2.0: Update HttpPipelinePolicy and transport types Azure.Core 2.0: Update HttpPipelinePolicy, RequestFailedException, and transport types Mar 1, 2024
@annelo-msft
Copy link
Copy Markdown
Member Author

Discussed offline with @KrzysztofCwalina that it is ok to merge this for now -- we can revisit our approach here separately.

@annelo-msft annelo-msft merged commit 8ac9315 into Azure:feature/azure.core-2.0 Mar 1, 2024
annelo-msft added a commit to annelo-msft/azure-sdk-for-net that referenced this pull request Mar 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants