Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failures in SocketsHttpHandler_HttpClientHandler_MaxResponseHeadersLength_Http3 #73930

Closed
MichalStrehovsky opened this issue Aug 15, 2022 · 11 comments · Fixed by #74479
Closed
Assignees
Labels
area-System.Net.Http disabled-test The test is disabled in source code against the issue
Milestone

Comments

@MichalStrehovsky
Copy link
Member

Full log: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-72145-merge-c270b69ee20f4e269e/System.Net.Http.Functional.Tests/3/console.142a186b.log?helixlogtype=result

Hit in https://github.com/dotnet/runtime/pull/72145/checks?check_run_id=7832255512

System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_MaxResponseHeadersLength_Http3.LargeSingleHeader_ThrowsException(maxResponseHeadersLength: 15) [FAIL]
      System.Net.Quic.QuicException : Stream aborted by peer (268).
      Stack Trace:
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs(157,0): at System.Net.Quic.ResettableValueTaskSource.TryComplete(Exception exception, Boolean final)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs(221,0): at System.Net.Quic.ResettableValueTaskSource.TrySetException(Exception exception, Boolean final)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs(516,0): at System.Net.Quic.QuicStream.HandleEventPeerReceiveAborted(_PEER_RECEIVE_ABORTED_e__Struct& data)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs(570,0): at System.Net.Quic.QuicStream.HandleStreamEvent(QUIC_STREAM_EVENT& streamEvent)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs(601,0): at System.Net.Quic.QuicStream.NativeCallback(QUIC_HANDLE* connection, Void* context, QUIC_STREAM_EVENT* streamEvent)
        --- End of stack trace from previous location ---
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs(267,0): at System.Net.Quic.ResettableValueTaskSource.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(168,0): at System.Net.Test.Common.Http3LoopbackStream.SendFrameAsync(Int64 frameType, ReadOnlyMemory`1 framePayload)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(123,0): at System.Net.Test.Common.Http3LoopbackStream.SendHeadersFrameAsync(Nullable`1 statusCode, IEnumerable`1 headers, Boolean qpackEncodeStatus)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(264,0): at System.Net.Test.Common.Http3LoopbackStream.SendResponseHeadersAsync(Nullable`1 statusCode, IEnumerable`1 headers)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(247,0): at System.Net.Test.Common.Http3LoopbackStream.SendResponseAsync(HttpStatusCode statusCode, IList`1 headers, String content, Boolean isFinal)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackConnection.cs(241,0): at System.Net.Test.Common.Http3LoopbackConnection.HandleRequestAsync(HttpStatusCode statusCode, IList`1 headers, String content)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs(93,0): at System.Net.Test.Common.Http3LoopbackServer.HandleRequestAsync(HttpStatusCode statusCode, IList`1 headers, String content)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs(93,0): at System.Net.Test.Common.Http3LoopbackServer.HandleRequestAsync(HttpStatusCode statusCode, IList`1 headers, String content)
        /_/src/libraries/Common/tests/System/Net/Http/HttpClientHandlerTest.MaxResponseHeadersLength.cs(84,0): at System.Net.Http.Functional.Tests.HttpClientHandler_MaxResponseHeadersLength_Test.<>c__DisplayClass4_0.<<LargeSingleHeader_ThrowsException>b__1>d.MoveNext()
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(120,0): at System.Threading.Tasks.TaskTimeoutExtensions.GetRealException(Task task)
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(90,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks)
        /_/src/libraries/Common/tests/System/Net/Http/GenericLoopbackServer.cs(38,0): at System.Net.Test.Common.LoopbackServerFactory.<>c__DisplayClass5_0.<<CreateClientAndServerAsync>b__0>d.MoveNext()
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs(111,0): at System.Net.Test.Common.Http3LoopbackServerFactory.CreateServerAsync(Func`3 funcAsync, Int32 millisecondsTimeout, GenericLoopbackOptions options)
        /_/src/libraries/Common/tests/System/Net/Http/HttpClientHandlerTest.MaxResponseHeadersLength.cs(72,0): at System.Net.Http.Functional.Tests.HttpClientHandler_MaxResponseHeadersLength_Test.LargeSingleHeader_ThrowsException(Int32 maxResponseHeadersLength)
        --- End of stack trace from previous location ---
    System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Authentication_Test.Credentials_DomainJoinedServerUsesKerberos_UseIpAddressAndHostHeader_Success [SKIP]
      Condition(s) not met: "IsDomainJoinedServerAvailable"
    System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Authentication_Test.Proxy_DomainJoinedProxyServerUsesKerberos_Success [SKIP]
      Condition(s) not met: "IsDomainJoinedServerAvailable"
    System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Authentication_Test.Credentials_ServerChallengesWithWindowsAuth_ClientSendsWindowsAuthHeader [SKIP]
      Condition(s) not met: "IsNtlmInstalled"
    System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_MaxResponseHeadersLength_Http3.ThresholdExceeded_ThrowsException(maxResponseHeadersLength: 1, headersLengthEstimate: 1024) [FAIL]
      System.Net.Quic.QuicException : Stream aborted by peer (268).
      Stack Trace:
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs(157,0): at System.Net.Quic.ResettableValueTaskSource.TryComplete(Exception exception, Boolean final)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs(221,0): at System.Net.Quic.ResettableValueTaskSource.TrySetException(Exception exception, Boolean final)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs(516,0): at System.Net.Quic.QuicStream.HandleEventPeerReceiveAborted(_PEER_RECEIVE_ABORTED_e__Struct& data)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs(570,0): at System.Net.Quic.QuicStream.HandleStreamEvent(QUIC_STREAM_EVENT& streamEvent)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs(601,0): at System.Net.Quic.QuicStream.NativeCallback(QUIC_HANDLE* connection, Void* context, QUIC_STREAM_EVENT* streamEvent)
        --- End of stack trace from previous location ---
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs(267,0): at System.Net.Quic.ResettableValueTaskSource.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(168,0): at System.Net.Test.Common.Http3LoopbackStream.SendFrameAsync(Int64 frameType, ReadOnlyMemory`1 framePayload)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(123,0): at System.Net.Test.Common.Http3LoopbackStream.SendHeadersFrameAsync(Nullable`1 statusCode, IEnumerable`1 headers, Boolean qpackEncodeStatus)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(264,0): at System.Net.Test.Common.Http3LoopbackStream.SendResponseHeadersAsync(Nullable`1 statusCode, IEnumerable`1 headers)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(247,0): at System.Net.Test.Common.Http3LoopbackStream.SendResponseAsync(HttpStatusCode statusCode, IList`1 headers, String content, Boolean isFinal)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackConnection.cs(241,0): at System.Net.Test.Common.Http3LoopbackConnection.HandleRequestAsync(HttpStatusCode statusCode, IList`1 headers, String content)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs(93,0): at System.Net.Test.Common.Http3LoopbackServer.HandleRequestAsync(HttpStatusCode statusCode, IList`1 headers, String content)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs(93,0): at System.Net.Test.Common.Http3LoopbackServer.HandleRequestAsync(HttpStatusCode statusCode, IList`1 headers, String content)
        /_/src/libraries/Common/tests/System/Net/Http/HttpClientHandlerTest.MaxResponseHeadersLength.cs(128,0): at System.Net.Http.Functional.Tests.HttpClientHandler_MaxResponseHeadersLength_Test.<>c__DisplayClass5_0.<<ThresholdExceeded_ThrowsException>b__1>d.MoveNext()
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(120,0): at System.Threading.Tasks.TaskTimeoutExtensions.GetRealException(Task task)
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(90,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks)
        /_/src/libraries/Common/tests/System/Net/Http/GenericLoopbackServer.cs(38,0): at System.Net.Test.Common.LoopbackServerFactory.<>c__DisplayClass5_0.<<CreateClientAndServerAsync>b__0>d.MoveNext()
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs(111,0): at System.Net.Test.Common.Http3LoopbackServerFactory.CreateServerAsync(Func`3 funcAsync, Int32 millisecondsTimeout, GenericLoopbackOptions options)
        /_/src/libraries/Common/tests/System/Net/Http/HttpClientHandlerTest.MaxResponseHeadersLength.cs(96,0): at System.Net.Http.Functional.Tests.HttpClientHandler_MaxResponseHeadersLength_Test.ThresholdExceeded_ThrowsException(Nullable`1 maxResponseHeadersLength, Int32 headersLengthEstimate)
        --- End of stack trace from previous location ---

Cc @MihaZupan

@MichalStrehovsky MichalStrehovsky added blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' area-System.Net.Quic labels Aug 15, 2022
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Aug 15, 2022
@ghost
Copy link

ghost commented Aug 15, 2022

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

Full log: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-72145-merge-c270b69ee20f4e269e/System.Net.Http.Functional.Tests/3/console.142a186b.log?helixlogtype=result

Hit in https://github.com/dotnet/runtime/pull/72145/checks?check_run_id=7832255512

System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_MaxResponseHeadersLength_Http3.LargeSingleHeader_ThrowsException(maxResponseHeadersLength: 15) [FAIL]
      System.Net.Quic.QuicException : Stream aborted by peer (268).
      Stack Trace:
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs(157,0): at System.Net.Quic.ResettableValueTaskSource.TryComplete(Exception exception, Boolean final)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs(221,0): at System.Net.Quic.ResettableValueTaskSource.TrySetException(Exception exception, Boolean final)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs(516,0): at System.Net.Quic.QuicStream.HandleEventPeerReceiveAborted(_PEER_RECEIVE_ABORTED_e__Struct& data)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs(570,0): at System.Net.Quic.QuicStream.HandleStreamEvent(QUIC_STREAM_EVENT& streamEvent)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs(601,0): at System.Net.Quic.QuicStream.NativeCallback(QUIC_HANDLE* connection, Void* context, QUIC_STREAM_EVENT* streamEvent)
        --- End of stack trace from previous location ---
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs(267,0): at System.Net.Quic.ResettableValueTaskSource.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(168,0): at System.Net.Test.Common.Http3LoopbackStream.SendFrameAsync(Int64 frameType, ReadOnlyMemory`1 framePayload)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(123,0): at System.Net.Test.Common.Http3LoopbackStream.SendHeadersFrameAsync(Nullable`1 statusCode, IEnumerable`1 headers, Boolean qpackEncodeStatus)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(264,0): at System.Net.Test.Common.Http3LoopbackStream.SendResponseHeadersAsync(Nullable`1 statusCode, IEnumerable`1 headers)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(247,0): at System.Net.Test.Common.Http3LoopbackStream.SendResponseAsync(HttpStatusCode statusCode, IList`1 headers, String content, Boolean isFinal)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackConnection.cs(241,0): at System.Net.Test.Common.Http3LoopbackConnection.HandleRequestAsync(HttpStatusCode statusCode, IList`1 headers, String content)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs(93,0): at System.Net.Test.Common.Http3LoopbackServer.HandleRequestAsync(HttpStatusCode statusCode, IList`1 headers, String content)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs(93,0): at System.Net.Test.Common.Http3LoopbackServer.HandleRequestAsync(HttpStatusCode statusCode, IList`1 headers, String content)
        /_/src/libraries/Common/tests/System/Net/Http/HttpClientHandlerTest.MaxResponseHeadersLength.cs(84,0): at System.Net.Http.Functional.Tests.HttpClientHandler_MaxResponseHeadersLength_Test.<>c__DisplayClass4_0.<<LargeSingleHeader_ThrowsException>b__1>d.MoveNext()
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(120,0): at System.Threading.Tasks.TaskTimeoutExtensions.GetRealException(Task task)
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(90,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks)
        /_/src/libraries/Common/tests/System/Net/Http/GenericLoopbackServer.cs(38,0): at System.Net.Test.Common.LoopbackServerFactory.<>c__DisplayClass5_0.<<CreateClientAndServerAsync>b__0>d.MoveNext()
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs(111,0): at System.Net.Test.Common.Http3LoopbackServerFactory.CreateServerAsync(Func`3 funcAsync, Int32 millisecondsTimeout, GenericLoopbackOptions options)
        /_/src/libraries/Common/tests/System/Net/Http/HttpClientHandlerTest.MaxResponseHeadersLength.cs(72,0): at System.Net.Http.Functional.Tests.HttpClientHandler_MaxResponseHeadersLength_Test.LargeSingleHeader_ThrowsException(Int32 maxResponseHeadersLength)
        --- End of stack trace from previous location ---
    System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Authentication_Test.Credentials_DomainJoinedServerUsesKerberos_UseIpAddressAndHostHeader_Success [SKIP]
      Condition(s) not met: "IsDomainJoinedServerAvailable"
    System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Authentication_Test.Proxy_DomainJoinedProxyServerUsesKerberos_Success [SKIP]
      Condition(s) not met: "IsDomainJoinedServerAvailable"
    System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Authentication_Test.Credentials_ServerChallengesWithWindowsAuth_ClientSendsWindowsAuthHeader [SKIP]
      Condition(s) not met: "IsNtlmInstalled"
    System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_MaxResponseHeadersLength_Http3.ThresholdExceeded_ThrowsException(maxResponseHeadersLength: 1, headersLengthEstimate: 1024) [FAIL]
      System.Net.Quic.QuicException : Stream aborted by peer (268).
      Stack Trace:
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs(157,0): at System.Net.Quic.ResettableValueTaskSource.TryComplete(Exception exception, Boolean final)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs(221,0): at System.Net.Quic.ResettableValueTaskSource.TrySetException(Exception exception, Boolean final)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs(516,0): at System.Net.Quic.QuicStream.HandleEventPeerReceiveAborted(_PEER_RECEIVE_ABORTED_e__Struct& data)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs(570,0): at System.Net.Quic.QuicStream.HandleStreamEvent(QUIC_STREAM_EVENT& streamEvent)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs(601,0): at System.Net.Quic.QuicStream.NativeCallback(QUIC_HANDLE* connection, Void* context, QUIC_STREAM_EVENT* streamEvent)
        --- End of stack trace from previous location ---
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs(267,0): at System.Net.Quic.ResettableValueTaskSource.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(168,0): at System.Net.Test.Common.Http3LoopbackStream.SendFrameAsync(Int64 frameType, ReadOnlyMemory`1 framePayload)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(123,0): at System.Net.Test.Common.Http3LoopbackStream.SendHeadersFrameAsync(Nullable`1 statusCode, IEnumerable`1 headers, Boolean qpackEncodeStatus)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(264,0): at System.Net.Test.Common.Http3LoopbackStream.SendResponseHeadersAsync(Nullable`1 statusCode, IEnumerable`1 headers)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(247,0): at System.Net.Test.Common.Http3LoopbackStream.SendResponseAsync(HttpStatusCode statusCode, IList`1 headers, String content, Boolean isFinal)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackConnection.cs(241,0): at System.Net.Test.Common.Http3LoopbackConnection.HandleRequestAsync(HttpStatusCode statusCode, IList`1 headers, String content)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs(93,0): at System.Net.Test.Common.Http3LoopbackServer.HandleRequestAsync(HttpStatusCode statusCode, IList`1 headers, String content)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs(93,0): at System.Net.Test.Common.Http3LoopbackServer.HandleRequestAsync(HttpStatusCode statusCode, IList`1 headers, String content)
        /_/src/libraries/Common/tests/System/Net/Http/HttpClientHandlerTest.MaxResponseHeadersLength.cs(128,0): at System.Net.Http.Functional.Tests.HttpClientHandler_MaxResponseHeadersLength_Test.<>c__DisplayClass5_0.<<ThresholdExceeded_ThrowsException>b__1>d.MoveNext()
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(120,0): at System.Threading.Tasks.TaskTimeoutExtensions.GetRealException(Task task)
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(90,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks)
        /_/src/libraries/Common/tests/System/Net/Http/GenericLoopbackServer.cs(38,0): at System.Net.Test.Common.LoopbackServerFactory.<>c__DisplayClass5_0.<<CreateClientAndServerAsync>b__0>d.MoveNext()
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs(111,0): at System.Net.Test.Common.Http3LoopbackServerFactory.CreateServerAsync(Func`3 funcAsync, Int32 millisecondsTimeout, GenericLoopbackOptions options)
        /_/src/libraries/Common/tests/System/Net/Http/HttpClientHandlerTest.MaxResponseHeadersLength.cs(96,0): at System.Net.Http.Functional.Tests.HttpClientHandler_MaxResponseHeadersLength_Test.ThresholdExceeded_ThrowsException(Nullable`1 maxResponseHeadersLength, Int32 headersLengthEstimate)
        --- End of stack trace from previous location ---

Cc @MihaZupan

Author: MichalStrehovsky
Assignees: -
Labels:

blocking-clean-ci, area-System.Net.Quic

Milestone: -

@ghost
Copy link

ghost commented Aug 15, 2022

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

Full log: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-72145-merge-c270b69ee20f4e269e/System.Net.Http.Functional.Tests/3/console.142a186b.log?helixlogtype=result

Hit in https://github.com/dotnet/runtime/pull/72145/checks?check_run_id=7832255512

System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_MaxResponseHeadersLength_Http3.LargeSingleHeader_ThrowsException(maxResponseHeadersLength: 15) [FAIL]
      System.Net.Quic.QuicException : Stream aborted by peer (268).
      Stack Trace:
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs(157,0): at System.Net.Quic.ResettableValueTaskSource.TryComplete(Exception exception, Boolean final)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs(221,0): at System.Net.Quic.ResettableValueTaskSource.TrySetException(Exception exception, Boolean final)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs(516,0): at System.Net.Quic.QuicStream.HandleEventPeerReceiveAborted(_PEER_RECEIVE_ABORTED_e__Struct& data)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs(570,0): at System.Net.Quic.QuicStream.HandleStreamEvent(QUIC_STREAM_EVENT& streamEvent)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs(601,0): at System.Net.Quic.QuicStream.NativeCallback(QUIC_HANDLE* connection, Void* context, QUIC_STREAM_EVENT* streamEvent)
        --- End of stack trace from previous location ---
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs(267,0): at System.Net.Quic.ResettableValueTaskSource.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(168,0): at System.Net.Test.Common.Http3LoopbackStream.SendFrameAsync(Int64 frameType, ReadOnlyMemory`1 framePayload)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(123,0): at System.Net.Test.Common.Http3LoopbackStream.SendHeadersFrameAsync(Nullable`1 statusCode, IEnumerable`1 headers, Boolean qpackEncodeStatus)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(264,0): at System.Net.Test.Common.Http3LoopbackStream.SendResponseHeadersAsync(Nullable`1 statusCode, IEnumerable`1 headers)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(247,0): at System.Net.Test.Common.Http3LoopbackStream.SendResponseAsync(HttpStatusCode statusCode, IList`1 headers, String content, Boolean isFinal)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackConnection.cs(241,0): at System.Net.Test.Common.Http3LoopbackConnection.HandleRequestAsync(HttpStatusCode statusCode, IList`1 headers, String content)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs(93,0): at System.Net.Test.Common.Http3LoopbackServer.HandleRequestAsync(HttpStatusCode statusCode, IList`1 headers, String content)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs(93,0): at System.Net.Test.Common.Http3LoopbackServer.HandleRequestAsync(HttpStatusCode statusCode, IList`1 headers, String content)
        /_/src/libraries/Common/tests/System/Net/Http/HttpClientHandlerTest.MaxResponseHeadersLength.cs(84,0): at System.Net.Http.Functional.Tests.HttpClientHandler_MaxResponseHeadersLength_Test.<>c__DisplayClass4_0.<<LargeSingleHeader_ThrowsException>b__1>d.MoveNext()
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(120,0): at System.Threading.Tasks.TaskTimeoutExtensions.GetRealException(Task task)
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(90,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks)
        /_/src/libraries/Common/tests/System/Net/Http/GenericLoopbackServer.cs(38,0): at System.Net.Test.Common.LoopbackServerFactory.<>c__DisplayClass5_0.<<CreateClientAndServerAsync>b__0>d.MoveNext()
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs(111,0): at System.Net.Test.Common.Http3LoopbackServerFactory.CreateServerAsync(Func`3 funcAsync, Int32 millisecondsTimeout, GenericLoopbackOptions options)
        /_/src/libraries/Common/tests/System/Net/Http/HttpClientHandlerTest.MaxResponseHeadersLength.cs(72,0): at System.Net.Http.Functional.Tests.HttpClientHandler_MaxResponseHeadersLength_Test.LargeSingleHeader_ThrowsException(Int32 maxResponseHeadersLength)
        --- End of stack trace from previous location ---
    System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Authentication_Test.Credentials_DomainJoinedServerUsesKerberos_UseIpAddressAndHostHeader_Success [SKIP]
      Condition(s) not met: "IsDomainJoinedServerAvailable"
    System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Authentication_Test.Proxy_DomainJoinedProxyServerUsesKerberos_Success [SKIP]
      Condition(s) not met: "IsDomainJoinedServerAvailable"
    System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Authentication_Test.Credentials_ServerChallengesWithWindowsAuth_ClientSendsWindowsAuthHeader [SKIP]
      Condition(s) not met: "IsNtlmInstalled"
    System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_MaxResponseHeadersLength_Http3.ThresholdExceeded_ThrowsException(maxResponseHeadersLength: 1, headersLengthEstimate: 1024) [FAIL]
      System.Net.Quic.QuicException : Stream aborted by peer (268).
      Stack Trace:
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs(157,0): at System.Net.Quic.ResettableValueTaskSource.TryComplete(Exception exception, Boolean final)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs(221,0): at System.Net.Quic.ResettableValueTaskSource.TrySetException(Exception exception, Boolean final)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs(516,0): at System.Net.Quic.QuicStream.HandleEventPeerReceiveAborted(_PEER_RECEIVE_ABORTED_e__Struct& data)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs(570,0): at System.Net.Quic.QuicStream.HandleStreamEvent(QUIC_STREAM_EVENT& streamEvent)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs(601,0): at System.Net.Quic.QuicStream.NativeCallback(QUIC_HANDLE* connection, Void* context, QUIC_STREAM_EVENT* streamEvent)
        --- End of stack trace from previous location ---
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs(267,0): at System.Net.Quic.ResettableValueTaskSource.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(168,0): at System.Net.Test.Common.Http3LoopbackStream.SendFrameAsync(Int64 frameType, ReadOnlyMemory`1 framePayload)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(123,0): at System.Net.Test.Common.Http3LoopbackStream.SendHeadersFrameAsync(Nullable`1 statusCode, IEnumerable`1 headers, Boolean qpackEncodeStatus)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(264,0): at System.Net.Test.Common.Http3LoopbackStream.SendResponseHeadersAsync(Nullable`1 statusCode, IEnumerable`1 headers)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(247,0): at System.Net.Test.Common.Http3LoopbackStream.SendResponseAsync(HttpStatusCode statusCode, IList`1 headers, String content, Boolean isFinal)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackConnection.cs(241,0): at System.Net.Test.Common.Http3LoopbackConnection.HandleRequestAsync(HttpStatusCode statusCode, IList`1 headers, String content)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs(93,0): at System.Net.Test.Common.Http3LoopbackServer.HandleRequestAsync(HttpStatusCode statusCode, IList`1 headers, String content)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs(93,0): at System.Net.Test.Common.Http3LoopbackServer.HandleRequestAsync(HttpStatusCode statusCode, IList`1 headers, String content)
        /_/src/libraries/Common/tests/System/Net/Http/HttpClientHandlerTest.MaxResponseHeadersLength.cs(128,0): at System.Net.Http.Functional.Tests.HttpClientHandler_MaxResponseHeadersLength_Test.<>c__DisplayClass5_0.<<ThresholdExceeded_ThrowsException>b__1>d.MoveNext()
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(120,0): at System.Threading.Tasks.TaskTimeoutExtensions.GetRealException(Task task)
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(90,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks)
        /_/src/libraries/Common/tests/System/Net/Http/GenericLoopbackServer.cs(38,0): at System.Net.Test.Common.LoopbackServerFactory.<>c__DisplayClass5_0.<<CreateClientAndServerAsync>b__0>d.MoveNext()
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackServer.cs(111,0): at System.Net.Test.Common.Http3LoopbackServerFactory.CreateServerAsync(Func`3 funcAsync, Int32 millisecondsTimeout, GenericLoopbackOptions options)
        /_/src/libraries/Common/tests/System/Net/Http/HttpClientHandlerTest.MaxResponseHeadersLength.cs(96,0): at System.Net.Http.Functional.Tests.HttpClientHandler_MaxResponseHeadersLength_Test.ThresholdExceeded_ThrowsException(Nullable`1 maxResponseHeadersLength, Int32 headersLengthEstimate)
        --- End of stack trace from previous location ---

Cc @MihaZupan

Author: MichalStrehovsky
Assignees: -
Labels:

area-System.Net.Http, blocking-clean-ci, untriaged

Milestone: -

@MichalStrehovsky
Copy link
Member Author

@ManickaP
Copy link
Member

The error is H3_REQUEST_CANCELLED (0x010c), @MihaZupan could this be recently broken by #73907?

@karelz
Copy link
Member

karelz commented Aug 15, 2022

I don't see any failures in Kusto -- perhaps they didn't propagate yet?
Is it 100% failure for you @MichalStrehovsky?

let failedTests = (testNameSubstring : string, methodName : string, messageSubstr: string, includePR : bool, includePassedOnRerun : bool) {
cluster('engsrvprod.kusto.windows.net').database('engineeringdata').AzureDevOpsTests
    | where TestName contains testNameSubstring
    | where includePassedOnRerun or (Outcome == 'Failed')
    | extend startOfTestName = indexof_regex(TestName, @"[^.]+$")
    | extend Method = substring(TestName, startOfTestName)
    | extend Type = substring(TestName, 0, startOfTestName - 1)
    | project-away startOfTestName
    | where (methodName == '') or (Method == methodName)
    | where Message contains messageSubstr
    | distinct JobId, WorkItemId, Message, StackTrace, Method, Type, Arguments, Outcome
    | join kind=inner (cluster('engsrvprod.kusto.windows.net').database('engineeringdata').Jobs
        | where ((Branch == 'refs/heads/main') or (Branch == 'refs/heads/master') or (includePR and (Source startswith "pr/")))
        | where Type startswith "test/functional/cli/"
            and not(Properties contains "runtime-staging")
        | where Branch <> 'refs/pull/73374/merge'
        | summarize arg_max(Finished, Properties, Type, Branch, Source, Started, QueueName) by JobId
        | project-rename JobType = Type) on JobId
    | extend PropertiesJson = parse_json(Properties)
    | extend OS = replace_regex(tostring(PropertiesJson.operatingSystem), @'\((.*)\).*|([^\(].*)', @'\1\2')
    | extend Runtime = iif(PropertiesJson.runtimeFlavor == "mono", "Mono", iif(PropertiesJson.DefinitionName contains "coreclr", "CoreCLR", ""))
    | extend TargetBranch = extractjson("$.['System.PullRequest.TargetBranch']", Properties)
    | extend Architecture = PropertiesJson.architecture
    | extend Scenario = iif(isempty(PropertiesJson.scenario), "--", PropertiesJson.scenario)
    //| extend DefinitionName = PropertiesJson.DefinitionName
    | project-away PropertiesJson
};
failedTests(
    '', //testNameSubstring
    'SocketsHttpHandler_HttpClientHandler_MaxResponseHeadersLength_Http3', //methodName
    '', //messageSubstr
    true,  //includePR
    true); //includePassedOnRerun

@MichalStrehovsky
Copy link
Member Author

Is it 100% failure for you @MichalStrehovsky?

100% failure in the two pull requests I was involved in today:

https://github.com/dotnet/runtime/pull/72145/checks?check_run_id=7832255512
https://github.com/dotnet/runtime/pull/73929/checks?check_run_id=7833384695

@MihaZupan
Copy link
Member

MihaZupan commented Aug 15, 2022

This is definitely related to #73907 - I just merged H2 and H3 versions of this test yesterday.

Shouldn't be 100% as I never saw this failure on CI on that PR.

Opened #73937 that may address this

Edit: It's passing 90% of the time on reruns

@MihaZupan
Copy link
Member

MihaZupan commented Aug 15, 2022

Hit issues on the PR (#73937 (comment)).

Disabling the test for H3 until we resolve that: #73945

@MihaZupan MihaZupan added this to the 7.0.0 milestone Aug 15, 2022
@MihaZupan MihaZupan removed the untriaged New issue has not been triaged by the area owner label Aug 15, 2022
@MihaZupan MihaZupan added disabled-test The test is disabled in source code against the issue and removed blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' labels Aug 15, 2022
@ghost ghost added in-pr There is an active PR which will close this issue when it is merged and removed in-pr There is an active PR which will close this issue when it is merged labels Aug 17, 2022
@karelz karelz removed the in-pr There is an active PR which will close this issue when it is merged label Aug 23, 2022
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Aug 23, 2022
@karelz
Copy link
Member

karelz commented Aug 23, 2022

Triage: Let's try to re-enable the test in main to see if we get still hits.
@MihaZupan's attempt to reproduce in PR did not succeed so far :(

@wfurt wfurt assigned wfurt and unassigned MihaZupan Aug 23, 2022
@wfurt
Copy link
Member

wfurt commented Aug 23, 2022

I have local repro and the test still fails. I see no point of trying it in main. I will investigate.

@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Aug 23, 2022
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Aug 24, 2022
@karelz
Copy link
Member

karelz commented Aug 25, 2022

Triage: @wfurt analyzed the problem as a problem in test + we need to slightly modify what we fail in this error case (that's a product change) -- neither is important for 7.0.
Along the way @wfurt added bunch of useful logging that is not critical for 7.0.
Moving to 8.0.

@karelz karelz modified the milestones: 7.0.0, 8.0.0 Aug 25, 2022
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Aug 26, 2022
@ghost ghost locked as resolved and limited conversation to collaborators Sep 26, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-System.Net.Http disabled-test The test is disabled in source code against the issue
Projects
None yet
5 participants