CI error: System.Net.Quic.QuicException: The connection timed out from inactivity

[steveharter]

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=640353
Build error leg or test failing: System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_MaxResponseHeadersLength_Http3.LargeSingleHeader_ThrowsException

Error Message

Multiple System.Net.Quic.QuicException : The connection timed out from inactivity. in System.Net.Quic tests on "smaller" platforms, e.g. Alpine3.15 arm 32

 System.Net.Quic.QuicException : The connection timed out from inactivity.
      Stack Trace:
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicConnection.cs(547,0): at System.Net.Quic.QuicConnection.HandleConnectionEvent(QUIC_CONNECTION_EVENT& connectionEvent)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicConnection.cs(581,0): at System.Net.Quic.QuicConnection.NativeCallback(QUIC_HANDLE* connection, Void* context, QUIC_CONNECTION_EVENT* connectionEvent)
        --- End of stack trace from previous location ---
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ValueTaskSource.cs(176,0): at System.Net.Quic.ValueTaskSource.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicConnection.cs(324,0): at System.Net.Quic.QuicConnection.FinishConnectAsync(QuicClientConnectionOptions options, CancellationToken cancellationToken)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicConnection.cs(72,0): at System.Net.Quic.QuicConnection.<ConnectAsync>g__StartConnectAsync|2_0(QuicClientConnectionOptions options, CancellationToken cancellationToken)
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicConnection.cs(77,0): at System.Net.Quic.QuicConnection.<ConnectAsync>g__StartConnectAsync|2_0(QuicClientConnectionOptions options, CancellationToken cancellationToken)
        /_/src/libraries/System.Net.Quic/tests/FunctionalTests/QuicTestBase.cs(214,0): at System.Net.Quic.Tests.QuicTestBase.CreateConnectedQuicConnection(QuicClientConnectionOptions clientOptions, QuicListener listener)
        /_/src/libraries/System.Net.Quic/tests/FunctionalTests/QuicTestBase.cs(245,0): at System.Net.Quic.Tests.QuicTestBase.CreateConnectedQuicConnection(QuicClientConnectionOptions clientOptions, QuicListener listener)
        /_/src/libraries/System.Net.Quic/tests/FunctionalTests/QuicConnectionTests.cs(315,0): at System.Net.Quic.Tests.QuicConnectionTests.AcceptStreamAsync_ConnectionDisposed_Throws()

{
  "ErrorPattern": "System.Net.Quic.QuicException ?: The connection timed out from inactivity",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=640353
Error message validated: [System.Net.Quic.QuicException ?: The connection timed out from inactivity]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 4/16/2024 11:07:05 AM UTC

Report

Build	Definition	Test	Pull Request
745864	dotnet/runtime	System.Net.Quic.Tests.MsQuicTests.WriteTests	#104680
743164	dotnet/runtime	System.Net.Quic.Tests.QuicStreamTests.WriteAsync_LocalAbort_Throws	#101624
742676	dotnet/runtime	System.Net.Quic.Functional.Tests.WorkItemExecution	#104856
742110	dotnet/runtime	System.Net.Http.Functional.Tests.SocksProxyTest_Http2.TestLoopbackAsync
741273	dotnet/runtime	System.Net.Quic.Tests.QuicStreamTests.WriteAsync_LocalAbort_Throws	#101624
740929	dotnet/runtime	System.Net.Quic.Tests.QuicStreamConformanceTests.ReadWrite_Success	#101624
739937	dotnet/runtime	System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_AutoRedirect.GetAsync_AllowAutoRedirectTrue_RetainsOriginalFragmentIfAppropriate
738918	dotnet/runtime	System.Net.Quic.Tests.QuicStreamTests.WriteAsync_LocalAbort_Throws	#101624
737434	dotnet/runtime	System.Net.Quic.Tests.QuicStreamTests.WriteAsync_LocalAbort_Throws	#101624
736217	dotnet/runtime	System.Net.Quic.Tests.MsQuicTests.WriteTests	#104684
735197	dotnet/runtime	System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Http11_Cancellation_Test.GetAsync_CancelDuringResponseHeadersReceived_TaskCanceledQuickly
734813	dotnet/runtime	System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http3.ServerClosesOutboundControlStream_ClientClosesConnection	#104630
713739	dotnet/runtime	System.Net.Quic.Tests.QuicStreamConformanceTests.ReadWrite_MessagesSmallerThanReadBuffer_Success	#103646

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	7	13

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=398498
Build error leg or test failing: System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_MaxResponseHeadersLength_Http3.LargeSingleHeader_ThrowsException
Pull request: #91290

Error Message

  Starting:    System.Net.Http.Functional.Tests (parallel test collections = on, max threads = 2)
    System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientEKUTest.HttpClient_ClientEKUServerAuth_Fails [SKIP]
      Condition(s) not met: "CanTestCertificates"
    System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientEKUTest.HttpClient_NoEKUServerAuth_Ok [SKIP]
      Condition(s) not met: "CanTestCertificates"
    System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientEKUTest.HttpClient_NoEKUClientAuth_Ok [SKIP]
      Condition(s) not met: "CanTestCertificates"
    System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientEKUTest.HttpClient_ServerEKUClientAuth_Fails [SKIP]
      Condition(s) not met: "CanTestCertificates"
    System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClient_SelectedSites_Test.RetrieveSite_Succeeds [SKIP]
      Condition(s) not met: "IsSelectedSitesTestEnabled"
    System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClient_SelectedSites_Test.RetrieveSite_Debug_Helper [SKIP]
      Condition(s) not met: "IsSelectedSitesTestEnabled"
    System.Net.Http.Functional.Tests.SyncHttpHandler_HttpClientHandler_Authentication_Test.Credentials_ServerUsesWindowsAuthentication_Success [SKIP]
      Condition(s) not met: "IsWindowsServerAvailable"
    System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_MaxResponseHeadersLength_Http3.LargeSingleHeader_ThrowsException(maxResponseHeadersLength: 1) [FAIL]
      System.AggregateException : One or more errors occurred. (Assert.Contains() Failure
      Not found: 1024
      In value:  System.Net.Http.HttpRequestException: An error occurred while sending the request.
                  ---> System.Net.Quic.QuicException: The connection timed out from inactivity.
                    at System.Net.Quic.ResettableValueTaskSource.TryComplete(Exception exception, Boolean final) in /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs:line 169
                    at System.Net.Quic.ResettableValueTaskSource.TrySetException(Exception exception, Boolean final) in /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs:line 233
                    at System.Net.Quic.QuicStream.HandleEventShutdownComplete(_SHUTDOWN_COMPLETE_e__Struct& data) in /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs:line 589
                    at System.Net.Quic.QuicStream.HandleStreamEvent(QUIC_STREAM_EVENT& streamEvent) in /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs:line 611
                    at System.Net.Quic.QuicStream.NativeCallback(QUIC_HANDLE* connection, Void* context, QUIC_STREAM_EVENT* streamEvent) in /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs:line 640
                 --- End of stack trace from previous location ---
                    at System.Net.Quic.ResettableValueTaskSource.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) in /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ResettableValueTaskSource.cs:line 247
                    at System.Net.Quic.QuicStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs:line 305
                    at System.Net.Http.Http3RequestStream.ReadFrameEnvelopeAsync(CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/Http3RequestStream.cs:line 815
                    at System.Net.Http.Http3RequestStream.ReadResponseAsync(CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/Http3RequestStream.cs:line 339
                    at System.Net.Http.Http3RequestStream.SendAsync(CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/Http3RequestStream.cs:line 206
                    --- End of inner exception stack trace ---
                    at System.Net.Http.Http3RequestStream.SendAsync(CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/Http3RequestStream.cs:line 310
                    at System.Net.Http.Http3RequestStream.SendAsync(CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/Http3RequestStream.cs:line 324
                    at System.Net.Http.Http3Connection.SendAsync(HttpRequestMessage request, Int64 queueStartingTimestamp, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/Http3Connection.cs:line 234
                    at System.Net.Http.Http3Connection.SendAsync(HttpRequestMessage request, Int64 queueStartingTimestamp, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/Http3Connection.cs:line 248
                    at System.Net.Http.HttpConnectionPool.TrySendUsingHttp3Async(HttpRequestMessage request, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 999
                    at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 1053
                    at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/RedirectHandler.cs:line 30
                    at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 530
                    at Xunit.Assert.RecordExceptionAsync(Func`1 testCode) in /_/src/Microsoft.DotNet.XUnitAssert/src/Record.cs:line 113) (The connection timed out from inactivity.)

{
  "ErrorMessage": "System.Net.Quic.QuicException: The connection timed out from inactivity.",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Author:	steveharter
Assignees:	-
Labels:	area-System.Net.Http, blocking-clean-ci, Known Build Error
Milestone:	-

[liveans]

This test was activated for the #74896 issue. The failures are different but they're coming from the same test, I'm investigating that.