Skip to content

Fix flaky test: use resilient HttpClient in YarpFunctionalTests#14920

Merged
radical merged 1 commit intomainfrom
fix-flaky-yarp-test
Mar 3, 2026
Merged

Fix flaky test: use resilient HttpClient in YarpFunctionalTests#14920
radical merged 1 commit intomainfrom
fix-flaky-yarp-test

Conversation

@radical
Copy link
Member

@radical radical commented Mar 3, 2026

Flaky Test Fix

Test

Root Cause

The test created a bare new HttpClient() with no retry logic. After both YARP and backend containers report healthy, Docker inter-container networking may not be fully established — YARP returns 502 BadGateway because it can't yet reach the backend container.

Fix

Replace new HttpClient() with app.CreateHttpClient(), which uses the DI container's IHttpClientFactory with AddStandardResilienceHandler() already configured by TestDistributedApplicationBuilder, providing automatic retries for transient failures.

Verification

Phase Run Config Result
Reproduction (pre-fix) link 5 runners × 10 iters × ubuntu-latest 11/50 failed on ubuntu-latest
Verification (post-fix) link 5 runners × 10 iters × ubuntu-latest All 50 passed on ubuntu-latest

Local runs:

  • Pre-fix: 10 on macOS — all passed (contention-sensitive, doesn't reproduce locally)
  • Post-fix: build verified, no regressions

Notes

  • [QuarantinedTest] attribute kept — unquarantining will happen separately after stability is confirmed in quarantine CI

Note: This PR intentionally does not close #9344. The test will remain quarantined until a separate unquarantine process confirms it has been stable (zero failures) for a sufficient period. Once stability is confirmed, the test will be unquarantined and the issue will be closed.


This fix was generated using the fix-flaky-test skill.

Copilot AI review requested due to automatic review settings March 3, 2026 22:12
@github-actions
Copy link
Contributor

github-actions bot commented Mar 3, 2026

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/dotnet/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 14920

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/dotnet/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 14920"

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses flakiness in YarpFunctionalTests.VerifyYarpResourceExtensionsConfig by switching the test’s HTTP client creation to use the test host’s DI-configured IHttpClientFactory (which includes standard resilience/retry behavior), reducing transient 502/connection timing failures when container networking isn’t fully ready.

Changes:

  • Replace a manually constructed HttpClient with app.CreateHttpClient(...) in YarpFunctionalTests.
  • Update the fix-flaky-test skill documentation to enforce a two-phase CI investigation workflow (reproduce first, then fix/verify), and to improve CI result validation guidance (including zero-test runs).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
tests/Aspire.Hosting.Yarp.Tests/YarpFunctionalTests.cs Uses DistributedApplication.CreateHttpClient so tests benefit from configured resilience/retries.
.github/skills/fix-flaky-test/SKILL.md Strengthens guidance for reproducible flaky-test investigations and accurate CI result interpretation.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 3, 2026

🎬 CLI E2E Test Recordings

The following terminal recordings are available for commit 6b79e95:

Test Recording
AddPackageInteractiveWhileAppHostRunningDetached ▶️ View Recording
AddPackageWhileAppHostRunningDetached ▶️ View Recording
AgentCommands_AllHelpOutputs_AreCorrect ▶️ View Recording
AgentInitCommand_MigratesDeprecatedConfig ▶️ View Recording
AgentInitCommand_WithMalformedMcpJson_ShowsErrorAndExitsNonZero ▶️ View Recording
AspireUpdateRemovesAppHostPackageVersionFromDirectoryPackagesProps ▶️ View Recording
Banner_DisplayedOnFirstRun ▶️ View Recording
Banner_DisplayedWithExplicitFlag ▶️ View Recording
CreateAndDeployToDockerCompose ▶️ View Recording
CreateAndDeployToDockerComposeInteractive ▶️ View Recording
CreateAndPublishToKubernetes ▶️ View Recording
CreateAndRunAspireStarterProject ▶️ View Recording
CreateAndRunAspireStarterProjectWithBundle ▶️ View Recording
CreateAndRunJsReactProject ▶️ View Recording
CreateAndRunPythonReactProject ▶️ View Recording
CreateAndRunTypeScriptStarterProject ▶️ View Recording
CreateEmptyAppHostProject ▶️ View Recording
CreateStartAndStopAspireProject ▶️ View Recording
CreateStartWaitAndStopAspireProject ▶️ View Recording
CreateTypeScriptAppHostWithViteApp ▶️ View Recording
DescribeCommandResolvesReplicaNames ▶️ View Recording
DescribeCommandShowsRunningResources ▶️ View Recording
DetachFormatJsonProducesValidJson ▶️ View Recording
DoctorCommand_DetectsDeprecatedAgentConfig ▶️ View Recording
DoctorCommand_WithSslCertDir_ShowsTrusted ▶️ View Recording
DoctorCommand_WithoutSslCertDir_ShowsPartiallyTrusted ▶️ View Recording
LogsCommandShowsResourceLogs ▶️ View Recording
PsCommandListsRunningAppHost ▶️ View Recording
PsFormatJsonOutputsOnlyJsonToStdout ▶️ View Recording
SecretCrudOnDotNetAppHost ▶️ View Recording
SecretCrudOnTypeScriptAppHost ▶️ View Recording
StagingChannel_ConfigureAndVerifySettings_ThenSwitchChannels ▶️ View Recording
StopAllAppHostsFromAppHostDirectory ▶️ View Recording
StopAllAppHostsFromUnrelatedDirectory ▶️ View Recording
StopNonInteractiveMultipleAppHostsShowsError ▶️ View Recording
StopNonInteractiveSingleAppHost ▶️ View Recording
StopWithNoRunningAppHostExitsSuccessfully ▶️ View Recording

📹 Recordings uploaded automatically from CI run #22645403852

@radical radical force-pushed the fix-flaky-yarp-test branch from 98ec616 to d7d398d Compare March 3, 2026 22:19
Replace bare 'new HttpClient()' with 'app.CreateHttpClient()' which uses
IHttpClientFactory with AddStandardResilienceHandler() already configured
by TestDistributedApplicationBuilder. This adds automatic retries for
transient failures (502 BadGateway) caused by Docker inter-container
networking not being fully established when the test makes its HTTP request.

Fixes #9344

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@radical radical enabled auto-merge (squash) March 3, 2026 22:33
@radical radical merged commit bd9fc4f into main Mar 3, 2026
384 checks passed
@radical radical deleted the fix-flaky-yarp-test branch March 3, 2026 23:03
@dotnet-policy-service dotnet-policy-service bot added this to the 13.3 milestone Mar 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky test: Aspire.Hosting.Yarp.Tests.YarpFunctionalTests.VerifyYarpResourceExtensionsConfig

3 participants