Re-connecting should be part of DCP request retry

[Karol Zadora-Przylecki (karolz-ms)]

Manual backport (forward-port? 😄) of #18096 into main branch

[github-actions]

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 18121

Or

• Run remotely in PowerShell:

iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 18121"

[Copilot]

Pull request overview

This PR is a manual forward-port of PR #18096 into main. It moves the DCP API server connection establishment (EnsureKubernetesAsync) inside the retry loop of ExecuteWithRetry, so that failures caused by a missing or partially-written kubeconfig file are properly retried. This addresses timing issues where DCP hasn't finished writing its kubeconfig when Aspire first tries to read it (issues #18050 and #18041).

Changes:

• Moved EnsureKubernetesAsync from before the retry pipeline into the resiliencePipeline.ExecuteAsync lambda, allowing kubeconfig read failures to be retried by the outer resilience pipeline. Removed the now-redundant dedicated timeout pipeline wrapper inside EnsureKubernetesAsync and switched to explicit FileStream with FileShare.ReadWrite | FileShare.Delete for non-exclusive kubeconfig reads.
• Added a new test class KubernetesServiceTests with two integration tests verifying the retry behavior when the kubeconfig is initially missing or partially written, using a lightweight fake DCP API server backed by Kestrel on a random port.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
src/Aspire.Hosting/Dcp/KubernetesService.cs	Moves EnsureKubernetesAsync inside the retry loop in ExecuteWithRetry; removes the inner timeout pipeline from EnsureKubernetesAsync; opens the kubeconfig file with FileShare.ReadWrite | FileShare.Delete to avoid interfering with DCP writes.
tests/Aspire.Hosting.Tests/Dcp/KubernetesServiceTests.cs	New test file with two tests for the retry-on-kubeconfig-unavailable behavior, along with test helpers (TestFileSystemService, TestDcpApiServer).

[Karol Zadora-Przylecki (karolz-ms)]

PR Testing Report

PR Information

• PR Number: Re-connecting should be part of DCP request retry #18121
• Title: Re-connecting should be part of DCP request retry
• Head Commit: 3c195e9
• State: Draft — manual forward-port of Reconnect if necessary during DCP request execution #18096 into main
• Tested At: 2026-06-11

Artifact Version Verification

• Expected Commit: 3c195e9
• Tested Artifact: Source checkout at PR head commit (no "Dogfood this PR" CLI artifact — the PR is a draft with no comments, so there is no published CLI to dogfood)
• Installed Version: N/A — built and tested the source checkout at the PR head commit
• Status: ✅ Verified (HEAD == PR head commit)

Changes Analyzed

Files Changed

• src/Aspire.Hosting/Dcp/KubernetesService.cs — Modified (+42 / -35)
• tests/Aspire.Hosting.Tests/Dcp/KubernetesServiceTests.cs — Added (+287)

Change Categories

• Hosting core changes (DCP KubernetesService retry/connection logic)
• Test changes (new KubernetesServiceTests)
• CLI changes
• Dashboard changes
• Template changes
• Client/Component changes
• CI infrastructure changes
• VS Code extension changes

What the change does

• Moves DCP connection establishment (EnsureKubernetesAsync) inside the resilience retry loop in ExecuteWithRetry, so re-connecting (e.g. due to a missing or partially-written kubeconfig) becomes part of the request retry rather than a one-time step before the loop.
• Reads the kubeconfig via a FileStream opened with FileShare.ReadWrite | FileShare.Delete so the host does not interfere with DCP if it opens the file mid-write.
• Restructures the read/timeout resilience pipeline (drops the separate timeout-wrapping pipeline; the read pipeline now drives the file-wait + build directly).

Test Scenarios Executed

Scenario 1: Check out PR branch + build

Objective: Confirm the PR source restores and builds cleanly.
Coverage Type: Build validation
Status: ✅ Passed

Steps:

1. git fetch origin pull/18121/head and checked out head commit 3c195e9d5.
2. Ran restore.cmd to set up the local .NET SDK.
3. Built Aspire.Hosting.Tests (which builds Aspire.Hosting) via dotnet test.

Observations:

• Restore succeeded: 0 warnings, 0 errors.
• Test project + dependencies compiled with 0 warnings, 0 errors.

---

Scenario 2: Run the 2 new KubernetesServiceTests

Objective: Directly validate the fix — connection is established/re-established inside the retry loop.
Coverage Type: Happy path + boundary (missing / partially-written kubeconfig)
Status: ✅ Passed (2/2)

Steps:

1. dotnet test --project tests/Aspire.Hosting.Tests/... -- --filter-class "*.KubernetesServiceTests" --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"

Tests run:

• ExecuteWithRetry_EstablishesConnection_WhenKubeconfigInitiallyMissing ✅ — kubeconfig absent at call time; the operation waits and succeeds once DCP writes it.
• ExecuteWithRetry_EstablishesConnection_WhenKubeconfigInitiallyPartiallyWritten ✅ — kubeconfig exists but is a partial/invalid prefix; the read is retried and succeeds once the complete file is appended.

Observations:

• Both tests passed (total 2, succeeded 2, failed 0) in ~6.8s. These exercise exactly the reconnect-in-retry behavior the PR introduces.

---

Scenario 3: DCP namespace regression sweep

Objective: Ensure the KubernetesService refactor didn't regress existing DCP behavior.
Coverage Type: Regression
Status: ✅ Passed (222/222)

Steps:

1. dotnet test --project tests/Aspire.Hosting.Tests/... --no-build -- --filter-namespace "Aspire.Hosting.Tests.Dcp" --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"

Observations:

• All 222 tests in Aspire.Hosting.Tests.Dcp passed (failed 0, skipped 0) in ~31s.
• Covers DcpExecutorTests, DcpCliArgsTests, DcpHostNotificationTests, DcpLogParserTests, ConfigureDefaultDcpOptionsTests, ProcessUtilTests, ResourceSnapshotBuilderTests, and the new KubernetesServiceTests.

Notes

• The test project (Aspire.Hosting.Tests) is single-targeted to $(DefaultTargetFramework) = net8.0 (per eng/Versions.props), so net8.0 is the only configured TFM; there is no additional TFM to run. The change is framework-agnostic C#.
• No end-to-end aspire run smoke test was performed (user opted for scenarios 1–3). The reconnect-on-missing/partial-kubeconfig path is precisely covered by the new unit tests, and normal DCP connect is covered by the broader DCP suite.

Summary

Scenario	Status	Notes
1. Build PR source	✅ Passed	Restore + build: 0 warnings, 0 errors
2. New KubernetesServiceTests	✅ Passed	2/2
3. DCP regression sweep	✅ Passed	222/222

Overall Result

✅ PR VERIFIED

The new tests directly validate the reconnect-inside-retry behavior, and the full DCP test namespace passes with no regressions. Build is clean.

Recommendations

• None blocking. Optionally, a manual aspire run smoke test against a real DCP could be done before merge for extra confidence, but it is largely redundant with the unit + regression coverage for this specific change.

[github-actions]

Re-running the failed jobs in the CI workflow for this pull request because 3 jobs were identified as retry-safe transient failures in the CI run attempt.
GitHub was asked to rerun all failed jobs for that attempt, and the rerun is being tracked in the rerun attempt.
The job links below point to the failed attempt jobs that matched the retry-safe transient failure rules.

• Tests / Hosting.Keycloak / Hosting.Keycloak (windows-latest) - Failed step 'Build test project' will be retried because the job log shows a likely transient infrastructure network failure. Matched pattern: /Unable to load the service index for source https:\/\/(?:pkgs\.dev\.azure\.com\/dnceng|dnceng\.pkgs\.visualstudio\.com)\/public\/_packaging\//i.
• Tests / MySqlConnector / MySqlConnector (ubuntu-latest) - Failed step 'Build test project' will be retried because the job log shows a likely transient infrastructure network failure. Matched pattern: /Unable to load the service index for source https:\/\/(?:pkgs\.dev\.azure\.com\/dnceng|dnceng\.pkgs\.visualstudio\.com)\/public\/_packaging\//i.
• Tests / Hosting-1 / Hosting-1 (windows-latest) - Job-level runner or infrastructure failure matched the transient allowlist.

[github-actions]

❌ CLI E2E Tests failed — 114 passed, 1 failed, 2 unknown (commit 3c195e9)

❌ Failed Tests

-	Test	Detail
❌	AllPublishMethodsBuildDockerImages	Recording · Job · CLI logs

View all recordings

-	Test	Detail
✅	AddPackageInteractiveWhileAppHostRunningDetached	Recording · Job · CLI logs
✅	AddPackageWhileAppHostRunningDetached	Recording · Job · CLI logs
✅	AgentCommands_AllHelpOutputs_AreCorrect	Recording · Job · CLI logs
✅	AgentInitCommand_DefaultSelection_InstallsDefaultSkills	Recording · Job · CLI logs
✅	AgentInitCommand_MigratesDeprecatedConfig	Recording · Job · CLI logs
✅	AgentInit_NonInteractive_BundleOnlySkillsNotInCatalog	Recording · Job · CLI logs
✅	AgentMcpListResources_ExcludesResourceMarkedWithExcludeFromMcp	Recording · Job · CLI logs
✅	AgentMcpListStructuredLogsReturnsLogsFromStarterApp	Recording · Job · CLI logs
✅	AgentMcpListStructuredLogsReturnsLogsFromStarterApp_DevLocalhost	Recording · Job · CLI logs
✅	AgentMcpListStructuredLogsReturnsLogsFromStarterApp_Isolated	Recording · Job · CLI logs
❌	AllPublishMethodsBuildDockerImages	Recording · Job · CLI logs
✅	AspireAddAndStartWorkAgainstLegacyAppHostTs	Recording · Job · CLI logs
✅	AspireAddPackageVersionToDirectoryPackagesProps	Recording · Job · CLI logs
✅	AspireInitSingleFileAppHostRunsViaDotnetRunAppHost	Recording · Job · CLI logs
✅	AspireInit_ExistingAppHostDir_RecreatesNuGetConfigKeepsFiles	Recording · Job · CLI logs
❔	AspireInit_SolutionFile_BuildsAgainstChannelHive	Recording · Job · CLI logs
✅	AspireStartUpdatesStaleTypeScriptAppHostPath	Recording · Job · CLI logs
✅	AspireUpdateRemovesAppHostPackageVersionFromDirectoryPackagesProps	Recording · Job · CLI logs
✅	AspireUpdateRemovesOrphanAppHostPackageVersionWhenSdkAlreadyCurrent	Recording · Job · CLI logs
✅	Banner_DisplayedOnFirstRun	Recording · Job · CLI logs
✅	Banner_DisplayedWithExplicitFlag	Recording · Job · CLI logs
✅	Banner_NotDisplayedWithNoLogoFlag	Recording · Job · CLI logs
✅	CertificatesClean_RemovesCertificates	Recording · Job · CLI logs
✅	CertificatesTrust_WithNoCert_CreatesAndTrustsCertificate	Recording · Job · CLI logs
✅	CertificatesTrust_WithUntrustedCert_TrustsCertificate	Recording · Job · CLI logs
✅	ConfigSetGet_CreatesNestedJsonFormat	Recording · Job · CLI logs
✅	CreateAndRunAspireStarterProject	Recording · Job · CLI logs
✅	CreateAndRunAspireStarterProjectWithBundle	Recording · Job · CLI logs
✅	CreateAndRunEmptyAppHostProject	Recording · Job · CLI logs
✅	CreateAndRunJavaEmptyAppHostProject	Recording · Job · CLI logs
✅	CreateAndRunJsReactProject	Recording · Job · CLI logs
✅	CreateAndRunPolyglotAppHostWithDevLocalhostUrls	Recording · Job · CLI logs
✅	CreateAndRunPythonReactProject	Recording · Job · CLI logs
✅	CreateAndRunTypeScriptEmptyAppHostProject	Recording · Job · CLI logs
✅	CreateAndRunTypeScriptStarterProject	Recording · Job · CLI logs
✅	CreateJavaAppHostWithViteApp	Recording · Job · CLI logs
✅	CreateTypeScriptAppHostWithViteApp_UsesConfiguredToolchain	Recording · Job · CLI logs
✅	DashboardRunWithAgentMcpListTracesReturnsNoTraces	Recording · Job · CLI logs
✅	DashboardRunWithAgentMcpListTracesReturnsNoTraces_DevLocalhost	Recording · Job · CLI logs
✅	DashboardRunWithOtelTracesReturnsNoTraces	Recording · Job · CLI logs
✅	DashboardRunWithOtelTracesReturnsNoTraces_DevLocalhost	Recording · Job · CLI logs
✅	DeployK8sBasicApiService	Recording · Job · CLI logs
✅	DeployK8sWithExternalHelmChart	Recording · Job · CLI logs
✅	DeployK8sWithGarnet	Recording · Job · CLI logs
✅	DeployK8sWithMongoDB	Recording · Job · CLI logs
✅	DeployK8sWithMySql	Recording · Job · CLI logs
✅	DeployK8sWithPostgres	Recording · Job · CLI logs
✅	DeployK8sWithRabbitMQ	Recording · Job · CLI logs
✅	DeployK8sWithRedis	Recording · Job · CLI logs
✅	DeployK8sWithSqlServer	Recording · Job · CLI logs
✅	DeployK8sWithValkey	Recording · Job · CLI logs
✅	DeployTypeScriptAppToKubernetes	Recording · Job · CLI logs
✅	DescribeCommandResolvesReplicaNames	Recording · Job · CLI logs
✅	DescribeCommandShowsRunningResources	Recording · Job · CLI logs
✅	DetachFormatJsonProducesValidJson	Recording · Job · CLI logs
✅	DetachFormatJsonProducesValidJsonWhenRestartingExistingInstance	Recording · Job · CLI logs
✅	DoPublishAndDeployListStepsWork	Recording · Job · CLI logs
✅	DocsCommand_RendersInteractiveMarkdownFromLocalSource	Recording · Job · CLI logs
✅	DoctorCommand_DetectsDeprecatedAgentConfig	Recording · Job · CLI logs
✅	DoctorCommand_TypeScriptAppHostReportsMissingConfiguredToolchain	Recording · Job · CLI logs
✅	DoctorCommand_WithSslCertDir_ShowsTrusted	Recording · Job · CLI logs
✅	DoctorCommand_WithoutSslCertDir_ShowsPartiallyTrusted	Recording · Job · CLI logs
✅	DotNetRunFileBasedAppHostUsesAspireCliBundle	Recording · Job · CLI logs
✅	DotNetRunProjectAppHostUsesAspireCliBundle	Recording · Job · CLI logs
✅	GatewayWithoutExternalEndpoint_FailsPublishWithGuidance	Recording · Job · CLI logs
✅	GeneratedAspireDevScript_StartsWatchMode_WithConfiguredToolchain	Recording · Job · CLI logs
✅	GlobalMigration_HandlesCommentsAndTrailingCommas	Recording · Job · CLI logs
✅	GlobalMigration_HandlesMalformedLegacyJson	Recording · Job · CLI logs
✅	GlobalMigration_PreservesAllValueTypes	Recording · Job · CLI logs
✅	GlobalMigration_SkipsWhenNewConfigExists	Recording · Job · CLI logs
✅	GlobalSettings_MigratedFromLegacyFormat	Recording · Job · CLI logs
✅	IngressWithoutExternalEndpoint_FailsPublishWithGuidance	Recording · Job · CLI logs
✅	InitTypeScriptAppHost_AugmentsExistingViteRepoInWorkspaceSubdirectory	Recording · Job · CLI logs
✅	InteractiveCSharpInitCreatesExpectedFiles	Recording · Job · CLI logs
✅	InvalidAppHostPathWithComments_IsHealedOnRun	Recording · Job · CLI logs
✅	JavaScriptHostingApisRunFromTypeScriptAppHost	Recording · Job · CLI logs
✅	LatestCliCanStartStableChannelAppHost	Recording · Job · CLI logs
✅	LatestCliCanStartStableChannelTypeScriptAppHost	Recording · Job · CLI logs
✅	LegacySettingsMigration_AdjustsRelativeAppHostPath	Recording · Job · CLI logs
✅	LogsCommandShowsResourceLogs	Recording · Job · CLI logs
✅	OtelLogsReturnsStructuredLogsFromStarterApp	Recording · Job · CLI logs
✅	OtelLogsReturnsStructuredLogsFromStarterAppIsolated	Recording · Job · CLI logs
✅	ProcessCommandCallbackReceivesCliArguments	Recording · Job · CLI logs
✅	PsCommandListsRunningAppHost	Recording · Job · CLI logs
✅	PsFormatJsonOutputsOnlyJsonToStdout	Recording · Job · CLI logs
✅	PublishJavaScriptPatternsGeneratesExpectedDockerComposeArtifacts	Recording · Job · CLI logs
✅	PublishWithConfigureEnvFileUpdatesEnvOutput	Recording · Job · CLI logs
✅	PublishWithDockerComposeServiceCallbackSucceeds	Recording · Job · CLI logs
✅	PublishWithoutOutputPathUsesAppHostDirectoryDefault	Recording · Job · CLI logs
✅	ResourceCommand_FailedExec_ShowsLogPathAndLogHasEntries	Recording · Job · CLI logs
✅	ResourceCommand_SetAndDeleteParameterUpdatesDescribeOutput	Recording · Job · CLI logs
✅	RestoreGeneratesSdkFiles	Recording · Job · CLI logs
✅	RestoreGeneratesSdkFiles_WithConfiguredToolchain	Recording · Job · CLI logs
✅	RestoreRefreshesGeneratedSdkAfterAddingIntegration	Recording · Job · CLI logs
✅	RestoreSupportsConfigOnlyHelperPackageAndCrossPackageTypes	Recording · Job · CLI logs
✅	RunFromParentDirectory_UsesExistingConfigNearAppHost	Recording · Job · CLI logs
✅	RunReportsSyntaxErrorsForDotNetAppHost	Recording · Job · CLI logs
✅	RunReportsSyntaxErrorsForTypeScriptAppHost	Recording · Job · CLI logs
✅	SecretCrudOnDotNetAppHost	Recording · Job · CLI logs
✅	SecretCrudOnTypeScriptAppHost	Recording · Job · CLI logs
✅	StagingChannel_ConfigureAndVerifySettings_ThenSwitchChannels	Recording · Job · CLI logs
✅	StartAndWaitForTypeScriptSqlServerAppHostWithNativeAssets	Recording · Job · CLI logs
✅	StartReportsSyntaxErrorsForDotNetAppHost	Recording · Job · CLI logs
✅	StartReportsSyntaxErrorsForTypeScriptAppHost	Recording · Job · CLI logs
✅	StopAllAppHostsFromAppHostDirectory	Recording · Job · CLI logs
✅	StopJavaPolyglotAppHostUsingApphostDirectory	Recording · Job · CLI logs
✅	StopNonInteractiveSingleAppHost	Recording · Job · CLI logs
✅	StopTypeScriptPolyglotAppHostUsingApphostDirectory	Recording · Job · CLI logs
✅	StopWithNoRunningAppHostExitsSuccessfully	Recording · Job · CLI logs
✅	TerminalAttachFrontend_ShowsViteHelpAndDetaches	Recording · Job · CLI logs
✅	TypeScriptAppHostRunDoesNotDeadlockWhenLazyOptionsInvokeAsyncCallback	Recording · Job · CLI logs
✅	TypeScriptAppHostWithVite_AllowsDifferentGuestPkgManager	Recording · Job · CLI logs
✅	UnAwaitedChainsCompileWithAutoResolvePromises	Recording · Job · CLI logs
❔	UpdateToStable_CSharpEmptyAppHost_KeepsConfigChannel	Recording · Job · CLI logs
✅	UpdateToStable_CSharpSingleFileInit_KeepsConfigChannel	Recording · Job · CLI logs
✅	UpdateToStable_TypeScriptSingleFileInit_KeepsConfigChannel	Recording · Job · CLI logs
✅	UpdateToStable_TypeScript_PreviewsStablePkgsAndKeepsChannel	Recording · Job · CLI logs

---

_{📹 Recordings uploaded automatically from CI run #27367459470}

[aspire-repo-bot]

✅ No documentation update needed.

docs_optional → bug_fix_restores_documented_behavior

No documentation update is needed for this PR.

Triggered signals: none (signal_count == 0)

Allowlist justification: The two changed files are:

• src/Aspire.Hosting/Dcp/KubernetesService.cs — moves EnsureKubernetesAsync inside the resilience retry loop so that a missing or partially-written DCP kubeconfig is retried automatically; removes an inner dedicated timeout pipeline; opens the kubeconfig with FileShare.ReadWrite | FileShare.Delete to avoid interfering with concurrent DCP writes.
• tests/Aspire.Hosting.Tests/Dcp/KubernetesServiceTests.cs — new test file covering the two retry scenarios.

No new public types, methods, options, configuration keys, CLI flags, resource strings, container image tags, or user-visible default values were introduced. The PR fixes issues #18050 and #18041 where Aspire could fail to connect to DCP if it tried to read the kubeconfig before DCP finished writing it. The intended behavior — that Aspire connects to DCP successfully on startup — was already the documented, expected behavior; this change makes the implementation match that expectation more reliably under timing pressure.