Skip to content

Report copilot-cli specifically & update tests to work in test explorer#18094

Merged
DamianEdwards merged 2 commits into
mainfrom
damianedwards/copilot-telemetry-tweak
Jun 10, 2026
Merged

Report copilot-cli specifically & update tests to work in test explorer#18094
DamianEdwards merged 2 commits into
mainfrom
damianedwards/copilot-telemetry-tweak

Conversation

@DamianEdwards

Copy link
Copy Markdown
Member

Report copilot-cli specifically & update tests to work in test explorer

Reports detection of copilot CLI as copilot-cli to be more specific than just copilot and addresses PR comment https://github.com/microsoft/aspire/pull/18065/changes#r3390821123

Checklist

  • Is this feature complete?
    • Yes. Ready to ship.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
  • Did you add public API?
    • No
  • Does the change make any security assumptions or guarantees?
    • No

Copilot AI review requested due to automatic review settings June 10, 2026 19:03
@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 18094

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 18094"

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR makes two focused changes: (1) renames the detected Copilot coding agent telemetry value from "copilot" to "copilot-cli" for specificity (removing COPILOT_AGENT from detection rules since it's set by the broader coding agent, not the CLI specifically), and (2) refactors the CodingAgentTelemetryTestCases from Dictionary<string, string?> parameters to (string, string?) tuples so tests serialize correctly and appear as individual entries in the test explorer.

Changes:

  • Renames the copilot agent detection name to "copilot-cli" and removes COPILOT_AGENT from its detection variable list in CodingAgentDetector.cs
  • Converts test data from Dictionary-based multi-entry parameters to single-tuple parameters for xUnit test explorer compatibility
  • Removes multi-agent combination test cases (these can't be expressed with single-tuple parameterization)

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
src/Aspire.Cli/Telemetry/CodingAgentDetector.cs Renames agent from "copilot" to "copilot-cli", removes COPILOT_AGENT env var, reorders COPILOT_CLI to front
tests/Aspire.Cli.Tests/Telemetry/AspireCliTelemetryTests.cs Refactors test method to accept tuple instead of Dictionary; updates expected values to "copilot-cli"; adds COPILOT_AGENTnull test; removes multi-agent combination test cases

Comment thread tests/Aspire.Cli.Tests/Telemetry/AspireCliTelemetryTests.cs Outdated
… agents; update tests for new detection logic
@DamianEdwards DamianEdwards enabled auto-merge (squash) June 10, 2026 19:45
@DamianEdwards DamianEdwards merged commit 7eba1eb into main Jun 10, 2026
651 of 665 checks passed
@DamianEdwards DamianEdwards deleted the damianedwards/copilot-telemetry-tweak branch June 10, 2026 21:26
@microsoft-github-policy-service microsoft-github-policy-service Bot added this to the 13.5 milestone Jun 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

CLI E2E Tests failed — 112 passed, 1 failed, 2 unknown (commit 397e81f)

❌ Failed Tests

- Test Detail
DeployK8sWithValkey Recording · Job · CLI logs
View all recordings
- Test Detail
AddPackageInteractiveWhileAppHostRunningDetached Recording · Job · CLI logs
AddPackageWhileAppHostRunningDetached Recording · Job · CLI logs
AgentCommands_AllHelpOutputs_AreCorrect Recording · Job · CLI logs
AgentInitCommand_DefaultSelection_InstallsDefaultSkills Recording · Job · CLI logs
AgentInitCommand_MigratesDeprecatedConfig Recording · Job · CLI logs
AgentInit_NonInteractive_BundleOnlySkillsNotInCatalog Recording · Job · CLI logs
AgentMcpListStructuredLogsReturnsLogsFromStarterApp Recording · Job · CLI logs
AgentMcpListStructuredLogsReturnsLogsFromStarterApp_DevLocalhost Recording · Job · CLI logs
AgentMcpListStructuredLogsReturnsLogsFromStarterApp_Isolated Recording · Job · CLI logs
AllPublishMethodsBuildDockerImages Recording · Job · CLI logs
AspireAddAndStartWorkAgainstLegacyAppHostTs Recording · Job · CLI logs
AspireAddPackageVersionToDirectoryPackagesProps Recording · Job · CLI logs
AspireInitSingleFileAppHostRunsViaDotnetRunAppHost Recording · Job · CLI logs
AspireInit_ExistingAppHostDir_RecreatesNuGetConfigKeepsFiles Recording · Job · CLI logs
AspireInit_SolutionFile_BuildsAgainstChannelHive Recording · Job · CLI logs
AspireStartUpdatesStaleTypeScriptAppHostPath Recording · Job · CLI logs
AspireUpdateRemovesAppHostPackageVersionFromDirectoryPackagesProps Recording · Job · CLI logs
AspireUpdateRemovesOrphanAppHostPackageVersionWhenSdkAlreadyCurrent Recording · Job · CLI logs
Banner_DisplayedOnFirstRun Recording · Job · CLI logs
Banner_DisplayedWithExplicitFlag Recording · Job · CLI logs
Banner_NotDisplayedWithNoLogoFlag Recording · Job · CLI logs
CertificatesClean_RemovesCertificates Recording · Job · CLI logs
CertificatesTrust_WithNoCert_CreatesAndTrustsCertificate Recording · Job · CLI logs
CertificatesTrust_WithUntrustedCert_TrustsCertificate Recording · Job · CLI logs
ConfigSetGet_CreatesNestedJsonFormat Recording · Job · CLI logs
CreateAndRunAspireStarterProject Recording · Job · CLI logs
CreateAndRunAspireStarterProjectWithBundle Recording · Job · CLI logs
CreateAndRunEmptyAppHostProject Recording · Job · CLI logs
CreateAndRunJavaEmptyAppHostProject Recording · Job · CLI logs
CreateAndRunJsReactProject Recording · Job · CLI logs
CreateAndRunPolyglotAppHostWithDevLocalhostUrls Recording · Job · CLI logs
CreateAndRunPythonReactProject Recording · Job · CLI logs
CreateAndRunTypeScriptEmptyAppHostProject Recording · Job · CLI logs
CreateAndRunTypeScriptStarterProject Recording · Job · CLI logs
CreateJavaAppHostWithViteApp Recording · Job · CLI logs
CreateTypeScriptAppHostWithViteApp_UsesConfiguredToolchain Recording · Job · CLI logs
DashboardRunWithAgentMcpListTracesReturnsNoTraces Recording · Job · CLI logs
DashboardRunWithAgentMcpListTracesReturnsNoTraces_DevLocalhost Recording · Job · CLI logs
DashboardRunWithOtelTracesReturnsNoTraces Recording · Job · CLI logs
DashboardRunWithOtelTracesReturnsNoTraces_DevLocalhost Recording · Job · CLI logs
DeployK8sBasicApiService Recording · Job · CLI logs
DeployK8sWithExternalHelmChart Recording · Job · CLI logs
DeployK8sWithGarnet Recording · Job · CLI logs
DeployK8sWithMongoDB Recording · Job · CLI logs
DeployK8sWithMySql Recording · Job · CLI logs
DeployK8sWithPostgres Recording · Job · CLI logs
DeployK8sWithRabbitMQ Recording · Job · CLI logs
DeployK8sWithRedis Recording · Job · CLI logs
DeployK8sWithSqlServer Recording · Job · CLI logs
DeployK8sWithValkey Recording · Job · CLI logs
DeployTypeScriptAppToKubernetes Recording · Job · CLI logs
DescribeCommandResolvesReplicaNames Recording · Job · CLI logs
DescribeCommandShowsRunningResources Recording · Job · CLI logs
DetachFormatJsonProducesValidJson Recording · Job · CLI logs
DetachFormatJsonProducesValidJsonWhenRestartingExistingInstance Recording · Job · CLI logs
DoPublishAndDeployListStepsWork Recording · Job · CLI logs
DocsCommand_RendersInteractiveMarkdownFromLocalSource Recording · Job · CLI logs
DoctorCommand_DetectsDeprecatedAgentConfig Recording · Job · CLI logs
DoctorCommand_TypeScriptAppHostReportsMissingConfiguredToolchain Recording · Job · CLI logs
DoctorCommand_WithSslCertDir_ShowsTrusted Recording · Job · CLI logs
DoctorCommand_WithoutSslCertDir_ShowsPartiallyTrusted Recording · Job · CLI logs
DotNetRunFileBasedAppHostUsesAspireCliBundle Recording · Job · CLI logs
DotNetRunProjectAppHostUsesAspireCliBundle Recording · Job · CLI logs
GatewayWithoutExternalEndpoint_FailsPublishWithGuidance Recording · Job · CLI logs
GeneratedAspireDevScript_StartsWatchMode_WithConfiguredToolchain Recording · Job · CLI logs
GlobalMigration_HandlesCommentsAndTrailingCommas Recording · Job · CLI logs
GlobalMigration_HandlesMalformedLegacyJson Recording · Job · CLI logs
GlobalMigration_PreservesAllValueTypes Recording · Job · CLI logs
GlobalMigration_SkipsWhenNewConfigExists Recording · Job · CLI logs
GlobalSettings_MigratedFromLegacyFormat Recording · Job · CLI logs
IngressWithoutExternalEndpoint_FailsPublishWithGuidance Recording · Job · CLI logs
InitTypeScriptAppHost_AugmentsExistingViteRepoInWorkspaceSubdirectory Recording · Job · CLI logs
InteractiveCSharpInitCreatesExpectedFiles Recording · Job · CLI logs
InvalidAppHostPathWithComments_IsHealedOnRun Recording · Job · CLI logs
JavaScriptHostingApisRunFromTypeScriptAppHost Recording · Job · CLI logs
LatestCliCanStartStableChannelAppHost Recording · Job · CLI logs
LatestCliCanStartStableChannelTypeScriptAppHost Recording · Job · CLI logs
LegacySettingsMigration_AdjustsRelativeAppHostPath Recording · Job · CLI logs
LogsCommandShowsResourceLogs Recording · Job · CLI logs
OtelLogsReturnsStructuredLogsFromStarterApp Recording · Job · CLI logs
OtelLogsReturnsStructuredLogsFromStarterAppIsolated Recording · Job · CLI logs
ProcessCommandCallbackReceivesCliArguments Recording · Job · CLI logs
PsCommandListsRunningAppHost Recording · Job · CLI logs
PsFormatJsonOutputsOnlyJsonToStdout Recording · Job · CLI logs
PublishJavaScriptPatternsGeneratesExpectedDockerComposeArtifacts Recording · Job · CLI logs
PublishWithConfigureEnvFileUpdatesEnvOutput Recording · Job · CLI logs
PublishWithDockerComposeServiceCallbackSucceeds Recording · Job · CLI logs
PublishWithoutOutputPathUsesAppHostDirectoryDefault Recording · Job · CLI logs
ResourceCommand_FailedExec_ShowsLogPathAndLogHasEntries Recording · Job · CLI logs
ResourceCommand_SetAndDeleteParameterUpdatesDescribeOutput Recording · Job · CLI logs
RestoreGeneratesSdkFiles Recording · Job · CLI logs
RestoreGeneratesSdkFiles_WithConfiguredToolchain Recording · Job · CLI logs
RestoreRefreshesGeneratedSdkAfterAddingIntegration Recording · Job · CLI logs
RestoreSupportsConfigOnlyHelperPackageAndCrossPackageTypes Recording · Job · CLI logs
RunFromParentDirectory_UsesExistingConfigNearAppHost Recording · Job · CLI logs
RunReportsSyntaxErrorsForDotNetAppHost Recording · Job · CLI logs
RunReportsSyntaxErrorsForTypeScriptAppHost Recording · Job · CLI logs
SecretCrudOnDotNetAppHost Recording · Job · CLI logs
SecretCrudOnTypeScriptAppHost Recording · Job · CLI logs
StagingChannel_ConfigureAndVerifySettings_ThenSwitchChannels Recording · Job · CLI logs
StartAndWaitForTypeScriptSqlServerAppHostWithNativeAssets Recording · Job · CLI logs
StartReportsSyntaxErrorsForDotNetAppHost Recording · Job · CLI logs
StartReportsSyntaxErrorsForTypeScriptAppHost Recording · Job · CLI logs
StopAllAppHostsFromAppHostDirectory Recording · Job · CLI logs
StopJavaPolyglotAppHostUsingApphostDirectory Recording · Job · CLI logs
StopNonInteractiveSingleAppHost Recording · Job · CLI logs
StopTypeScriptPolyglotAppHostUsingApphostDirectory Recording · Job · CLI logs
StopWithNoRunningAppHostExitsSuccessfully Recording · Job · CLI logs
TypeScriptAppHostRunDoesNotDeadlockWhenLazyOptionsInvokeAsyncCallback Recording · Job · CLI logs
TypeScriptAppHostWithVite_AllowsDifferentGuestPkgManager Recording · Job · CLI logs
UnAwaitedChainsCompileWithAutoResolvePromises Recording · Job · CLI logs
UpdateToStable_CSharpEmptyAppHost_KeepsConfigChannel Recording · Job · CLI logs
UpdateToStable_CSharpSingleFileInit_KeepsConfigChannel Recording · Job · CLI logs
UpdateToStable_TypeScriptSingleFileInit_KeepsConfigChannel Recording · Job · CLI logs
UpdateToStable_TypeScript_PreviewsStablePkgsAndKeepsChannel Recording · Job · CLI logs

📹 Recordings uploaded automatically from CI run #27301403912

@aspire-repo-bot

Copy link
Copy Markdown
Contributor

✅ No documentation update needed.

docs_optional → internal_refactor: No documentation update needed.

  • Triggered signals: none (signal_count = 0; recommendation: "docs_optional")
  • Changed files: src/Aspire.Cli/Telemetry/CodingAgentDetector.cs (internal telemetry identifier rename: "copilot""copilot-cli", plus new "copilot-vscode" rule for COPILOT_AGENT) and tests/Aspire.Cli.Tests/Telemetry/AspireCliTelemetryTests.cs (test data refactor from Dictionary to tuples)
  • The telemetry tag values (copilot, copilot-cli) are internal identifiers sent to Microsoft's backend; they are not documented or referenced on aspire.dev
  • No new CLI flags, options, commands, public types, configuration keys, or user-visible strings were introduced

@DamianEdwards

Copy link
Copy Markdown
Member Author

/backport to release/13.4

@github-actions

Copy link
Copy Markdown
Contributor

Started backporting to release/13.4 (link to workflow run)

@aspire-repo-bot

Copy link
Copy Markdown
Contributor

@DamianEdwards backporting to release/13.4 failed, the patch most likely resulted in conflicts. Please backport manually!

git am output
$ git am --3way --empty=keep --ignore-whitespace --keep-non-patch changes.patch

Applying: Report copilot-cli specifically & update tests to work in test explorer
Using index info to reconstruct a base tree...
A	src/Aspire.Cli/Telemetry/CodingAgentDetector.cs
M	tests/Aspire.Cli.Tests/Telemetry/AspireCliTelemetryTests.cs
Falling back to patching base and 3-way merge...
CONFLICT (modify/delete): src/Aspire.Cli/Telemetry/CodingAgentDetector.cs deleted in HEAD and modified in Report copilot-cli specifically & update tests to work in test explorer.  Version Report copilot-cli specifically & update tests to work in test explorer of src/Aspire.Cli/Telemetry/CodingAgentDetector.cs left in tree.
Auto-merging tests/Aspire.Cli.Tests/Telemetry/AspireCliTelemetryTests.cs
CONFLICT (content): Merge conflict in tests/Aspire.Cli.Tests/Telemetry/AspireCliTelemetryTests.cs
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Patch failed at 0001 Report copilot-cli specifically & update tests to work in test explorer
Error: The process '/usr/bin/git' failed with exit code 128

Link to workflow output

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants