Loosen LLM telemetry env detection to presence-based and add missing agent variables#54659
Merged
Conversation
…agent variables Loosen the AI-agent/LLM environment detector used for telemetry so detection is presence-based (variable set/non-empty) rather than requiring a literal `true` value. Converts the BooleanEnvironmentRule usages for gemini (GEMINI_CLI), legacy copilot (GITHUB_COPILOT_CLI_MODE), droid (DROID_CLI), kimi (KIMI_CLI) and the generic agent flag (AGENT_CLI) to AnyPresentEnvironmentRule, matching Pulumi's agentdetect approach. The shared BooleanEnvironmentRule type is retained (still used by the CI detector). Add the agent variable sets present in pulumi/pulumi's sdk/go/common/util/agentdetect/agentdetect.go: cursor (CURSOR_TRACE_ID, CURSOR_AGENT), codex (CODEX_CI, CODEX_THREAD_ID), copilot (COPILOT_MODEL, COPILOT_ALLOW_ALL, COPILOT_GITHUB_TOKEN), goose (GOOSE_PROVIDER), claude (CLAUDE_CODE), and new rules cowork (CLAUDE_CODE_IS_COWORK), replit (REPL_ID), augment (AUGMENT_AGENT) and antigravity (ANTIGRAVITY_AGENT). Extends the telemetry tests to cover presence-based matching of the loosened variables and detection of each newly added variable/agent. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the CLI’s LLM/agent environment detection used for telemetry to align with presence-based env-var detection and expands the set of recognized agents/variables, with corresponding test coverage updates.
Changes:
- Switched several LLM detector rules from strict boolean parsing to “env var present/non-empty” matching.
- Added additional env vars and new agent identifiers (e.g., cowork, replit, augment, antigravity) to improve detection coverage.
- Expanded
TelemetryCommonPropertiesTeststo validate the loosened matching behavior and all newly added variables/agents, including multi-detection output.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| test/dotnet.Tests/TelemetryTests/TelemetryCommonPropertiesTests.cs | Extends LLM telemetry test matrix to cover presence-based matching and newly recognized agents/variables, including combined detections. |
| src/Cli/dotnet/Telemetry/LLMEnvironmentDetectorForTelemetry.cs | Updates LLM environment detection rules to presence-based matching and adds missing/new agent env var indicators; keeps multi-match output behavior. |
14 tasks
Address review feedback: the Copilot CLI sets COPILOT_CLI (and related GITHUB_COPILOT_CLI_MODE/GH_COPILOT_WORKING_DIRECTORY/COPILOT_AGENT/COPILOT_MODEL/ COPILOT_ALLOW_ALL/COPILOT_GITHUB_TOKEN) variables, whereas GitHub Copilot in VS and VS Code does not set environment variables. Report the more specific `copilot-cli` value for these CLI experiences so they can be distinguished from (currently undetectable) editor-hosted Copilot, and clarify the rule comments to note that not all rules are presence-based and that all matching rules contribute to the result. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
GitHub Copilot agent mode in VS Code injects AI_AGENT=github_copilot_vscode_agent and COPILOT_AGENT=1 into the terminals it runs commands in (see vscode's toolTerminalCreator.ts), so COPILOT_AGENT is not a Copilot CLI signal. Move it out of the copilot-cli rule into a new copilot-vscode rule keyed on COPILOT_AGENT and the AI_AGENT=github_copilot_vscode_agent marker, so editor- hosted agent usage is reported as copilot-vscode rather than copilot-cli. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
nagilson
approved these changes
Jun 15, 2026
Member
Author
|
/ba-g flaky test |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Enhances the AI-agent / LLM environment detector used for telemetry (
LLMEnvironmentDetectorForTelemetry), drawing on Pulumi's equivalent detector inpulumi/pulumisdk/go/common/util/agentdetect/agentdetect.go.(a) Loosen checks to presence-based
Several rules previously used
BooleanEnvironmentRule, which only matched when the variable parsed as the literal valuetrue. These are now presence-based (the variable being set/non-empty is enough), matching Pulumi's "is this env var non-empty?" approach. Converted toAnyPresentEnvironmentRule:GEMINI_CLIGITHUB_COPILOT_CLI_MODE(the copilot rule'sAnyMatchEnvironmentRulewrapper is collapsed into a singleAnyPresentEnvironmentRule)DROID_CLIKIMI_CLIAGENT_CLIThe
OR_APP_NAMEexact-value rules (aider/plandex/openhands) are left asEnvironmentVariableValueRulesince those distinguish multiple tools sharing one variable. The sharedBooleanEnvironmentRuletype is not removed — it is still used byCIEnvironmentDetectorForTelemetry.(b) Add missing variable sets from Pulumi
CURSOR_TRACE_ID,CURSOR_AGENTCODEX_CI,CODEX_THREAD_IDCOPILOT_MODEL,COPILOT_ALLOW_ALL,COPILOT_GITHUB_TOKENGOOSE_PROVIDERCLAUDE_CODECLAUDE_CODE_IS_COWORK(placed beforeclaude, mirroring Pulumi's specific-before-broad ordering), replit →REPL_ID, augment →AUGMENT_AGENT, antigravity →ANTIGRAVITY_AGENTAll existing extra agents that Pulumi lacks (aider, plandex, amp, qwen, droid, kimi, zed, cline, roo, windsurf, opencode) are preserved.
(c) Test coverage
Extended
TelemetryCommonPropertiesTests:GEMINI_CLI=0→gemini; the previouslynull-expectedfalse-valued cases for gemini/copilot/agent/droid/kimi now match).cowork, claudecase verifyingCLAUDE_CODE_IS_COWORKis reported independently ofCLAUDE_CODE.Notes
string.Join(", ", ...)(unlike Pulumi, which returns the first match), so overlapping environments can produce multi-value output such ascowork, claude.Validation
Clean build (0 warnings); all telemetry test cases pass via the repo-local SDK.