Python: Add OpenTelemetry integration for GitHubCopilotAgent#5142
Conversation
There was a problem hiding this comment.
Pull request overview
Adds OpenTelemetry tracing support for the Python GitHubCopilotAgent by aligning it with the framework’s established “Raw* + telemetry-enabled wrapper” pattern and updating the GitHub Copilot SDK integration accordingly.
Changes:
- Split the agent into
RawGitHubCopilotAgent(core) andGitHubCopilotAgent(AgentTelemetryLayer, RawGitHubCopilotAgent)(instrumented). - Bump
github-copilot-sdkdependency to>=0.2.0,<0.3.0and update call sites for the SDK’s updated APIs/types. - Add an observability sample + README updates documenting OTel environment variables.
Reviewed changes
Copilot reviewed 16 out of 17 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| python/uv.lock | Updates locked dependencies, including github-copilot-sdk to 0.2.x. |
| python/packages/github_copilot/pyproject.toml | Bumps github-copilot-sdk requirement to >=0.2.0,<0.3.0. |
| python/packages/github_copilot/agent_framework_github_copilot/_agent.py | Introduces RawGitHubCopilotAgent and an OTel-enabled GitHubCopilotAgent wrapper; updates SDK calls for 0.2.x. |
| python/packages/github_copilot/agent_framework_github_copilot/init.py | Exports RawGitHubCopilotAgent from the package public surface. |
| python/packages/core/agent_framework/github/init.py | Adds RawGitHubCopilotAgent to the lazy re-export mapping. |
| python/packages/core/agent_framework/github/init.pyi | Adds RawGitHubCopilotAgent to typing exports. |
| python/packages/github_copilot/tests/test_github_copilot_agent.py | Updates tests for SDK 0.2.x API/type changes and updated call signatures. |
| python/scripts/sample_validation/create_dynamic_workflow_executor.py | Updates permission result import for SDK 0.2.x. |
| python/samples/02-agents/providers/github_copilot/README.md | Documents OTel environment variables and links to the new observability sample. |
| python/samples/02-agents/providers/github_copilot/github_copilot_basic.py | Updates permission result import for SDK 0.2.x. |
| python/samples/02-agents/providers/github_copilot/github_copilot_with_file_operations.py | Updates permission result import for SDK 0.2.x. |
| python/samples/02-agents/providers/github_copilot/github_copilot_with_mcp.py | Updates permission/mcp config imports for SDK 0.2.x. |
| python/samples/02-agents/providers/github_copilot/github_copilot_with_multiple_permissions.py | Updates permission result import for SDK 0.2.x. |
| python/samples/02-agents/providers/github_copilot/github_copilot_with_session.py | Updates permission result import for SDK 0.2.x. |
| python/samples/02-agents/providers/github_copilot/github_copilot_with_shell.py | Updates permission result import for SDK 0.2.x. |
| python/samples/02-agents/providers/github_copilot/github_copilot_with_url.py | Updates permission result import for SDK 0.2.x. |
| python/samples/02-agents/providers/github_copilot/github_copilot_with_observability.py | New sample demonstrating configure_otel_providers() with GitHubCopilotAgent. |
droideronline
force-pushed
the
dinesh/github-copilot-otel-integration
branch
from
28955c8 to
60d9177
Compare
|
Waiting for #5107 to be merged |
droideronline
force-pushed
the
dinesh/github-copilot-otel-integration
branch
2 times, most recently
from
6c50266 to
42680ad
Compare
|
@moonbox3 @eavanvalkenburg @chetantoshniwal - could you please review this PR when you get a chance? Branch has been rebased on latest main and all tests pass (80/80). Thanks! |
droideronline
force-pushed
the
dinesh/github-copilot-otel-integration
branch
from
126d3c4 to
21022fd
Compare
eavanvalkenburg
left a comment
eavanvalkenburg
left a comment
There was a problem hiding this comment.
one final thing needed, otherwise good to go
|
@droideronline we do have a failing test, from this update, please have a look |
Head branch was pushed to by a user without write access
droideronline
force-pushed
the
dinesh/github-copilot-otel-integration
branch
from
6700fb2 to
fbac838
Compare
|
@eavanvalkenburg @moonbox3 — while working on this PR, I discovered a bug in the core OTel layer: This was also the root cause of I've filed the details in #5231 and raised a fix in #5232. Would appreciate a look when you get a chance. |
droideronline
force-pushed
the
dinesh/github-copilot-otel-integration
branch
from
fbac838 to
b8f1d98
Compare
|
@eavanvalkenburg @moonbox3 the branch is now rebased on the latest main and all tests pass, including |
|
@eavanvalkenburg @moonbox3 the core fix from #5232 has now been merged and the branch is freshly rebased. Could you please take one more look? |
|
@moonbox3 @eavanvalkenburg - I'd appreciate if we can take a look again please. thanks |
|
@moonbox3 @eavanvalkenburg - I'd appreciate if we can get this before our next release, we are in need of this. thanks |
moonbox3
left a comment
moonbox3
left a comment
There was a problem hiding this comment.
Automated Code Review
Reviewers: 4 | Confidence: 87%
✓ Correctness
This PR cleanly refactors GitHubCopilotAgent into a layered architecture by extracting a RawGitHubCopilotAgent (no telemetry) and composing GitHubCopilotAgent via MRO with AgentMiddlewareLayer and AgentTelemetryLayer. The MRO order is intentionally reversed from the Agent class to place middleware outside the telemetry span. The init chain, run() chain, and default_options property all work correctly through the MRO. Tests are well-constructed. No blocking correctness issues found.
✓ Security Reliability
This PR refactors GitHubCopilotAgent into a two-layer design: RawGitHubCopilotAgent (core) and GitHubCopilotAgent (with middleware + telemetry via MRO). The MRO chain, init parameter forwarding, and default_options property are implemented correctly. The middleware-outside-telemetry ordering is intentional and documented. No security or reliability issues found. One minor type annotation inconsistency noted.
✓ Test Coverage
The PR refactors GitHubCopilotAgent into a RawGitHubCopilotAgent base class and a new GitHubCopilotAgent composed via MRO with AgentMiddlewareLayer and AgentTelemetryLayer. Three new tests cover the
default_optionsproperty, which is good. However, the newly exportedRawGitHubCopilotAgentclass has zero direct test coverage (confirmed: 0 occurrences in test files), and there is no integration test verifying that the MRO-based middleware/telemetry layer composition actually invokes the layers duringrun()— which is the primary architectural change in this PR.
✗ Design Approach
The telemetry wrapper is a reasonable direction, but the middleware approach is not wired through correctly. The new
GitHubCopilotAgentadvertises full middleware support by inheritingAgentMiddlewareLayer, yet the underlying Copilot implementation drops the forwardedmiddlewarekwarg and invokes tools directly, so chat/function middleware are silently ignored instead of participating in execution. That makes the public API broader than the implementation and is worth fixing before release.
Automated review by moonbox3's agents
- Split GitHubCopilotAgent into RawGitHubCopilotAgent (core, no OTel) and GitHubCopilotAgent(AgentTelemetryLayer, RawGitHubCopilotAgent) with tracing - Add default_options property to expose model for span attributes - Export RawGitHubCopilotAgent from all public namespaces - Add github_copilot_with_observability.py sample and update README
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add middleware param to GitHubCopilotAgent.run() overloads so per-call middleware is explicitly forwarded through AgentTelemetryLayer - Remove github_copilot_with_observability.py sample per feedback; replace with inline snippet + link to observability samples in README
- Add middleware param to RawGitHubCopilotAgent.run() overloads for interface compatibility with AgentTelemetryLayer - Fix import in README observability snippet to use agent_framework.github
Follow FoundryAgent pattern: AgentMiddlewareLayer runs outside the telemetry span so middleware execution time is not captured in traces. Overloads removed as AgentMiddlewareLayer.run() handles dispatch via MRO.
droideronline
force-pushed
the
dinesh/github-copilot-otel-integration
branch
from
28122d1 to
295976f
Compare
…ions - Add assert "timeout" not in opts to test_default_options_includes_model_for_telemetry to document the intentional asymmetry where timeout is extracted into _settings and not returned in default_options. - Replace silent del middleware with a logged warning when per-run middleware is passed to RawGitHubCopilotAgent, making it clear that the GitHub Copilot SDK handles tool execution internally and chat/function middleware cannot be injected.
eavanvalkenburg
left a comment
eavanvalkenburg
left a comment
There was a problem hiding this comment.
Small comment left, otherwise good to go
Address review feedback: use typing.Self (3.11+) / typing_extensions.Self (3.10) for __aenter__ so subclasses like GitHubCopilotAgent get the correct return type from async context manager usage.
…-otel-integration
Previous commit incorrectly renamed the [1.1.1] header to [1.2.0], which wiped the historical 1.1.1 entries and wrongly attributed them to 1.2.0. This restores [1.1.1] to its origin/main content and adds a new [1.2.0] section above containing only the commits in python-1.1.1..HEAD: - microsoft#4238 functional workflow API - microsoft#5142 GitHub Copilot OpenTelemetry - microsoft#2403 A2A bridge support - microsoft#5070 oauth_consent_request events in Foundry clients - microsoft#5447 FoundryAgent hosted agent sessions - microsoft#5459 hosting server dependency upgrade + types - microsoft#5389 AG-UI reasoning/multimodal parsing fix - microsoft#5440 stop [TOOLBOXES] warning spam - microsoft#5455 user agent prefix fix Also corrects the [1.2.0] compare base to python-1.1.1 (not 1.1.0) and adds the missing [1.1.1] reference link.
* Bump Python package versions for 1.2.0 release Released tier bumps 1.1.1 -> 1.2.0 (core, openai, foundry, root) to reflect additive public APIs landed since 1.1.0: functional workflow API (#4238) and FunctionTool SKIP_PARSING sentinel (#5424). All beta packages stamped 1.0.0b260424, alpha packages 1.0.0a260424. All 26 non-core agent-framework-core floors raised to >=1.2.0,<2. CHANGELOG consolidates the never-tagged 1.1.1 entries with the post-merge additions into [1.2.0]. * Update CHANGELOG footer links for 1.2.0 Advance [Unreleased] comparison base from python-1.1.0 to python-1.2.0 and add a [1.2.0] reference link comparing python-1.1.0...python-1.2.0 so the heading links resolve correctly. * Fix CHANGELOG: restore [1.1.1] section and add proper [1.2.0] Previous commit incorrectly renamed the [1.1.1] header to [1.2.0], which wiped the historical 1.1.1 entries and wrongly attributed them to 1.2.0. This restores [1.1.1] to its origin/main content and adds a new [1.2.0] section above containing only the commits in python-1.1.1..HEAD: - #4238 functional workflow API - #5142 GitHub Copilot OpenTelemetry - #2403 A2A bridge support - #5070 oauth_consent_request events in Foundry clients - #5447 FoundryAgent hosted agent sessions - #5459 hosting server dependency upgrade + types - #5389 AG-UI reasoning/multimodal parsing fix - #5440 stop [TOOLBOXES] warning spam - #5455 user agent prefix fix Also corrects the [1.2.0] compare base to python-1.1.1 (not 1.1.0) and adds the missing [1.1.1] reference link.
* Sync Agent Framework Python docs for PRs since 2026-04-22 Updates docs to reflect changes in microsoft/agent-framework Python PRs merged after MicrosoftDocs/semantic-kernel-pr#982 (2026-04-22). - get-started/workflows.md, AGENTS.md: rename 01-get-started/05_first_workflow.py -> 07_first_graph_workflow.py (microsoft/agent-framework#4238). - integrations/a2a.md: add Python section 'Exposing an Agent Framework agent over A2A' documenting A2AExecutor and the new agent_framework_to_a2a.py sample (microsoft/agent-framework#2403). - agents/providers/github-copilot.md: add Observability subsection in Python zone showing configure_otel_providers() and RawGitHubCopilotAgent (microsoft/agent-framework#5142). - agents/providers/microsoft-foundry.md: add 'Connecting to a deployed (hosted) Foundry agent' subsection covering allow_preview=True / v2 and explicit AIProjectClient session creation (microsoft/agent-framework#5447). - integrations/hyperlight.md: note that result_parser does not run on the sandbox path; tools must format in-sandbox themselves (microsoft/agent-framework#5424). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Defer #4238 (functional workflow API) doc updates to PR author Revert the get-started/workflows.md sample rename and the AGENTS.md mapping update. The author of microsoft/agent-framework#4238 will handle the docs (rename + new functional workflow page) in a dedicated follow-up. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Summary
Adds OpenTelemetry tracing support for
GitHubCopilotAgentfollowing the same pattern used byClaudeAgentandFoundryAgent.GitHubCopilotAgentintoRawGitHubCopilotAgent(core, no OTel) andGitHubCopilotAgent(AgentTelemetryLayer, RawGitHubCopilotAgent)(OTel-enabled, recommended)default_optionsproperty to expose the configured model name in span attributesRawGitHubCopilotAgentfromagent_framework.githubandagent_framework_github_copilotgithub_copilot_with_observability.pysample and update README with OTel environment variablesCloses #5141
Test plan
uv run poe test- 77 passed)uv run poe lint)uv run poe pyright- 0 errors)uv run mypy- no issues)Notes
GitHubCopilotAgentremains the default import; existing code is unaffectedRawGitHubCopilotAgent.run()accepts**kwargsto absorb extra parameters passed byAgentTelemetryLayer(e.g.,compaction_strategy,tokenizer)default_optionsproperty mergesmodelfrom_settingsso it appears in OTel span attributes