Skip to content

Dotnet: Fixing FoundryToolboxMcp sample to use created toolbox#5786

Merged
alliscode merged 1 commit into
microsoft:mainfrom
alliscode:mcp-sample-fix
May 13, 2026
Merged

Dotnet: Fixing FoundryToolboxMcp sample to use created toolbox#5786
alliscode merged 1 commit into
microsoft:mainfrom
alliscode:mcp-sample-fix

Conversation

@alliscode
Copy link
Copy Markdown
Member

This pull request introduces a new Python package, agent-framework-tools, providing built-in tools for the Microsoft Agent Framework, with a primary focus on robust, cross-platform shell command execution. It also updates the .NET sample for Foundry Toolbox MCP to simplify toolbox endpoint management and clarify documentation.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • The PR follows the Contribution Guidelines
  • All unit tests pass, and I have added new tests where possible
  • Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

Copilot AI review requested due to automatic review settings May 12, 2026 19:25
@moonbox3 moonbox3 added documentation Improvements or additions to documentation python .NET labels May 12, 2026
@github-actions github-actions Bot changed the title Fixing FoundryToolboxMcp sample to use created toolbox .NET: Fixing FoundryToolboxMcp sample to use created toolbox May 12, 2026
@github-actions github-actions Bot changed the title Fixing FoundryToolboxMcp sample to use created toolbox Python: Fixing FoundryToolboxMcp sample to use created toolbox May 12, 2026
@moonbox3
Copy link
Copy Markdown
Contributor

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
packages/tools/agent_framework_tools/shell
   _docker.py1853780%80, 88–89, 310, 313–315, 335, 360, 362–365, 369, 371, 408–409, 411, 424–425, 427, 436–443, 482, 504–505, 518–522
   _environment.py131596%92, 139, 170, 200, 209
   _executor.py34488%30, 58, 79–80
   _executor_base.py80100% 
   _killtree.py653447%42–47, 57, 61, 70, 73–74, 77–78, 94, 105–107, 116–132
   _policy.py43197%122
   _resolve.py432248%35–36, 38–42, 46, 48–50, 53–55, 58, 60–61, 69–72, 78
   _session.py2317667%114, 116, 137, 145, 157–158, 165–166, 169–172, 179–183, 188–189, 192, 209, 223–224, 244–250, 255, 257, 269–274, 277–283, 291, 293–300, 349–350, 379–382, 403–408, 411–415, 417–419, 438–439
   _tool.py991584%144, 146, 167–168, 170, 189, 232–233, 239, 267–271, 291
   _truncate.py140100% 
   _types.py250100% 
TOTAL34720410188% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
6849 34 💤 0 ❌ 0 🔥 1m 48s ⏱️

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new first-party Python tools package (agent-framework-tools) focused on cross-platform shell execution, and updates the Foundry Toolbox MCP .NET sample to create/connect to a toolbox endpoint programmatically instead of relying on a pre-specified endpoint.

Changes:

  • Introduces python/packages/tools (agent-framework-tools) with LocalShellTool, DockerShellTool, ShellPolicy, and ShellEnvironmentProvider, plus unit tests and docs.
  • Wires Python workspace packaging/lockfiles to include the new package, and updates the OpenAI local-shell sample to use LocalShellTool.
  • Updates the .NET Foundry Toolbox MCP sample to create a toolbox at startup and connect via Streamable HTTP with required preview headers.
Show a summary per file
File Description
python/uv.lock Adds agent-framework-tools to the uv workspace lock.
python/pyproject.toml Registers agent-framework-tools as a workspace package.
python/samples/02-agents/tools/local_shell_with_environment_provider.py New sample showing ShellEnvironmentProvider with LocalShellTool.
python/samples/02-agents/tools/local_shell_with_allowlist.py New sample showing strict allow-list usage via ShellPolicy.
python/samples/02-agents/providers/openai/client_with_local_shell.py Refactors sample to use LocalShellTool instead of ad-hoc subprocess execution.
python/packages/tools/README.md Documents agent-framework-tools, safety model, and usage examples.
python/packages/tools/LICENSE Adds MIT license file for the new package.
python/packages/tools/pyproject.toml Defines the new package metadata, deps (incl. psutil), and tooling config.
python/packages/tools/agent_framework_tools/init.py Package init with version discovery.
python/packages/tools/agent_framework_tools/py.typed Marks the package as typed.
python/packages/tools/agent_framework_tools/shell/init.py Public exports for shell tools, policy, provider, and helpers.
python/packages/tools/agent_framework_tools/shell/_types.py Defines ShellResult and shell-related exception types.
python/packages/tools/agent_framework_tools/shell/_truncate.py Implements shared head/tail truncation helpers.
python/packages/tools/agent_framework_tools/shell/_tool.py Implements LocalShellTool facade and agent-framework tool wiring.
python/packages/tools/agent_framework_tools/shell/_session.py Implements persistent shell session protocol via sentinels/readers.
python/packages/tools/agent_framework_tools/shell/_resolve.py Cross-platform shell discovery and argv resolution.
python/packages/tools/agent_framework_tools/shell/_policy.py Implements ShellPolicy allow/deny/custom filtering.
python/packages/tools/agent_framework_tools/shell/_killtree.py Implements cross-OS process-tree termination (psutil-backed).
python/packages/tools/agent_framework_tools/shell/_executor.py Stateless subprocess executor with timeout + truncation.
python/packages/tools/agent_framework_tools/shell/_executor_base.py Defines the ShellExecutor protocol.
python/packages/tools/agent_framework_tools/shell/_environment.py Implements ShellEnvironmentProvider and default prompt formatter.
python/packages/tools/agent_framework_tools/shell/_docker.py Implements DockerShellTool plus pure argv builders and availability check.
python/packages/tools/tests/init.py Initializes the new test package.
python/packages/tools/tests/test_shell_truncate_and_quote.py Tests truncation and quoting helpers.
python/packages/tools/tests/test_shell_result.py Tests ShellResult.format_for_model() behavior.
python/packages/tools/tests/test_shell_parse_rc.py Tests sentinel rc parsing for persistent sessions.
python/packages/tools/tests/test_shell_environment_provider.py Tests provider probing, caching, concurrency behavior, and formatting.
python/packages/tools/tests/test_security.py Adds security regression tests and documented residual-risk surface.
python/packages/tools/tests/test_policy.py Tests ShellPolicy default/deny/allow/custom behavior.
python/packages/tools/tests/test_local_shell_tool.py Tests local tool stateless/persistent behavior, policy, timeouts, and concurrency.
python/packages/tools/tests/test_docker_shell_tool.py Tests docker argv builders and docker tool behavior (with gated integration tests).
dotnet/samples/02-agents/AgentsWithFoundry/Agent_Step25_FoundryToolboxMcp/README.md Updates sample docs to describe startup creation + endpoint connection flow.
dotnet/samples/02-agents/AgentsWithFoundry/Agent_Step25_FoundryToolboxMcp/Program.cs Updates sample to create toolbox and connect via Streamable HTTP with headers.

Copilot's findings

  • Files reviewed: 31/33 changed files
  • Comments generated: 4

Comment thread python/packages/tools/agent_framework_tools/shell/_session.py Outdated
Comment thread python/packages/tools/agent_framework_tools/shell/_policy.py Outdated
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Code Review

Reviewers: 4 | Confidence: 78%

✓ Correctness

The .NET sample change in CreateSampleToolboxAsync constructs the toolbox MCP endpoint URL using api-version=v{created.Version}, which incorrectly substitutes the toolbox version (e.g., "2025-09-01") for the Azure API version (should be "2025-05-01-preview"). This is confirmed by FoundryToolboxOptions.cs:26 which defines ApiVersion = "2025-05-01-preview" and FoundryToolboxService.cs:195 which constructs the same class of URL with api-version={this._options.ApiVersion}. The existing test at FoundryAIToolExtensionsTests.cs:55 confirms ToolboxVersion.Version is a toolbox revision identifier like "2025-09-01", not an API version string. The Python shell tools package appears correct.

✓ Security Reliability

No actionable issues found in this dimension.

✓ Test Coverage

No actionable issues found in this dimension.

✗ Design Approach

It tells readers that the deny-list overrides the allow-list and that disabling approval is safe because the allow-list does the gating, but the framework’s own ShellPolicy contract says the opposite: allow rules short-circuit deny rules, and pattern matching is not the security boundary. That makes the sample teach the wrong deployment model for unattended local execution.

Flagged Issues

  • The local_shell_with_allowlist.py sample documents an unsafe policy model: it says 'the deny-list still wins' and justifies approval_mode="never_require" on that basis, but the framework's ShellPolicy contract explicitly states that allow patterns short-circuit deny patterns and that policy regexes are only a UX pre-filter, not a security control (ShellPolicy.cs:127-132, 178-205).

Automated review by alliscode's agents

@alliscode alliscode changed the title Python: Fixing FoundryToolboxMcp sample to use created toolbox Dotnet: Fixing FoundryToolboxMcp sample to use created toolbox May 13, 2026
@alliscode alliscode added this pull request to the merge queue May 13, 2026
Merged via the queue into microsoft:main with commit bd0d607 May 13, 2026
27 checks passed
@moonbox3
Copy link
Copy Markdown
Contributor

@alliscode is the PR description accurate? "This pull request introduces a new Python package, agent-framework-tools"?

This pull request introduces a new Python package, agent-framework-tools, providing built-in tools for the Microsoft Agent Framework, with a primary focus on robust, cross-platform shell command execution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation .NET python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants