Skip to content

feat(protocols): implement T5 local_shell tool + call/output items - #1338

Merged
slin1237 merged 2 commits into
mainfrom
feat/audit-t5-local-shell
Apr 23, 2026
Merged

slin1237 merged 2 commits into
mainfrom
feat/audit-t5-local-shell

Conversation

@slin1237

@slin1237 slin1237 commented Apr 22, 2026 •

Copy link
Copy Markdown
Member

Summary

Implements audit task T5: adds the local_shell built-in tool on ResponseTool plus the matching local_shell_call / local_shell_call_output items on both ResponseInputOutputItem and ResponseOutputItem.

What changed

  • crates/protocols/src/responses.rs:
    • ResponseTool::LocalShell — unit variant ({ type: "local_shell" }).
    • ResponseInputOutputItem::LocalShellCall { id, call_id, action, status } + mirror on ResponseOutputItem.
    • ResponseInputOutputItem::LocalShellCallOutput { id, output, status? } + mirror on ResponseOutputItem.
    • New supporting types: LocalShellExec::Exec { command, env, timeout_ms?, user?, working_directory? } and LocalShellCallStatus (in_progress | completed | incomplete).
    • 2 local cascade arms (validate_input_item, request text collector).
  • crates/protocols/tests/responses.rs: 5 integration round-trip tests (tool, input call, input call with all optionals, input output both status-present/absent, output-item mirror).
  • crates/mcp/src/core/session.rs, model_gateway/src/routers/grpc/harmony/builder.rs, model_gateway/src/routers/grpc/regular/responses/conversions.rs, model_gateway/src/routers/openai/responses/utils.rs: 4 forced-cascade match arms for the newly added variants, each one line / schema-only (no behavior).

Why

Per the Responses API spec (.claude/_audit/openai-responses-api-spec.md):

  • L219-226 — LocalShellCall / LocalShellCallOutput item shapes.
  • L462 — LocalShell { type: "local_shell" } tool.
  • L507-516 — lists both variants under output: array of ResponseOutputItem.

Status-optional on LocalShellCallOutput mirrors the OpenAI Python SDK v2.8.1 (openai==2.8.1, types/responses/response_input_item_param.py::LocalShellCallOutput).

Verification

  • cargo check -p openai-protocol --tests passes
  • cargo test -p openai-protocol --test responses passes (70 tests, incl. 5 new)
  • cargo check -p smg --lib passes
  • cargo fmt --all --check passes
  • cargo clippy -p openai-protocol -p smg --lib --tests -- -D warnings clean
  • Spec cross-checked: .claude/_audit/openai-responses-api-spec.md §LocalShellCall L219-226, §tools L462

Blast radius

  • Touched: crates/protocols/src/responses.rs, crates/protocols/tests/responses.rs, plus minimal forced-cascade arms in crates/mcp/src/core/session.rs, model_gateway/src/routers/grpc/harmony/builder.rs, model_gateway/src/routers/grpc/regular/responses/conversions.rs, model_gateway/src/routers/openai/responses/utils.rs.
  • Matches audit blast radius: schema only; no new router/gateway behavior.

Out of scope

  • No runtime handler for executing local shell commands (schema-only task).
  • No MCP/Harmony routing of local_shell_call beyond the forced-cascade placeholders.
  • No gRPC proto / conversions for the new variants.

Refs: T5 (.claude/_audit/responses-api-gap-audit.md L503-515).

Summary by CodeRabbit

  • New Features

    • Added protocol support for a new local_shell built-in tool so shell-call requests and outputs appear in responses.
    • Session visibility updated so local shell calls and their outputs are shown to clients.
  • Behavior

    • Routing/text-assembly now ignores local shell call items so they don't contribute to routing text.
    • Some gateway paths explicitly treat local shell items as unsupported and return a standard unsupported-item error.
  • Tests

    • Added serialization/deserialization tests for minimal and full local shell call/output shapes, including conditional status emission.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@github-actions github-actions Bot added grpc gRPC client and router changes mcp MCP related changes tests Test changes protocols Protocols crate changes model-gateway Model gateway crate changes openai OpenAI router changes labels Apr 22, 2026
@coderabbitai

coderabbitai Bot commented Apr 22, 2026 •

Copy link
Copy Markdown

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

Adds a new local_shell responses tool and its call/output item variants across protocols, routers, routing-text extraction, session visibility, and tests; marks local_shell outputs as client-visible and treats local_shell input items as unsupported in gateway conversions.

Changes

Cohort / File(s) Summary
Protocol Definitions
crates/protocols/src/responses.rs
Adds ResponseTool::LocalShell, ResponseInputOutputItem::LocalShellCall/LocalShellCallOutput, ResponseOutputItem variants, LocalShellExec, and LocalShellCallStatus; updates routing-text extraction and input validation pass-through.
Protocol Tests
crates/protocols/tests/responses.rs
Adds serde round-trip tests for local_shell tool, call and output shapes, and conditional status serialization behavior.
Session Visibility
crates/mcp/src/core/session.rs
is_client_visible_output_item now treats LocalShellCall and LocalShellCallOutput as client-visible.
gRPC Harmony Router
model_gateway/src/routers/grpc/harmony/builder.rs
Maps ResponseTool::LocalShell to "local_shell" and treats LocalShellCall/LocalShellCallOutput input items as unsupported.
gRPC Regular Router
model_gateway/src/routers/grpc/regular/responses/conversions.rs
responses_to_chat short-circuits with "Unsupported input item type" for LocalShellCall/LocalShellCallOutput.
OpenAI Router
model_gateway/src/routers/openai/responses/utils.rs
response_tool_to_value now returns a JSON value for ResponseTool::LocalShell.
Bench / Routing Text
model_gateway/benches/routing_allocation_bench.rs
Legacy extract_text_for_routing_old updated to skip LocalShellCall and LocalShellCallOutput.

Sequence Diagram(s)

sequenceDiagram
    participant Model as Model
    participant Gateway as Model Gateway
    participant Router as Router
    participant MCP as MCP Session
    participant Client as Client

    Model->>Gateway: emit Response (tool: local_shell, items)
    Gateway->>Router: parse/allowlist tool & items
    Router->>Gateway: mark local_shell input items unsupported / or pass through
    Gateway->>MCP: forward response items
    MCP->>MCP: is_client_visible_output_item -> true for local_shell outputs
    MCP->>Client: deliver client-visible local_shell output
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested reviewers

  • CatherineSue
  • key4ng

Poem

"🐇 I found a tiny JSON shell,
Hop-scattered fields that ring a bell,
Through protocols and routers I prance,
Now client-seen — a jaunty dance,
Hooray, local_shell's our spell!"

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat(protocols): implement T5 local_shell tool + call/output items' accurately summarizes the main change: adding schema-only support for a LocalShell tool and associated input/output items to the Responses protocol.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/audit-t5-local-shell

Comment @coderabbitai help to get the list of available commands and usage tips.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean protocol extension — no issues found. New LocalShell tool type and LocalShellCall/LocalShellCallOutput variants follow established patterns, serde renames match spec, all match sites updated, and tests comprehensively cover round-tripping.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 71687e0068

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +1254 to +1255
#[serde(rename = "local_shell_call")]
LocalShellCall {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Update benchmark matches for new local shell variants

Adding ResponseInputOutputItem::LocalShellCall / LocalShellCallOutput expands this enum, but model_gateway/benches/routing_allocation_bench.rs (extract_text_for_routing_old) still uses an exhaustive match over ResponseInputOutputItem without these new arms or a wildcard, so bench targets now fail to compile when running cargo check --benches or cargo bench --no-run due to non-exhaustive patterns.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Fixed in 456d1a5 by adding the forced-cascade arm (returns None to match sibling tool-call variants in the bench). Verified cargo clippy --all-targets --all-features -- -D warnings and cargo check --benches are clean locally.

slin1237 added a commit that referenced this pull request Apr 23, 2026
Adds forced-cascade match arms for the new
`ResponseInputOutputItem::LocalShellCall` and `LocalShellCallOutput`
variants in `extract_text_for_routing_old` under
`model_gateway/benches/routing_allocation_bench.rs`, so
`cargo clippy --all-targets --all-features -- -D warnings`
(CI lint step) and `cargo check --benches` pass with the T5
schema additions. Returns `None` to match every other
tool-call/output arm in the bench — no behavior.

Found by Codex Review on PR #1338.

Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
@github-actions github-actions Bot added the benchmarks Benchmark changes label Apr 23, 2026
@mergify

mergify Bot commented Apr 23, 2026

Copy link
Copy Markdown
Contributor

Hi @slin1237, this PR has merge conflicts that must be resolved before it can be merged. Please rebase your branch:

git fetch origin main
git rebase origin/main
# resolve any conflicts, then:
git push --force-with-lease

@mergify mergify Bot added the needs-rebase PR has merge conflicts that need to be resolved label Apr 23, 2026
slin1237 added a commit that referenced this pull request Apr 23, 2026
Adds forced-cascade match arms for the new
`ResponseInputOutputItem::LocalShellCall` and `LocalShellCallOutput`
variants in `extract_text_for_routing_old` under
`model_gateway/benches/routing_allocation_bench.rs`, so
`cargo clippy --all-targets --all-features -- -D warnings`
(CI lint step) and `cargo check --benches` pass with the T5
schema additions. Returns `None` to match every other
tool-call/output arm in the bench — no behavior.

Found by Codex Review on PR #1338.

Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
@slin1237
slin1237 force-pushed the feat/audit-t5-local-shell branch from 456d1a5 to a03679b Compare April 23, 2026 08:33
@mergify mergify Bot removed the needs-rebase PR has merge conflicts that need to be resolved label Apr 23, 2026
Comment on lines +772 to +774
ResponseInputOutputItem::LocalShellCall { .. }
| ResponseInputOutputItem::LocalShellCallOutput { .. } => {
Err("Unsupported input item type".to_string())

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Nit: Missing warn! log — the adjacent ShellCall arm (L763-770) logs a warning before returning Err, but this arm silently returns an error. The log is useful for diagnosing unexpected items that reach the Harmony conversion path.

Suggested change
ResponseInputOutputItem::LocalShellCall { .. }
| ResponseInputOutputItem::LocalShellCallOutput { .. } => {
Err("Unsupported input item type".to_string())
ResponseInputOutputItem::LocalShellCall { .. }
| ResponseInputOutputItem::LocalShellCallOutput { .. } => {
warn!(
function = "parse_response_item_to_harmony_message",
"LocalShell tool item reached Harmony conversion"
);
Err("Unsupported input item type".to_string())
}

Comment on lines +182 to +184
ResponseInputOutputItem::LocalShellCall { .. }
| ResponseInputOutputItem::LocalShellCallOutput { .. } => {
return Err("Unsupported input item type".to_string());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Nit: Same as the Harmony builder — the adjacent ShellCall arm (L173-179) logs a warn! before returning, but this arm is silent. Adding the log keeps the two tool types consistent and preserves debuggability.

Suggested change
ResponseInputOutputItem::LocalShellCall { .. }
| ResponseInputOutputItem::LocalShellCallOutput { .. } => {
return Err("Unsupported input item type".to_string());
ResponseInputOutputItem::LocalShellCall { .. }
| ResponseInputOutputItem::LocalShellCallOutput { .. } => {
warn!(
function = "responses_to_chat",
"LocalShell tool item reached chat conversion"
);
return Err("Unsupported input item type".to_string());
}

@mergify

mergify Bot commented Apr 23, 2026

Copy link
Copy Markdown
Contributor

Hi @slin1237, this PR has merge conflicts that must be resolved before it can be merged. Please rebase your branch:

git fetch origin main
git rebase origin/main
# resolve any conflicts, then:
git push --force-with-lease

@mergify mergify Bot added the needs-rebase PR has merge conflicts that need to be resolved label Apr 23, 2026
slin1237 added a commit that referenced this pull request Apr 23, 2026
Adds forced-cascade match arms for the new
`ResponseInputOutputItem::LocalShellCall` and `LocalShellCallOutput`
variants in `extract_text_for_routing_old` under
`model_gateway/benches/routing_allocation_bench.rs`, so
`cargo clippy --all-targets --all-features -- -D warnings`
(CI lint step) and `cargo check --benches` pass with the T5
schema additions. Returns `None` to match every other
tool-call/output arm in the bench — no behavior.

Found by Codex Review on PR #1338.

Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
@slin1237
slin1237 force-pushed the feat/audit-t5-local-shell branch from a03679b to 68481ec Compare April 23, 2026 08:47
@mergify mergify Bot removed the needs-rebase PR has merge conflicts that need to be resolved label Apr 23, 2026
@mergify

mergify Bot commented Apr 23, 2026

Copy link
Copy Markdown
Contributor

Hi @slin1237, this PR has merge conflicts that must be resolved before it can be merged. Please rebase your branch:

git fetch origin main
git rebase origin/main
# resolve any conflicts, then:
git push --force-with-lease

@mergify mergify Bot added the needs-rebase PR has merge conflicts that need to be resolved label Apr 23, 2026
Adds the `local_shell` built-in Responses tool (unit) plus the
matching `local_shell_call` / `local_shell_call_output` items on
both `ResponseInputOutputItem` and `ResponseOutputItem`.

Protocol schema (`crates/protocols/src/responses.rs`):
- `ResponseTool::LocalShell` — unit variant `{ type: "local_shell" }`.
- `ResponseInputOutputItem::LocalShellCall { id, call_id, action,
  status }` + `ResponseOutputItem::LocalShellCall` mirror.
- `ResponseInputOutputItem::LocalShellCallOutput { id, output, status? }`
  + `ResponseOutputItem::LocalShellCallOutput` mirror.
- `LocalShellExec::Exec { command, env, timeout_ms?, user?,
  working_directory? }` as the `action` payload.
- `LocalShellCallStatus` enum (`in_progress | completed | incomplete`).

Spec (`.claude/_audit/openai-responses-api-spec.md`):
- L219-226 — LocalShellCall + LocalShellCallOutput shapes.
- L462 — `LocalShell { type: "local_shell" }` tool definition.
- L507-516 — listed under `output: array of ResponseOutputItem`.

Tests (`crates/protocols/tests/responses.rs`, 5 new):
- `local_shell_tool_round_trips_spec_shape`
- `local_shell_call_input_item_round_trips_spec_shape`
- `local_shell_call_input_item_round_trips_with_all_optionals`
- `local_shell_call_output_input_item_round_trips_spec_shape`
  (covers both status-present and status-absent forms per SDK v2.8.1)
- `local_shell_output_item_variants_round_trip_spec_shape`

Forced-cascade match arms (schema-only, no behavior): 4 sites in
`smg` / `smg-mcp` (harmony builder tool-type projection,
harmony builder input item mapper, `responses_to_chat` converter,
openai `response_tool_to_value`) plus 2 sites inside the protocol
crate itself (`validate_input_item`, request text collector).

Refs: audit playbook §T5 (.claude/_audit/responses-api-gap-audit.md L503-515).
Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
Adds forced-cascade match arms for the new
`ResponseInputOutputItem::LocalShellCall` and `LocalShellCallOutput`
variants in `extract_text_for_routing_old` under
`model_gateway/benches/routing_allocation_bench.rs`, so
`cargo clippy --all-targets --all-features -- -D warnings`
(CI lint step) and `cargo check --benches` pass with the T5
schema additions. Returns `None` to match every other
tool-call/output arm in the bench — no behavior.

Found by Codex Review on PR #1338.

Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
@slin1237
slin1237 force-pushed the feat/audit-t5-local-shell branch from 68481ec to 69611ba Compare April 23, 2026 09:50
@mergify mergify Bot removed the needs-rebase PR has merge conflicts that need to be resolved label Apr 23, 2026
@slin1237
slin1237 merged commit 1159c6a into main Apr 23, 2026
16 of 17 checks passed
@slin1237
slin1237 deleted the feat/audit-t5-local-shell branch April 23, 2026 09:52

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 69611ba8d6

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

ResponseTool::Shell(_) => "shell",
ResponseTool::ApplyPatch => "apply_patch",
// T5 schema-only: forced-cascade arm, no behavior.
ResponseTool::LocalShell => "local_shell",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Classify local_shell as built-in in Harmony path

Adding ResponseTool::LocalShell => "local_shell" here feeds has_custom_tools(&tool_types), but BUILTIN_TOOLS in the same file does not include "local_shell". For requests that only declare the new built-in tool, this incorrectly sets with_custom_tools = true, which changes prompt construction (keeps commentary channel and injects a developer message despite no custom/function tools). This is a behavior regression for local-shell-only requests and should be fixed by adding local_shell to the built-in classification path.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

benchmarks Benchmark changes grpc gRPC client and router changes mcp MCP related changes model-gateway Model gateway crate changes openai OpenAI router changes protocols Protocols crate changes tests Test changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant