Repository navigation
fix(server): resolve a namespaced tool call to the offered tool - #1248
justinchuby wants to merge 1 commit into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## fix/honour-declared-sampling #1248 +/- ##
==============================================================
Coverage 79.89% 79.90%
==============================================================
Files 359 359
Lines 157720 157905 +185
Branches 157720 157905 +185
==============================================================
+ Hits 126011 126171 +160
- Misses 27100 27121 +21
- Partials 4609 4613 +4
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
This package's tool instructions carry a namespaced example — `example_tool_name.example_function_name` — so a model offered a bare name sometimes answers with a qualified one. OpenCode rejects the call outright: "Model tried to call unavailable tool 'glob.glob'. Available tools: bash, edit, glob, grep, read, write." The namespace is the model's spelling, not the caller's, and a client can only dispatch a name it offered. When the qualified name is not on offer but its final segment is, the call names that tool and is now resolved to it. Anything else is left exactly as the model spelled it, so a caller keeps reporting the unavailable tool it already knows how to report rather than being handed a target it never offered. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0190e2eb-abe4-451f-b36d-44a035a99b7e
ea24844 to
cba7254
Compare
71964fc to
d17e10b
Compare
…aces #1248) (#1254) Replacement for #1248, rebased onto current `main` after #1197/#1224/#1236 landed as squashes (the stacked base diverged). The original functional commit plus two added tests, re-validated on top of the merged stack. ## Symptom A model driven with flat tool names (`bash`, `edit`, `glob`, `grep`, `read`, `write`) emits a namespaced call such as `glob.glob`, which the client rejects: it can only dispatch a name it offered. ## Cause (as claimed by the PR, and what I could and could not confirm) The caller offers flat tool *definitions* over the API. The instruction text the model reads is rendered by the **package's own chat template**, which the PR says assumes namespaced names, takes `name.split('.')[0]`, and renders `# Valid recipients: "self", "glob.*", ..., "user".` — so the model follows a grammar that is wrong for a flat offer and writes `glob.glob`. **I could not inspect that template with my own eyes.** It ships inside the model package (Muse Glimmer / harmony-style), which is not in this repository or in my local model set; searching the repo and local packages for the `split('.')` / "Valid recipients" grammar found nothing. What I *can* confirm from the server side corroborates the addressing scheme: the ATEM parser recognises `to=functions.<name>` channels and `<atem:invoke name="...">` blocks (`extract_atem_tool_calls`, `ATEM_USER_CHANNEL = "to=user<|message|>"`), so a namespaced recipient like `functions.glob` producing a call named `glob.glob` is consistent with the diagnosis. **Template fix vs dispatch fix — my conclusion:** resolving at dispatch is the right layer, and this is a considered choice, not merely the first available. The template belongs to the model author and is rendered in the jinja sandbox; the server cannot rewrite an arbitrary third-party package's template at runtime, and the namespaced grammar is not universally wrong (it is correct for a caller that offers namespaced tools — only a flat offer breaks it). The server is the **only** component that sees both the caller's offer and the model's emitted call, so it is the only place that can reconcile them without editing model files. A template correction for flat offers would still be worthwhile in the package, but it cannot be the server's runtime responsibility. ## Fix When a parsed call names a tool that is not on offer but whose **final `.`-segment exactly equals an offered tool's whole name**, the call is resolved to that offered tool; anything else is left exactly as the model spelled it. Applied at the three call sites that already have the request in scope (buffered, streaming, and the logprob path), via `ParsedAssistantOutput::aligned_to` / `align_tool_calls`. **Ambiguity is impossible by construction, not merely avoided:** the resolution target is always the single final segment string, and it resolves only if that exact string is an offered (unique) name — there is never a choice between two candidates. When two offered tools merely share a final segment (`fs.read`, `net.read`) and the model writes a third namespace (`svc.read`), the bare segment `read` is offered by neither, so the call is left alone for the caller to reject. ## Verification (mine, on this branch atop the merged stack) - **216 lib + 31 http + 11 bundle tests pass**; `cargo clippy -p onnx-genai-server --all-targets -- -D warnings` clean. - **Falsified the positive claim** (revert → RED → restore → GREEN): neutering `align_tool_calls` to a no-op made `a_namespaced_call_resolves_to_the_offered_tool` and the new full-parse-path test return `glob.glob` instead of `glob`. - **Confirmed the negative tests have teeth, and characterised their limits honestly:** a no-op revert leaves the negatives green (they guard against *over*-resolution, not absence). Dropping the exact-offer guard makes `an_offered_name_is_left_alone` go RED (`fs.read` → `read`), which is exactly what that test protects. - **Added tests:** `a_shared_final_segment_is_not_resolved_arbitrarily` (the ambiguity case above) and `a_namespaced_atem_call_dispatches_through_the_full_parse_path`, which drives a raw `<atem:invoke name="glob.glob">` turn through the real `parse_assistant_output` → `align_tool_calls` path and asserts the dispatched name is `glob`. - **End-to-end note:** I could not reproduce the OpenCode/Muse path — no local model emits ATEM namespaced calls — so the end-to-end evidence is the full-parse-path unit test above (the server's actual code path), clearly labelled as such rather than an HTTP round-trip. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0190e2eb-abe4-451f-b36d-44a035a99b7e
|
Landed as #1254 (squash 39771f3), rebased onto current main after #1197/#1224/#1236 merged and the stacked base diverged. The replacement carries the original functional commit plus two added tests (ambiguity guard + full-parse-path e2e-equivalent). Validation summary is in the #1254 body. Closing this in favour of #1254. |
🔴 Benchmark Regression DetectedComparison of criterion micro-benchmarks: PR head vs merge-base, measured on the same runner in the same job (base first → PR second).
Visual flags: Host infoWhat this cannot catch
|
Stacked on #1236.
Symptom
Driving this model from OpenCode, a tool call fails outright:
Cause
Two different authors meet here. The caller supplies tool definitions over the API — OpenCode offers the flat names
bash,edit,glob,grep,read,write. The instruction text the model actually reads is rendered by this package's own chat template, which the caller never sees.That template assumes tool names are namespaced. It takes
name.split('.')[0]as the namespace and builds the addressing grammar from it, so a flat offer renders:The model is told in as many words that the valid recipient is
glob.*, neverglob. Under that grammar the only well-formed address it can write isglob.<something>, and the natural completion isglob.glob. The worked example reinforces it:So the model is following its instructions correctly; the instructions are the ones that are wrong for a flat offer. The client can only dispatch the name it offered, so it rejects the call and the turn is wasted.
The server is the only place that sees both what the caller offered and what the model said, which is where this can be reconciled.
Fix
When a call names a tool that is not on offer but whose final segment is, the call is resolved to the offered tool. Anything else is left exactly as the model spelled it, so a caller keeps reporting the unavailable tool it already knows how to report rather than being handed a target it never offered.
Parsing stays about the wire format; resolution happens where the request — and therefore the offer — is in scope, at the three call sites that already had it.
Verification
--all-targetsclean.