fix(claude): preserve tool_result adjacency in native and CC-compatible paths - #1555
Merged
diegosouzapw merged 4 commits intoApr 24, 2026
Merged
diegosouzapw merged 4 commits into
diegosouzapw merged 4 commits into
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces account-level concurrency limiting via a semaphore system, integrated into both streaming and non-streaming request paths in chatCore.ts. It also enhances Claude-specific message processing by adding content stripping for incompatible types, preserving tool_result blocks during passthrough, and improving the extraction of system messages from OpenAI-style payloads. Additionally, the changes include improved tracking of pending requests, audit logging for provider warnings, and synchronization of Claude extra-usage states. Feedback was provided regarding the robustness of the semaphore release mechanism in the streaming path, specifically to ensure resources are freed during stream cancellations or when handling empty bodies.
Comment on lines
+2006
to
+2025
| if (stream) { | ||
| const originalBody = res.response.body; | ||
| const wrappedBody = originalBody | ||
| ? originalBody.pipeThrough( | ||
| new TransformStream({ | ||
| flush: () => { | ||
| acquireAccountSemaphoreRelease(); | ||
| }, | ||
| }) | ||
| ) | ||
| : null; | ||
| return { | ||
| ...res, | ||
| response: new Response(wrappedBody, { | ||
| status: res.response.status, | ||
| statusText: res.response.statusText, | ||
| headers: res.response.headers, | ||
| }), | ||
| }; | ||
| } |
Contributor
There was a problem hiding this comment.
Implement a robust release mechanism for the streaming path that handles empty bodies and stream cancellation (client disconnects).
References
- Resources like semaphores or locks must be released in all terminal states, including success, error, and cancellation.
Register Nous Research as an OpenAI-compatible gateway with remote model discovery and validation against chat completions. Add Petals provider metadata, default config, validation, and a specialized executor that maps OpenAI-style requests to the public generate endpoint. Also allow optional API keys and configurable base URLs for Petals in the dashboard and provider schemas. Expand provider model and catalog tests to cover both integrations.
Await runtime request queue updates so limiter settings and auto-enabled API key protections are recomputed when resilience settings change. Preserve cancelled batch state for in-flight work by marking input files processed without generating output artifacts, and replace cached synced models with an empty set when remote discovery returns no models so the providers route falls back to the local catalog instead of stale cache.
Owner
|
Great work! 🚀 I've merged this into |
Merged
Poid-ZA
pushed a commit
to Poid-ZA/OmniRoute
that referenced
this pull request
Aug 5, 2026
* fix(claude): preserve tool_result adjacency in native and CC-compatible paths * feat(providers): add Petals and Nous Research provider support Register Nous Research as an OpenAI-compatible gateway with remote model discovery and validation against chat completions. Add Petals provider metadata, default config, validation, and a specialized executor that maps OpenAI-style requests to the public generate endpoint. Also allow optional API keys and configurable base URLs for Petals in the dashboard and provider schemas. Expand provider model and catalog tests to cover both integrations. * fix(resilience): sync queue updates and clear stale discovery caches Await runtime request queue updates so limiter settings and auto-enabled API key protections are recomputed when resilience settings change. Preserve cancelled batch state for in-flight work by marking input files processed without generating output artifacts, and replace cached synced models with an empty set when remote discovery returns no models so the providers route falls back to the local catalog instead of stale cache. --------- Co-authored-by: congvc <congvc-dev@gmail.com> Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
* fix(claude): preserve tool_result adjacency in native and CC-compatible paths * feat(providers): add Petals and Nous Research provider support Register Nous Research as an OpenAI-compatible gateway with remote model discovery and validation against chat completions. Add Petals provider metadata, default config, validation, and a specialized executor that maps OpenAI-style requests to the public generate endpoint. Also allow optional API keys and configurable base URLs for Petals in the dashboard and provider schemas. Expand provider model and catalog tests to cover both integrations. * fix(resilience): sync queue updates and clear stale discovery caches Await runtime request queue updates so limiter settings and auto-enabled API key protections are recomputed when resilience settings change. Preserve cancelled batch state for in-flight work by marking input files processed without generating output artifacts, and replace cached synced models with an empty set when remote discovery returns no models so the providers route falls back to the local catalog instead of stale cache. --------- Co-authored-by: congvc <congvc-dev@gmail.com> Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tool_resultblocks for native Claude passthrough requeststool_use/tool_resultadjacencysystem/developersource messages into top-level Claudesystemblocks without corrupting tool historyProblem
Claude-native tool use is strict: when an assistant emits
tool_use, the immediately following user turn must contain the matchingtool_resultblocks.OmniRoute was mutating this history in two places:
tool_resultblocks into plain text ([Tool Result: ...])That caused Anthropic 400s like:
tool_use ids were found without tool_result blocks immediately afterScope
This fix is not specific to
claude-opus-4-7.It applies to the native Claude provider path and the Claude Code-compatible request builder, so it affects Claude-family models that travel through those paths, including Sonnet/Opus/Haiku variants on Anthropic-native routing.
The live reproduction I used during diagnosis happened on
cc/claude-opus-4-7, but the code changes themselves are provider/path scoped rather than model-id scoped.Fix
normalizeClaudeUpstreamMessages()to preserve structuredtool_resultblocks for native Claude requeststool_use/tool_resultboundariestool_useturns that are still awaiting the next usertool_resultsystem/developermessages cleanly into Claude top-levelsystemValidation
node --import tsx/esm --test tests/unit/claude-code-compatible-request.test.tstests/unit/claude-code-compatible-request.test.tstests/unit/chatcore-sanitization.test.ts/v1/messageswith native Claude routing and assistanttool_use+ next-usertool_resultreturned 200 after the patchNote
The live reproduction used
cc/claude-opus-4-7because that was the failing route available during debugging. That should be read as validated example coverage, not as a model-specific fix.