Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 75 additions & 0 deletions docs/fixes/TOOL_CALL_INTEGRITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Tool Call Integrity Fix

## Проблема

В OmniRoute v3.8.21 при streaming tool calls аргументы функций могли
искажаться в клиенте дублированием или повторной вставкой фрагментов, например
`find` превращался в `fifnd`, а `grep` — в `grreep`. Симптом проявлялся только
на machine JSON полях tool calls (`function.arguments` / `partial_json`) и был
независим от провайдера, потому что повреждение происходило в общем
OmniRoute SSE/translation pipeline после upstream-ответа.

## Корневая причина

Tool-call argument chunks обрабатывались как обычный человекочитаемый текст в
нескольких общих слоях:

- `src/lib/sseTextTransform.ts` рекурсивно передавал строковые поля
`arguments` и `partial_json` в текстовый processor.
- `src/lib/streamingPiiTransform.ts` буферизовал эти поля через rolling-window
PII sanitizer. Для machine JSON deltas это недопустимо: chunk может быть
дельтой, snapshot или overlap-fragment, а sanitizer не знает семантику
tool-call JSON.
- `open-sse/transformer/responsesTransformer.ts`,
`open-sse/translator/response/openai-to-claude.ts`,
`open-sse/translator/response/openai-responses.ts` и
`open-sse/handlers/sseParser.ts` накапливали аргументы простым `+=`, поэтому
повторный snapshot или overlapping delta добавлялся второй раз.

Обычный чат не ломался, потому что текстовые `content` deltas допускают
санитизацию и буферизацию. Tool calls ломались, потому что `arguments` — это
машинный JSON-контракт, который должен проходить byte-preserving до клиента.

## Исправление

В core source добавлена явная защита tool-call JSON:

1. `src/lib/sseTextTransform.ts` пропускает `toolArgs` и `partialJson` без
текстового processor.
2. `src/lib/streamingPiiTransform.ts` возвращает `toolArgs` и `partialJson`
как есть, без rolling-window buffering.
3. Общие stream assemblers используют `appendToolCallArgumentDelta()` вместо
слепого `+=`, чтобы повторные snapshots и overlapping chunks добавлялись
ровно один раз.
4. Responses/OpenAI/Claude translation paths эмитят клиенту только новый suffix
tool arguments, а не повторный snapshot.

## Как предотвратить регрессию

- `tool_calls.function.name`, `tool_calls.function.arguments`, Responses
`function_call.arguments` и Claude `input_json_delta.partial_json` никогда не
должны проходить через text/PII/compression/dedup transforms.
- Любой transform для SSE должен различать human text (`content`, `reasoning`)
и machine JSON (`arguments`, `partial_json`).
- Regression tests находятся в:
- `tests/unit/sseTextTransform.test.ts`
- `tests/unit/streamingPiiTransform.test.ts`
- `tests/unit/sse-parser.test.ts`
- `tests/unit/responses-transformer.test.ts`
- `tests/unit/translator-resp-openai-responses.test.ts`
- E2E smoke script: `tests/e2e-tool-calls.sh`.

## Конфигурация

Для coding-сессий можно дополнительно отключить risky text transforms:

```env
PII_RESPONSE_SANITIZATION=false
COMPRESSION_LEVEL=off
RTK_ENABLED=false
CAVEMAN_ENABLED=false
```

Основной фикс не зависит от этих env-переменных: machine tool-call JSON
защищён в core pipeline и не должен изменяться даже при включённой PII
response sanitization.
15 changes: 10 additions & 5 deletions open-sse/handlers/sseParser.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import { appendToolCallArgumentDelta } from "../utils/toolCallArguments.ts";
/**
* Convert OpenAI-style SSE chunks into a single non-streaming JSON response.
* Used as a fallback when upstream returns text/event-stream for stream=false.
Expand Down Expand Up @@ -181,7 +182,10 @@ export function parseSSEToOpenAIResponse(rawSSE, fallbackModel) {
existing.function.name = tc.function.name;
}
existing.function = existing.function || {};
existing.function.arguments = `${existing.function.arguments || ""}${deltaArgs}`;
existing.function.arguments = appendToolCallArgumentDelta(
existing.function.arguments,
deltaArgs
);
accumulatedToolCalls.set(key, existing);
}
}
Expand Down Expand Up @@ -505,7 +509,10 @@ function ensureResponsesReasoningItem(outputItems, outputIndex, itemId) {

const next = {
...(existing && typeof existing === "object" ? existing : {}),
id: itemId || (existing?.id != null ? String(existing.id) : null) || `rs_${Date.now()}_${outputIndex}`,
id:
itemId ||
(existing?.id != null ? String(existing.id) : null) ||
`rs_${Date.now()}_${outputIndex}`,
type: "reasoning",
summary: Array.isArray(existing?.summary)
? existing.summary.map((summaryPart) => ({ ...toRecord(summaryPart) }))
Expand Down Expand Up @@ -539,9 +546,7 @@ function ensureResponsesFunctionCallItem(outputItems, outputIndex, itemId, callI
const next = {
...(existing && typeof existing === "object" ? existing : {}),
id:
normalizedItemId ||
existingId ||
`fc_${normalizedCallId || `${Date.now()}_${outputIndex}`}`,
normalizedItemId || existingId || `fc_${normalizedCallId || `${Date.now()}_${outputIndex}`}`,
type: "function_call",
call_id: normalizedCallId || existingCallId || "",
name: name || existing?.name || "",
Expand Down
11 changes: 8 additions & 3 deletions open-sse/transformer/responsesTransformer.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import { appendToolCallArgumentDelta } from "../utils/toolCallArguments.ts";
import * as fs from "fs";
import * as path from "path";
/**
Expand Down Expand Up @@ -527,15 +528,19 @@ export function createResponsesApiTransformStream(logger = null, keepaliveInterv
.replace(/"[a-zA-Z0-9_]+":\s*\[\s*\],?/g, "");
}

if (refCallId) {
const existingArgs = state.funcArgsBuf[tcIdx] || "";
const nextArgs = appendToolCallArgumentDelta(existingArgs, deltaStr);
const emittedDelta = nextArgs.slice(existingArgs.length);
state.funcArgsBuf[tcIdx] = nextArgs;

if (refCallId && emittedDelta) {
emit(controller, "response.function_call_arguments.delta", {
type: "response.function_call_arguments.delta",
item_id: `fc_${refCallId}`,
output_index: tcIdx,
delta: deltaStr,
delta: emittedDelta,
});
}
state.funcArgsBuf[tcIdx] += deltaStr;
}
}
}
Expand Down
11 changes: 8 additions & 3 deletions open-sse/translator/response/openai-responses.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
*/
import { register } from "../registry.ts";
import { FORMATS } from "../formats.ts";
import { appendToolCallArgumentDelta } from "../../utils/toolCallArguments.ts";

function normalizeToolName(value) {
return typeof value === "string" ? value.trim() : "";
Expand Down Expand Up @@ -348,15 +349,19 @@ function emitToolCall(state, emit, tc) {

if (tc.function?.arguments) {
const refCallId = state.funcCallIds[tcIdx] || newCallId;
if (refCallId) {
const existingArgs = state.funcArgsBuf[tcIdx] || "";
const nextArgs = appendToolCallArgumentDelta(existingArgs, tc.function.arguments);
const emittedDelta = nextArgs.slice(existingArgs.length);
state.funcArgsBuf[tcIdx] = nextArgs;

if (refCallId && emittedDelta) {
emit("response.function_call_arguments.delta", {
type: "response.function_call_arguments.delta",
item_id: `fc_${refCallId}`,
output_index: tcIdx,
delta: tc.function.arguments,
delta: emittedDelta,
});
}
state.funcArgsBuf[tcIdx] += tc.function.arguments;
}
}

Expand Down
12 changes: 7 additions & 5 deletions open-sse/translator/response/openai-to-claude.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { register } from "../registry.ts";
import { FORMATS } from "../formats.ts";
import { CLAUDE_OAUTH_TOOL_PREFIX } from "../request/openai-to-claude.ts";
import { hasToolCallShim, applyToolCallShimToBuffer } from "../helpers/toolCallShim.ts";
import { appendToolCallArgumentDelta } from "../../utils/toolCallArguments.ts";

// Helper: stop thinking block if started
function stopThinkingBlock(state, results) {
Expand Down Expand Up @@ -192,15 +193,16 @@ export function openaiToClaudeResponse(chunk, state) {
if (toolInfo) {
// Always buffer the raw stream so shimmed tools can re-emit a
// corrected JSON at stop time.
toolInfo.argBuffer = (toolInfo.argBuffer || "") + tc.function.arguments;
const existingArgs = toolInfo.argBuffer || "";
const nextArgs = appendToolCallArgumentDelta(existingArgs, tc.function.arguments);
let deltaStr = nextArgs.slice(existingArgs.length);
toolInfo.argBuffer = nextArgs;

if (toolInfo.shimmed) {
// Suppress passthrough; we emit one corrective delta at finish.
if (toolInfo.shimmed || !deltaStr) {
// Suppress passthrough for shimmed tools; emit one corrective delta at finish.
continue;
}

let deltaStr = tc.function.arguments;

// Fix #1852: Strip empty string and array placeholders from streaming tool arguments
if (deltaStr.includes('""') || deltaStr.includes("[]") || deltaStr.includes("[ ]")) {
deltaStr = deltaStr
Expand Down
17 changes: 17 additions & 0 deletions open-sse/utils/toolCallArguments.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
export function appendToolCallArgumentDelta(current: unknown, incoming: unknown): string {
const existing = typeof current === "string" ? current : "";
const next = typeof incoming === "string" ? incoming : "";

if (!existing || !next) return existing + next;
if (existing.endsWith(next)) return existing;
if (next.startsWith(existing)) return next;

const maxOverlap = Math.min(existing.length, next.length);
for (let size = maxOverlap; size > 0; size -= 1) {
if (existing.endsWith(next.slice(0, size))) {
return existing + next.slice(size);
}
}
Comment thread
Mffff4 marked this conversation as resolved.
Resolved

return existing + next;
}
Loading