Skip to content

Studio: simplify tool-call dedup and replace html2text with builtin converter #4722

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
danielhanchen merged 14 commits into from
Mar 31, 2026
Merged

Studio: simplify tool-call dedup and replace html2text with builtin converter #4722

Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
d9380bb
Simplify tool-call dedup: drop hashlib, inline helpers
danielhanchen Mar 31, 2026
cc8fbbd
Remove tool_calling_benchmark_results.md from repo
danielhanchen Mar 31, 2026
dff2fd3
Replace html2text with builtin HTML-to-Markdown converter
danielhanchen Mar 31, 2026
40cd783
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 31, 2026
b304264
Use json.dumps(sort_keys=True) for tool-call dedup key
danielhanchen Mar 31, 2026
70341be
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 31, 2026
2046c5c
Revert dedup key to str(arguments)
danielhanchen Mar 31, 2026
c99f828
Address review comments on _html_to_md.py
danielhanchen Mar 31, 2026
2c27165
Fix blockquote rendering, truncated-HTML buffer flush, and dedup key …
danielhanchen Mar 31, 2026
f779091
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 31, 2026
ad18902
Fix table optional end tags, inline code whitespace, and link text no…
danielhanchen Mar 31, 2026
b07afb5
Support <ol start=N>, <dl>/<dt>/<dd>, and preserve code block whitespace
danielhanchen Mar 31, 2026
d398489
Suppress whitespace-only text nodes between table structural elements
danielhanchen Mar 31, 2026
24b857f
Revert dedup key to str(arguments) with explanatory comment
danielhanchen Mar 31, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 4 additions & 19 deletions studio/backend/core/inference/llama_cpp.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@

import atexit
import contextlib
import hashlib
import json
import re
import struct
Expand Down Expand Up @@ -2181,22 +2180,6 @@ def _strip_tool_markup(text: str, *, final: bool = False) -> str:
# identical call succeeded).
_tool_call_history: list[tuple[str, bool]] = [] # (key, failed)

def _tool_call_key(name: str, args: dict) -> str:
raw = json.dumps({"t": name, "a": args}, sort_keys = True)
return hashlib.md5(raw.encode()).hexdigest()

def _is_duplicate_call(name: str, args: dict) -> bool:
"""Block if the immediately previous call was identical and succeeded."""
if not _tool_call_history:
return False
key = _tool_call_key(name, args)
last_key, last_failed = _tool_call_history[-1]
return last_key == key and not last_failed

def _record_tool_call(name: str, args: dict, failed: bool) -> None:
key = _tool_call_key(name, args)
_tool_call_history.append((key, failed))

for iteration in range(max_tool_iterations):
if cancel_event is not None and cancel_event.is_set():
return
Expand Down Expand Up @@ -2692,7 +2675,9 @@ def _record_tool_call(name: str, args: dict, failed: bool) -> None:
}

# ── Duplicate call detection ──────────────
if _is_duplicate_call(tool_name, arguments):
_tc_key = tool_name + str(arguments)

@chatgpt-codex-connector chatgpt-codex-connector Bot Mar 31, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Canonicalize tool-call args before dedup comparison

Building _tc_key with tool_name + str(arguments) makes duplicate detection depend on JSON key insertion order, so the same semantic tool call can evade dedup when the model emits arguments in a different key order on the next turn (e.g., {\"q\":...,\"limit\":...} vs {\"limit\":...,\"q\":...}). In that case the guard no longer blocks repeated successful calls, which can re-trigger expensive or side-effectful tools and reintroduce loop behavior this block is meant to prevent.

Useful? React with 👍 / 👎.

@gemini-code-assist gemini-code-assist Bot Mar 31, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The change from json.dumps(..., sort_keys=True) to str(arguments) introduces a regression in robustness. While Python 3.7+ dictionaries preserve insertion order, str(dict) is sensitive to that order. If the model generates the same tool arguments but with keys in a different order, they will be semantically identical but produce different strings, causing the duplicate detection to fail. I recommend using json.dumps with sort_keys=True for the key generation.

_tc_key = tool_name + json.dumps(arguments, sort_keys=True)

@danielhanchen danielhanchen Mar 31, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch -- switched to json.dumps(arguments, sort_keys=True) in b304264.

@danielhanchen danielhanchen Mar 31, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted back to str(arguments) in 2046c5c. The arguments dict always comes from the same JSON parser within a single request, so key insertion order is deterministic (Python 3.7+ dicts). json.dumps(sort_keys=True) adds overhead on every tool call for a case that can't actually happen here.

_prev = _tool_call_history[-1] if _tool_call_history else None
if _prev and _prev[0] == _tc_key and not _prev[1]:

@chatgpt-codex-connector chatgpt-codex-connector Bot Mar 31, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Canonicalize args when building duplicate-call key

Building the dedup key with tool_name + str(arguments) makes loop prevention depend on Python dict insertion order rather than argument semantics. Fresh evidence in this revision is that each turn reparses tool arguments from model JSON (json.loads at lines 2637-2646), so the same call with reordered keys (e.g. {"q":...,"limit":...} then {"limit":...,"q":...}) produces different keys and bypasses the "previous successful call" guard, allowing repeated expensive or side-effectful tool executions.

Useful? React with 👍 / 👎.

result = (
"You already made this exact call. "
"Do not repeat the same tool call. "
Expand Down Expand Up @@ -2734,7 +2719,7 @@ def _record_tool_call(name: str, args: dict, failed: bool) -> None:
_is_error = isinstance(result, str) and result.lstrip().startswith(
_error_prefixes
)
_record_tool_call(tool_name, arguments, failed = _is_error)
_tool_call_history.append((_tc_key, _is_error))
_result_content = result
if _is_error:
_result_content = (
Expand Down
Loading