fix(repair): Pass 1.5 must match tool results on call_id, not just id - #547
Merged
Conversation
`_repair_message_sequence` Pass 1.5 resolved a tool_call's answered budget
against `tc.get("id")` ONLY, while Pass 1 correctly registers the
`id`/`call_id` SUPERSET (NousResearch#58168). An assistant turn whose tool_call carries
only `call_id` (or a Codex-Responses call whose `id` and `call_id` differ)
therefore read as entirely unanswered even when the following `tool` message
answered it -- and the "none answered" branch DELETED the valid assistant
turn.
Repro on fork/main:
[user, assistant(tool_calls=[{call_id: "call_XYZ"}]),
tool(tool_call_id="call_XYZ"), user]
repairs -> 1 (expected 0); roles -> ['user','tool','user']
That leaves a stray tool result with no preceding tool_calls -- the exact
HTTP 400 shape this pass exists to prevent.
Fix: iterate ("id", "call_id") when consuming the answered budget, mirroring
Pass 1. Also removes the two `@pytest.mark.xfail(strict=False)` markers that
were parked on the upstream tests pending this fix, so they become real gates.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Symptom
AIAgent._repair_message_sequencedeletes a validassistant(tool_calls)turn whose tool_call carries onlycall_id(noid), even though the immediately-followingtoolmessage answers it correctly.Reproduced on
fork/main(0b2a95966f) before any change:The result is a stray
toolmessage with no precedingtool_calls— precisely the HTTP 400 shape this repair exists to prevent ("Messages with role 'tool' must be a response to a preceding message with 'tool_calls'" on DeepSeek/Kimi; "tool_useids were found withouttool_resultblocks" on Anthropic).Root cause
agent/agent_runtime_helpers.py:827(pre-fix):Pass 1 registers the answer budget against the superset of both keys (
for key in ("id", "call_id"), per NousResearch#58168) — but Pass 1.5 consumes that budget againsttc.get("id")alone. A tool_call carrying onlycall_id(or a Codex-Responses call whoseid=fc_...andcall_id=call_...differ) therefore matches nothing,answered_flagsis all-False, and the "NONE answered — the whole tool-call turn is an orphan" branch drops the turn.This is pre-existing on
fork/main, not parity-merge damage. The 2026-08-07 parity merge only surfaced it by restoring the upstream tests that catch it.Fix
Minimal and surgical — Pass 1.5 now consumes the budget on
idORcall_id, mirroring Pass 1. No other pass touched:The multiset/count semantics are preserved exactly (each entry consumes at most one result slot, so a
[X, X]turn answered once still keeps exactly one X), and thebreakkeeps a single tool_call from consuming two slots whenidandcall_idare both present and both answered.RED -> GREEN proof
RED first. New focused regression test
test_repair_pass_1_5_keeps_call_id_only_assistant_turnwritten and run before any source change:GREEN after the fix —
scripts/run_tests.sh(canonical parallel runner), exact observed counts:scripts/run_tests.sh tests/run_agent/test_message_sequence_repair.py -qscripts/run_tests.sh tests/run_agent -qscripts/run_tests.sh tests/agent -qruff check(both changed files)MUTATION PROOF
Explicitly performed. Reverting only the source change (
git checkout HEAD~1 -- agent/agent_runtime_helpers.py, leaving the tests in place) and re-running the file:Restoring the source (
git checkout HEAD -- agent/agent_runtime_helpers.py) returns the file to 26 passed, 0 failed. The tests fail without the fix and pass with it — they are real gates on this exact behavior, not vacuous greens.Removed xfail markers
Both
@pytest.mark.xfail(strict=False)markers intests/run_agent/test_message_sequence_repair.py— parked on the two upstream tests pending exactly this fix — are deleted, so those tests now gate for real:test_repair_keeps_tool_matching_codex_call_id(distinctid+call_id)test_repair_keeps_tool_matching_only_call_id(call_idonly)The now-unused
import pytestwas dropped with them (ruff-clean). Note both werestrict=Falseand had begun xpassing for thecodex_call_idcase in some orderings — as markers they proved nothing; the mutation run above shows they genuinely fail against the unfixed source.Files changed
agent/agent_runtime_helpers.py(+18 / -6) — Pass 1.5id/call_idsuperset match + explanatory commenttests/run_agent/test_message_sequence_repair.py(+30 / -39) — new regression test; two xfail markers removed