Skip to content

fix(gateway): suppress hidden-only incomplete Codex turns - #51657

Closed
LeonSGP43 wants to merge 1 commit into
NousResearch:mainfrom
LeonSGP43:fix/51628-gateway-incomplete-turn-safety
Closed

fix(gateway): suppress hidden-only incomplete Codex turns#51657
LeonSGP43 wants to merge 1 commit into
NousResearch:mainfrom
LeonSGP43:fix/51628-gateway-incomplete-turn-safety

Conversation

@LeonSGP43

Copy link
Copy Markdown
Contributor

Closes #51628

Summary

  • detect retry-exhausted hidden-reasoning-only Codex turns before gateway fallback text is emitted
  • persist only the user turn for those incomplete gateway runs so collaboration threads do not ingest a fake assistant reply
  • document safer Slack reasoning-effort guidance for Codex-backed peer-agent channels

Testing

  • uv run --frozen pytest -q tests/gateway/test_incomplete_gateway_turns.py -o addopts=''
  • uv run --frozen ruff check gateway/run.py tests/gateway/test_incomplete_gateway_turns.py
  • git diff --check

@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery provider/openai OpenAI / Codex Responses API P2 Medium — degraded but workaround exists sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages labels Jun 24, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for targeting a real gateway/session-safety failure. The current patch needs rework against current main.

Problems

  • agent/conversation_loop.py:4436 returns retry exhaustion with a non-empty final_response. gateway/run.py:2604 returns any non-empty response before the new classifier can run, and the proposed helper also rejects non-empty agent_result["final_response"]. The suppression path therefore cannot match the production result shape.
  • tests/gateway/test_incomplete_gateway_turns.py fabricates final_response=None and mocks _run_agent, so it does not exercise that current path.
  • The later #51628 reproduction reports Codex Responses stream did not emit a terminal response; the proposed exact matcher only recognizes remained incomplete after.

Suggested changes

  • Classify this structured incomplete/no-visible-answer state before the non-empty fallback is returned, then add an end-to-end gateway persistence/delivery test using the current result shape.
  • Cover the no-terminal-stream reproduction or use a provider-agnostic structured criterion.

Automated hermes-sweeper review.

Comment thread gateway/run.py
if not agent_result.get("partial"):
return False
error_text = str(agent_result.get("error", "") or "").lower()
return "remained incomplete after" in error_text

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This predicate cannot match the current retry-exhaustion result: agent/conversation_loop.py:4436 sets this same diagnostic as a non-empty final_response, while the helper rejects non-empty final_response and the normalizer returns non-empty response at gateway/run.py:2604 before reaching this code. Please classify the structured incomplete/no-visible-output state before that early return and test the real result shape.

@teknium1 teknium1 added sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 15, 2026
teknium1 added a commit that referenced this pull request Jul 15, 2026
… final_response

Follow-up to the salvaged #51657: the conversation loop returns the
retry-exhaustion sentinel as BOTH final_response and error, so the
original detector (which required final_response to be falsy) never
fired on real exhaustion turns — the sentinel text was delivered
verbatim into the channel, exactly the #51628 poisoning vector. Detect
the sentinel echo, blank it before empty-response normalization, and
never suppress a turn whose final_response is genuine model text.
Also: dedupe-guard mock fix in the test fixture (has_platform_message_id
must return False, not a truthy MagicMock) and two guard tests
(real answer never suppressed; interrupted/failed never classified).
@teknium1

Copy link
Copy Markdown
Contributor

Merged via PR #65062 — your commit was cherry-picked onto current main with your authorship preserved in git log (rebase merge). One substantive follow-up on top: the detector required final_response to be falsy, but the conversation loop returns the retry-exhaustion sentinel as BOTH final_response and error — so as written it never fired on real exhaustion turns. The hardened version recognizes the sentinel echo (and never suppresses a genuinely different final text), blanks it at the delivery site, and your test now exercises the real loop shape. The suppression + user-only persistence design was exactly right for the #51628 poisoning vector — thanks for the careful gateway work.

@teknium1 teknium1 closed this Jul 15, 2026
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
… final_response

Follow-up to the salvaged NousResearch#51657: the conversation loop returns the
retry-exhaustion sentinel as BOTH final_response and error, so the
original detector (which required final_response to be falsy) never
fired on real exhaustion turns — the sentinel text was delivered
verbatim into the channel, exactly the NousResearch#51628 poisoning vector. Detect
the sentinel echo, blank it before empty-response normalization, and
never suppress a turn whose final_response is genuine model text.
Also: dedupe-guard mock fix in the test fixture (has_platform_message_id
must return False, not a truthy MagicMock) and two guard tests
(real answer never suppressed; interrupted/failed never classified).
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
… final_response

Follow-up to the salvaged NousResearch#51657: the conversation loop returns the
retry-exhaustion sentinel as BOTH final_response and error, so the
original detector (which required final_response to be falsy) never
fired on real exhaustion turns — the sentinel text was delivered
verbatim into the channel, exactly the NousResearch#51628 poisoning vector. Detect
the sentinel echo, blank it before empty-response normalization, and
never suppress a turn whose final_response is genuine model text.
Also: dedupe-guard mock fix in the test fixture (has_platform_message_id
must return False, not a truthy MagicMock) and two guard tests
(real answer never suppressed; interrupted/failed never classified).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists provider/openai OpenAI / Codex Responses API sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Codex hidden-reasoning-only incomplete turns can poison gateway sessions

3 participants