Skip to content

fix(agent): ignore quoted scratchpad tags in retry detection - #11743

Open
sgaofen wants to merge 1 commit into
NousResearch:mainfrom
sgaofen:codex/scratchpad-false-positive
Open

fix(agent): ignore quoted scratchpad tags in retry detection#11743
sgaofen wants to merge 1 commit into
NousResearch:mainfrom
sgaofen:codex/scratchpad-false-positive

Conversation

@sgaofen

@sgaofen sgaofen commented Apr 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • stop has_incomplete_scratchpad() from treating quoted or code-form scratchpad tag mentions as an unfinished reasoning block
  • strip fenced code blocks, inline code, and blockquotes before checking for unclosed <REASONING_SCRATCHPAD> tags
  • add focused regression coverage for real incomplete tags plus the false-positive markdown cases

Root Cause

The retry guard in agent/trajectory.py used a plain substring check for <REASONING_SCRATCHPAD> without understanding markdown context. That meant grep output, quoted user text, or inline code containing the tag could trigger the incomplete-scratchpad path, causing wasted retries and dropped output.

Closes #11663.

Validation

  • python3 -m py_compile agent/trajectory.py tests/agent/test_trajectory.py
  • uv run --extra dev pytest -o addopts='' tests/agent/test_trajectory.py -q

Platform Tested

  • macOS 15.x (Apple Silicon)

Contribution Guide Notes

  • Reviewed CONTRIBUTING.md and checked for existing open PRs before submitting this scoped change.
  • Ran the targeted verification commands listed above for this PR. I have not claimed a full repo-wide pytest tests/ -q pass unless explicitly noted.

@trevorgordon981

Copy link
Copy Markdown
Contributor

LGTM. Switching from presence to count-based comparison and stripping markdown context before checking is the right approach. Tests cover the key false-positive vectors (fenced, inline, blockquote) and the "real tag after quote" case pins the true-positive path.

One minor edge case to be aware of (not blocking): _BLOCKQUOTE_RE only matches single blockquote lines. A multi-line continuation blockquote like:

> first line
> <REASONING_SCRATCHPAD>

is handled correctly since each > line matches independently. But a bare continuation-style blockquote without repeated > markers on subsequent lines would not be stripped. In practice this probably never fires since models/users don't tend to produce bare continuation blockquotes containing scratchpad tags.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Resilience] has_incomplete_scratchpad false positive causes wasted retries and lost output

3 participants