[codex] Fix workspace-prefixed user turn leakage

[swftwolfzyq]

Summary

• prevent internal [Workspace::v1: ...] metadata from leaking into the visible user transcript when a provider failure/retry path echoes draft text before the workspace-prefixed prompt
• normalize these echoed current-user turns back to the submitted human prompt before display persistence
• detect silent/no-response failures using only the current turn delta instead of counting assistant messages from prior history

Root cause

WebUI sends Hermes a workspace-prefixed user message for model context while the UI transcript should keep only the human text. In failed provider paths, returned messages can contain a partial optimistic draft followed by the internal workspace prefix and the full prompt. The existing dedupe logic only stripped prefixes at the beginning of a user message, so the internal tag could be persisted visibly. The no-response guard also checked all result messages, allowing prior assistant replies to mask a failed current turn.

Tests

• ~/.local/bin/uv run --with pytest pytest tests/test_issue1217_transcript_compaction.py tests/test_workspace_display_prefix.py tests/test_issues_373_374_375.py tests/test_session_save_mode.py tests/test_issue1361_cancel_data_loss.py -q (71 passed)
• ~/.local/bin/uv run --with pytest pytest tests/test_issue1217_transcript_compaction.py tests/test_workspace_display_prefix.py -q (13 passed)
• python -m py_compile api/streaming.py

[nesquena-hermes]

@swftwolfzyq These have been sitting as draft for 4 days. If you'd like maintainer review, please:

1. Flip out of draft (or comment confirming you'd like a draft review)
2. Confirm the diff is at its final shape

Otherwise we'll close in a few days to keep the queue clean — you can reopen anytime. Both PRs touch interesting surfaces (#2145 the workspace-prefix display leakage, #2146 the release-tag-based update banner), so they're worth a look once you've signaled they're ready.

[swftwolfzyq]

Ready for review now. I’ve resolved the latest merge conflicts and consider the diff to be in its final shape for maintainer review.

[nesquena-hermes]

@nesquena Ready signal from @swftwolfzyq received — moving out of hold. This PR is now in the independent-review queue for the next stage batch.

CI is green on all three Python versions (3.11/3.12/3.13), mergeable: MERGEABLE, the diff touches api/streaming.py (sensitive seam) plus a new regression test on tests/test_issue1217_transcript_compaction.py. Worth a careful look at the workspace-prefix display-leakage path before merge.

[nesquena]

Review — end-to-end ✅ (clean APPROVE, no fix pushed)

What this ships

External-contributor PR (codex/workspace-prefix-display-fix branch — @swftwolfzyq, OpenAI Codex bot). Two narrow fixes in api/streaming.py:

1. Workspace-prefix leakage — Stop the internal [Workspace::v1: /path] metadata tag from leaking into the visible user transcript when a provider failure/retry path echoes an optimistic draft followed by the workspace-prefixed prompt.
2. Silent-failure detection — Only consider messages after the current user turn when deciding whether the agent produced an assistant reply, so prior assistant messages from earlier turns can't mask a current-turn silent failure.

Surfaces touched: api/streaming.py (+72/-11), tests/test_issue1217_transcript_compaction.py (+91 — 3 new tests).

Traced against upstream hermes-agent

Pulled a fresh tarball. No cross-tool surface — both fixes live entirely in webui-internal message-merge / failure-detection helpers. Agent receives the workspace-prefixed message as before; only the WebUI's display-merge and silent-failure detection are touched. No config.yaml, no session schema, no AIAgent param change.

End-to-end trace

New _looks_like_current_user_turn(msg, msg_text) at api/streaming.py:996-1014:

needle = " ".join(str(msg_text or '').split())
if not needle: return False
text = _message_text(msg.get('content', ''))
candidates = [_strip_workspace_prefix(text, include_legacy=True)]   # leading-strip
for pattern in (_WORKSPACE_PREFIX_ANY_RE, _LEGACY_WORKSPACE_PREFIX_ANY_RE):
    for match in pattern.finditer(text):
        candidates.append(text[match.end():])                       # post-prefix tail
return any(" ".join(str(c or '').split()) == needle for c in candidates)

• Whitespace-normalized exact-match comparison against the submitted prompt.
• Considers both the leading-prefix-stripped variant (existing path) AND the text after any embedded workspace prefix (new path).
• Returns False for non-user roles, empty prompts, or any text whose post-prefix tail doesn't exactly match.

New regex _WORKSPACE_PREFIX_ANY_RE and _LEGACY_WORKSPACE_PREFIX_ANY_RE at lines 977-978 — same shape as the existing leading-only regex but without the ^ anchor. ReDoS-safe: alternation \\.|[^\]\\] is deterministic (escaped char OR a non-]-non-\ char; no overlap).

**Updated _find_current_user_turn at api/streaming.py:2138 — adds the new check BEFORE the existing strip-then-compare path:

if _looks_like_current_user_turn(msg, msg_text):
    return idx
# ... existing _strip_workspace_prefix + exact-match logic ...

Shapes the leading-strip caught still hit (since candidates includes the leading-stripped variant first). Shapes that previously fell through to fallback now correctly identify the current user turn.

Updated _merge_display_messages_after_agent_result at lines 2283-2348 — adds _looks_like_current_user_turn checks in three places:

1. "current_user_already_checkpointed" guard (whether the previous merged tail already represents the current turn)
2. "skip current turn already added" dedupe (when the same identity OR leaked-prefix variant arrives a second time)
3. Display-content rewrite — when the message IS the current turn, copy and overwrite content = msg_text so the visible transcript shows the clean prompt regardless of what the provider echoed.

New _assistant_reply_added_after_current_turn(result_messages, previous_context, msg_text) at lines 2361-2375:

if _messages_have_prefix(result_messages, previous_context):
    candidates = result_messages[len(previous_context):]     # append-only normal case
else:
    current_user_idx = _find_current_user_turn(result_messages, msg_text)
    candidates = result_messages[current_user_idx + 1:] if current_user_idx is not None else result_messages
return any(
    m.get('role') == 'assistant' and not m.get('_error') and str(m.get('content') or '').strip()
    for m in candidates
)

Two cases:

• Append-only (typical): candidates = the delta after previous_context.
• Compacted/replayed: locate the current user turn, look at what comes after.

Returns True only if at least one non-error, non-empty assistant message exists in the post-current-turn slice.

Updated call site at lines 4001-4009:

_all_result_messages = result.get('messages') or []
_prev_len = len(_previous_context_messages)
_assistant_added = _assistant_reply_added_after_current_turn(
    _all_result_messages,
    _previous_context_messages,
    msg_text,
)

Replaces the old _has_new_assistant_reply(_all_result_messages, _prev_len). _has_new_assistant_reply is still in the module (line 159) and still called from the self-heal path at line 4095 — kept as-is for the heal flow which has a different shape.

Other audit — things that are correct already

Security

• Pure Python text + regex. No SQL, shell, path, XSS surface. The regex (?:\\.|[^\]\\])+ is deterministic (escaped-char OR non-]-non-\ char with no overlap); no ReDoS catastrophic backtracking.
• msg_text and message content come from the WebUI's own session storage; admin-controlled. No untrusted input flows here.

Backward compat

• The new _looks_like_current_user_turn is OR'd with the existing _message_identity == current_user_key check at every merge call site. Messages without leak still match via identity. No regression.
• _has_new_assistant_reply kept intact for the heal path; its tests at tests/test_silent_failure_detection.py still cover its behavior. Existing call sites in tests still work.

False-positive guard

• The whitespace-normalized exact-match requirement means a message matches only when the post-prefix tail EQUALS the prompt verbatim. Cannot accidentally match a different message that happens to contain the workspace prefix.

Display rewrite at line 2348

• copy.deepcopy(msg) + display_msg['content'] = msg_text — ensures the visible transcript shows the clean prompt. Original message dict not mutated; only the merged copy. Idempotent.

Test coverage (3 new tests):

1. test_embedded_workspace_prefixed_current_user_delta_is_deduped — leaked-prefix message in candidates, previous_display already contains the cleaned current turn → merge result equals previous_display (no duplicate added, no leaked tag visible).
2. test_embedded_workspace_prefixed_current_user_delta_displays_clean_prompt — leaked-prefix message + new assistant reply → merge shows clean prompt + assistant, no Workspace::v1 in any output.
3. test_assistant_added_detection_ignores_prior_history — pre-context has an assistant; current-turn delta has only user; helper correctly reports False (silent failure), but True when assistant is appended.

Behavioural harness

12-case Python harness exercising both new helpers:

=== _looks_like_current_user_turn ===
1  clean (no prefix)               → True   ✓
2  leading workspace prefix        → True   ✓
3  embedded prefix (bug shape)     → True   ✓
4  legacy workspace prefix         → True   ✓
5  different post-prefix content   → False  ✓ (false-positive guard)
6  assistant role                  → False  ✓ (role guard)
7  empty prompt                    → False  ✓
8  whitespace normalization        → True   ✓

=== _assistant_reply_added_after_current_turn ===
9  normal flow (assistant added)   → True   ✓
10 silent failure (no assistant)   → False  ✓
11 empty-content assistant         → False  ✓
12 _error-flagged assistant        → False  ✓

All 12 cases pass. The new helpers correctly identify the bug shape, reject false-positive shapes, and handle all the failure-mode permutations.

Edge-case trace

Scenario	Expected	Actual
Clean current-turn user message	identified	✅ harness 1
Leading [Workspace::v1:...] prefix	identified, displayed clean	✅ harness 2
Embedded (mid-text) workspace prefix + clean tail	identified, displayed clean	✅ harness 3 + test 1
Legacy [Workspace:...] prefix	identified	✅ harness 4
Message with prefix but different content	NOT identified (no false positive)	✅ harness 5
Assistant role with prompt content	NOT identified	✅ harness 6
Empty / whitespace prompt	not identified	✅ harness 7
Whitespace-normalized post-prefix tail	identified	✅ harness 8
Append-only flow with assistant	reply detected	✅ harness 9
Append-only flow, NO assistant	silent failure detected	✅ harness 10 + test 3
Empty-content assistant	not counted as reply	✅ harness 11
_error=True assistant	not counted	✅ harness 12
Compacted result (not append-only)	locates current turn, scans post-tail	✅ _messages_have_prefix fallback
Cross-tool: agent sees same workspace-prefixed prompt	unaffected	✅ webui-only display logic

Tests

• PR-targeted: 64/64 pass (test_issue1217_transcript_compaction.py, test_workspace_display_prefix.py, test_issues_373_374_375.py, test_session_save_mode.py).
• test_silent_failure_detection.py (existing tests for _has_new_assistant_reply): 7/7 pass — the kept-as-is helper still works for the heal path.
• Full local suite (Python 3.14): 5713 passed, 63 skipped, 3 xpassed, 0 failed.
• CI: not run (gh pr checks 2145 returned "no checks reported on the 'codex/workspace-prefix-display-fix' branch"). Same situation as PR #2146 — Codex bot branch, GitHub Actions didn't trigger.

Minor observations (non-blocking)

1. Heal-path detection NOT updated. The self-heal _heal_ok check at line 4095 still uses the older _has_new_assistant_reply(_heal_all_msgs, _prev_len) or _token_sent. If the same workspace-leak shape could appear in heal results, the heal path would mis-detect. PR scopes the fix narrowly to the reported bug path; updating heal could be a follow-up but isn't blocking.
2. _has_new_assistant_reply retained for heal path only — single remaining production caller, plus its own test file. Acceptable; the function isn't dead code.
3. _looks_like_current_user_turn is O(n*m) in worst case (finditer over text + comparison per match). In practice n and m are small (single-message text, few prefix matches). Not a hot path.
4. No CI runs. Same as PR #2146 — Codex-bot branch is older than the workflow-file change window. Maintainer should re-trigger or rebase before merge.
5. No CHANGELOG entry. Release agent stamps at merge time per project convention.
6. External Codex-bot PR. Code quality is high: clean separation of concerns, narrow scope, good test naming, double-quote/Unicode-aware test strings for the Chinese-prompt bug repro. Reasonable contribution; encourage.

Recommendation

Approved. Both fixes are correct, surgical, and well-tested. The workspace-leak fix has a precise regex shape that's ReDoS-safe and a defensive whitespace-normalized exact-match comparison. The silent-failure detection correctly distinguishes append-only vs compacted result shapes. Behavioural harness confirms all 12 input permutations.

✅ Parked at approval — ready for the release agent's merge/tag pipeline.