Skip to content

fix: polish WebUI assistant replies - #3138

Merged
1 commit merged into
nesquena:masterfrom
AJV20:fix/final-answer-polish
May 29, 2026
Merged

1 commit merged into
nesquena:masterfrom
AJV20:fix/final-answer-polish

Conversation

@AJV20

@AJV20 AJV20 commented May 29, 2026

Copy link
Copy Markdown
Contributor

Thinking Path

  • Browser chat should preserve normal Hermes answer quality while still allowing concise progress updates during tool-heavy work.
  • Terse planning fragments such as "Need check logs" are useful as internal scratchpad shorthand, but they are confusing when persisted as visible assistant text.
  • The fix should be prompt-level and narrowly scoped before adding any aggressive output sanitizer.

What Changed

  • Tightens the WebUI progress guidance so visible assistant text must not include terse scratchpad/planning fragments.
  • Reuses the same progress guidance in the synchronous /api/chat path so both WebUI chat paths get the same final-answer polish instruction.
  • Adds regression coverage asserting the WebUI ephemeral prompt includes the new final-answer polish guidance.
  • Adds a changelog entry for the user-visible behavior change.

Why It Matters

Users should see clear, readable final answers and progress updates, not internal shorthand such as "Need inspect email" or "maybe invite" at the start of assistant replies.

Verification

  • python -m py_compile api/routes.py api/streaming.py
  • python -m pytest tests/test_webui_surface_context.py -q
  • git diff --check

Risks / Follow-ups

  • This is intentionally prompt-level guidance, not a content sanitizer. If models still emit scratchpad fragments, a follow-up can add a carefully bounded sanitizer after more examples are collected.

@nesquena-hermes

Copy link
Copy Markdown
Collaborator

Summary

Read the diff on PR #3138 plus api/streaming.py (the home of _WEBUI_PROGRESS_PROMPT), api/routes.py:_handle_chat_sync (the new prompt-injection callsite), and agent/conversation_loop.py on the agent side to confirm what the prompt actually does. The change is correct in spirit and small: one extra bullet in the WebUI progress guidance forbidding scratchpad fragments like "Need check logs", and a re-export so the otherwise-unused sync chat path gets the same guidance. The test additions assert the new strings end up in the ephemeral prompt build.

Code reference

api/streaming.py:199-211 (PR HEAD):

_WEBUI_PROGRESS_PROMPT = """
WebUI progress guidance:
- ...
- Do not include terse planning fragments or scratchpad shorthand in visible
  assistant text. Avoid fragments like "Need check logs", "Need inspect
  email", or "maybe invite"; either omit them or rewrite them as clear
  user-facing progress.
- For direct answers or very short tasks, skip progress updates and answer normally.
""".strip()

And the new sync-path injection at api/routes.py:9628-9651:

from api.streaming import (
    _WEBUI_PROGRESS_PROMPT,
    ...
)
workspace_system_msg = (
    f"Active workspace at session start: {s.workspace}\n"
    ... existing workspace tag instructions ...
    "Never fall back to a hardcoded path when this tag is present.\n\n"
    f"{_WEBUI_PROGRESS_PROMPT}"
)

Diagnosis

Two observations worth raising before merge — neither is a blocker.

1. Streaming vs sync path use different injection mechanisms. In the streaming path the progress prompt is set on agent.ephemeral_system_prompt (api/streaming.py:4955), which agent/conversation_loop.py:966-967 appends to the effective system every turn but never persists. In the sync path it's now baked into workspace_system_msg, which is passed as system_message to run_conversation and on first turn ends up in agent._cached_system_prompt = agent._build_system_prompt(system_message) (agent/conversation_loop.py:286-288), which then gets persisted into the session DB's system_prompt column via _restore_or_build_system_prompt. So a session started on the sync path persists the progress guidance into the prefix cache row; a streaming session does not. The two paths converge for behavior but diverge for caching semantics. In practice the sync endpoint is documented as "Fallback synchronous chat endpoint (POST /api/chat). Not used by frontend." (api/routes.py:9551), so the divergence is mostly theoretical. If you ever want parity, the cleaner shape is to also set agent.ephemeral_system_prompt on the sync path instead of inlining into system_message.

2. The guard is purely a prompt-level instruction. The PR description acknowledges this and that's correct — there's no output sanitizer. Models that ignore the instruction will still emit "Need inspect email" as visible text. The PR is the right first fix (cheapest place to enforce), but expect to need a small bounded sanitizer (regex-match a leading "Need " or "maybe " fragment at the start of a visible delta and either drop it or treat it as scratchpad). Worth filing as a follow-up issue rather than expanding this PR.

Test coverage

tests/test_webui_surface_context.py:25-27 covers the streaming path through _webui_ephemeral_system_prompt, asserting the three new strings:

assert "Do not include terse planning fragments" in prompt
assert "Need inspect email" in prompt
assert "clear user-facing progress" in prompt

That confirms the prompt builder includes the new bullet, but it doesn't exercise the sync path's workspace_system_msg construction. A quick parallel test that builds the sync workspace message and asserts _WEBUI_PROGRESS_PROMPT in workspace_system_msg would lock the import-side of the sync change in too. Cheap to add.

CI green across 3.11/3.12/3.13. The risk profile is genuinely low. LGTM with the caching-semantics note above as a follow-up.

@nesquena-hermes nesquena-hermes closed this pull request by merging all changes into nesquena:master in 1ebd160 May 29, 2026
AJV20 pushed a commit to AJV20/hermes-webui that referenced this pull request May 29, 2026
# Conflicts:
#	CHANGELOG.md
SysAdminDoc pushed a commit to SysAdminDoc/hermes-webui that referenced this pull request Jun 26, 2026
# Conflicts:
#	CHANGELOG.md
bernyforce pushed a commit to bernyforce/hermes-webui that referenced this pull request Jul 29, 2026
# Conflicts:
#	CHANGELOG.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants