Skip to content

fix(tui): keep model switch markers out of API history - #48356

Open
markoub wants to merge 2 commits into
NousResearch:mainfrom
markoub:markoub/tui-model-switch-session-meta
Open

fix(tui): keep model switch markers out of API history#48356
markoub wants to merge 2 commits into
NousResearch:mainfrom
markoub:markoub/tui-model-switch-session-meta

Conversation

@markoub

@markoub markoub commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes TUI model-switch metadata so it cannot become a mid-conversation system message in provider API history.

When /model changes the active runtime, Hermes already refreshes and persists the session system prompt. This PR keeps the additional marker as transcript-only session_meta and filters both new session_meta rows and legacy model-switch system markers before handing history to AIAgent.

The merge path then reconciles the agent's append-style result back onto the raw TUI transcript so transcript-only rows are preserved locally without being sent to the provider.

This covers new switches and already-persisted markers from older versions that could trigger strict OpenAI-compatible backends such as vLLM/Qwen to return HTTP 400.

Related Issue

Fixes #48338

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • Tests (adding or improving test coverage)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Performance improvement
  • Code refactoring

Changes Made

  • tui_gateway/server.py: store model-switch markers as session_meta, not system.
  • tui_gateway/server.py: filter transcript-only session_meta rows and legacy TUI model-switch system markers before prompt.submit calls AIAgent.
  • tui_gateway/server.py: merge append-style agent results back onto the raw TUI transcript so filtered local metadata is not lost.
  • tests/test_tui_gateway_server.py: cover new session_meta markers, legacy system model-switch markers, preserved ordinary system messages, and transcript merge behavior.

How to Test

  1. Switch model in an active TUI session, then send another prompt through a strict OpenAI-compatible backend such as vLLM/Qwen.
  2. Confirm the provider request contains the rebuilt leading system prompt but no model-switch marker as a later system message.
  3. Run:
    • uv run scripts/run_tests.sh tests/test_tui_gateway_server.py -- -k 'model_switch or config_set_model or preserves_transcript_meta' --tb=long
    • uv run scripts/run_tests.sh tests/run_agent/test_session_meta_filtering.py -- --tb=long
    • git diff --check && uv run ruff check tui_gateway/server.py tests/test_tui_gateway_server.py

Checklist

Code

  • I've read the Contributing Guide and this PR follows the project's coding standards
  • My commit messages follow the conventional commits format
  • I searched for existing PRs/issues to avoid duplicates
  • My PR contains only changes related to the described issue
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for new functionality or bug fixes
  • I've tested on my platform: macOS 15 / Python 3.11 via uv run scripts/run_tests.sh

Documentation

  • I've updated relevant documentation where needed, or this change does not require documentation updates
  • I've updated cli-config.yaml.example if adding config options, or this change does not add config options
  • I've updated CONTRIBUTING.md or AGENTS.md if changing development workflow, or this change does not affect development workflow
  • I've considered cross-platform impact: no process, shell, or path behavior changed
  • I've updated tool descriptions/schemas if adding or changing tools, or this change does not affect tool schemas

For New Skills

N/A

Screenshots / Logs

uv run scripts/run_tests.sh tests/test_tui_gateway_server.py -- -k 'model_switch or config_set_model or preserves_transcript_meta' --tb=long
passed 12 tests.

uv run scripts/run_tests.sh tests/run_agent/test_session_meta_filtering.py -- --tb=long
passed 5 tests.

git diff --check && uv run ruff check tui_gateway/server.py tests/test_tui_gateway_server.py
passed.

Note: I also ran the full tests/test_tui_gateway_server.py file. It passed 277/278 tests; the only failure was the unrelated local browser-launch assertion test_browser_manage_connect_default_local_reports_launch_hint, which also fails when run alone in this checkout.

@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery comp/tui Terminal UI (ui-tui/ + tui_gateway/) provider/qwen Qwen / Alibaba Cloud (OAuth) P2 Medium — degraded but workaround exists labels Jun 18, 2026
…tch-session-meta-repair

# Conflicts:
#	tests/test_tui_gateway_server.py
#	tui_gateway/server.py

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for separating transcript-only model-switch metadata from provider-visible history. The current-main premise remains real: tui_gateway/server.py:2448 persists a switch marker as a user message, and _run_prompt_submit passes raw history to AIAgent at tui_gateway/server.py:9048.

Problems

  • The new legacy filter at tui_gateway/server.py:4663 only removes markers whose role is system. Current main changed new markers to user in commit 61622bb (tui_gateway/server.py:2444-2470), so already-persisted user-role model-switch markers remain in the provider history after this change.

Suggested changes

  • Recognize the exact model-switch marker independent of its stored role, then filter legacy user and system rows plus new session_meta rows. Add a test covering a persisted user-role marker without filtering ordinary user turns.

Automated hermes-sweeper review.

Comment thread tui_gateway/server.py
# provider-visible instructions.
if msg.get("role") == "session_meta":
continue
if msg.get("role") == "system" and _is_model_switch_marker(msg):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current main has persisted model-switch markers as role="user" since 61622bb. This condition removes only older system rows, so those user-role markers still reach conversation_history; recognize the exact marker content independently of role.

@teknium1 teknium1 added sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery comp/tui Terminal UI (ui-tui/ + tui_gateway/) P2 Medium — degraded but workaround exists provider/qwen Qwen / Alibaba Cloud (OAuth) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: _append_model_switch_marker injects role:"system" mid-conversation → HTTP 400 on strict providers (vLLM, Qwen)

3 participants