Skip to content

feat(gateway,desktop): per-turn model note + display-only mode label (composer-mode frame) - #108242

Closed
LisandroNahuelH wants to merge 14 commits into
NousResearch:mainfrom
LisandroNahuelH:feat/composer-mode-note
Closed

LisandroNahuelH wants to merge 14 commits into
NousResearch:mainfrom
LisandroNahuelH:feat/composer-mode-note

Conversation

@LisandroNahuelH

@LisandroNahuelH LisandroNahuelH commented Sep 11, 2026 •

Copy link
Copy Markdown

What does this PR do?

Adds an optional per-turn frame (note + mode) to the turn RPCs, so a client can frame ONE send — a Cursor-style composer mode, an internal note, a surface hint — without typing it into the user's message:

  • note (sanitized + capped via parse_turn_note) is a model instruction for THIS send. It is delivered through the per-turn api_content sidecar — the same channel the existing per-turn notes use (/steer markers, speech-interrupted, hud surface) — so the durable content and every bubble keep the user's own words.
  • mode is an opaque, display-only label stored in the row's display_metadata (popped from every outbound copy). It never reaches the wire and is never policy.

Why the core: only the backend knows which model/provider serves a turn, and a mid-turn steer never runs a client's composer middleware — so a plugin can only fake this by rewriting the draft (the note then lives in the transcript forever) or silently losing the mode on every steer.

Fixes #108241

Related work

Adjacent, still-open PRs on the same surfaces — cross-linked, no overlap:

Type of Change

  • ✨ New feature (non-breaking change that adds functionality)

Changes Made

Gateway (tui_gateway/)

  • methods_prompt.py — parse_turn_note(params) + prompt.submit threads turn_note/turn_mode through _run_after_agent_ready, the busy path, and the compute-host dispatch.
  • prompt_turn.py — _run_prompt_submit merges mode into display_metadata and prepends note to the run message in _prepare_turn_input (a mode-only turn still persists display_metadata).
  • session_auto_continue.py — _handle_busy_submit / _ac_try_correction / _enqueue_prompt carry the frame; a framed arrival keeps its OWN queue envelope (never merged into a plain slot) and survives _sanitize_queued_entry_vs_inflight_user; the drain hands it back to the turn.
  • methods_session.py — session.steer / session.redirect parse the frame and pass it to the agent (only when present, so agents without the kwargs keep working).
  • compute_host_bridge.py / compute_host.py — the isolated-turn frame carries note/mode.

Agent (agent/)

  • interrupt_control.py — steer() / redirect() accept note/mode, queue them beside their pending slot (_queue_correction_note, notes concatenate, label last-wins) and expose a one-shot _take_correction_note; interrupt/clear paths drop the frame with their slot.
  • prompt_builder.py (steer_user_row) — note → api_content, label → display_metadata.
  • conversation_loop.py (_apply_active_turn_redirect) — note prepended to the correction's api_content, label carried on the row.
  • agent_runtime_helpers.py / turn_finalizer.py — the steer delivery takes the frame one-shot; a requeued or leftover steer keeps/drops it explicitly, so a stale frame can never attach to a later correction.

Desktop (apps/desktop/src/)

  • composer/contrib.ts + composer/index.tsx — ComposerModeFrame / ComposerDraft.note|mode, forwarded into SubmitTextOptions → prompt.submit {note, mode}.
  • use-prompt-actions/index.ts (redirectPrompt) — runs runComposerMiddleware exactly ONCE, before the RPC and before the session-not-found retry; returns 'canceled' for a cancel (never confused with a rejection, which keeps queueing the words).
  • session-tile-actions.ts (steerPrompt) — same, BEFORE the optimistic append so a cancel leaves no bubble behind.
  • use-composer-submit.ts (steerDraft) — 'canceled' restores the draft (loadIntoComposer) and queues nothing; use-composer-queue.ts (steerQueuedNow) consumes a queued entry only on a delivered redirect (accepted === true) — 'canceled' is a truthy string, so the old truthiness test silently ate the queued words.
  • store/composer-queue.ts + use-composer-queue.ts — a queued entry OWNS its frame: queueCurrentDraft runs the chain once at enqueue and seals {mode, note} onto the QueuedPromptEntry; the foreground drain, the background drain, and steerQueuedNow hand the sealed frame back as submit options with fromQueue, and the wrapper passes note/mode/fromQueue INTO the chain so the middleware hands the frame back untouched. Changing modes while entries are still queued no longer restamps every drain with the live mode (nor erases a queued Plan/Ask/Debug frame on an Agent switch).
  • composer/queue-frame.ts (sealQueuedFrame) + the enqueue paths — every requeue path seals: a steer rejected mid-turn (reconnect / settle race) used to requeue raw text (use-composer-submit fallback) — a frameless entry whose drain arrived note-less (observed live during a backend recycle). The chain runs once per seal; a cancel yields an empty frame — queueing never loses the words over a frame lookup.

Docs — apps/desktop/AGENTS.md, tui_gateway/AGENTS.md.

Reference client compatibility (composer-modes v12.1, verified)

The local plugin that motivated the frame now SHIPS the frame (its v10.10–v12 line,
independent of this change):

  • its composer.middleware derives {mode, note} per send — every send carries its
    own frame (Agent = no note, by design) and a queued drain passes the sealed entry
    frame through untouched (fromQueue short-circuits re-derivation). The wrapper and
    queue plumbing added here are what make that pass-through possible;
  • its direct prompt.submit {session_id, text, display_kind} (the plan-approve path)
    keeps working — the frame params are optional and only sent when present;
  • mid-turn steers now run the middleware chain (its v9 header documented the opposite
    as a known limit — the simple Enter during a live turn went straight to
    session.redirect, bypassing the chain). The mode now rides the steer instead of
    being dropped;
  • live-verified on the author's desktop client: a message queued in Ask and drained
    after switching the plugin to Agent kept content = the user's words while
    api_content carried the Ask note (the frame frozen at enqueue), and the 4-mode
    matrix (ask/plan/debug framed, agent clean) landed as designed;
  • the earlier migration path is DONE: the client emits {text, note, mode}, the
    transcript keeps the user's words, and the label lands in display_metadata;
  • v12.1 adds the STRICT ASK shield: the ask note is a full read-only contract
    (allowed/forbidden lists + a mandatory Spanish closing sentence on actionable
    requests) and rides BOTH ends of the send — api_content = note + text + note,
    ask-only (the sandwich this PR implements). A rejected steer's requeue seals its
    frame too (sealQueuedFrame), closing the one frameless path a live
    backend-recycle repro exposed;
  • session titles come from the pristine user text (agent._persist_user_message_override
    first), so API-only scaffolding — ask sandwich included — never becomes a sidebar title.

How to Test

  1. python -m pytest tests/tui_gateway/test_turn_note_frame.py -q → 7 passed (new contract suite; its 6 behaviour tests fail on the pre-fix tree).
  2. python -m pytest tests/run_agent/test_steer.py tests/test_tui_gateway_queue_on_busy.py tests/agent/test_api_content_sidecar.py tests/agent/test_gateway_turn_sidecar.py tests/agent/test_compression_busy_steer_anchor.py -q → 111 passed.
  3. cd apps/desktop && npx vitest run --project ui src/store/composer-queue.test.ts src/app/chat/composer/hooks/use-composer-queue.test.tsx src/app/chat/composer/hooks/use-composer-submit.test.tsx src/app/chat/composer/contrib.test.ts src/app/chat/session-tile-actions.test.ts src/app/session/hooks/use-prompt-actions/index.test.tsx src/app/session/hooks/use-message-stream/steer-arrival-order.test.tsx → 216 passed (the cancel-semantics cases, the queued-frame seal/drain cases, and the steer-fallback seal fail with the old logic). The gateway contract suite stays green with the ask sandwich added: bash scripts/run_tests.sh tests/tui_gateway/test_turn_note_frame.py → 8 passed.
  4. npx tsc -p apps/desktop/tsconfig.json --noEmit → clean; ruff check on the touched Python files → clean; eslint on the touched TS files → clean.
  5. Manual (needs a client that sends the frame): send a message in a mode and check the persisted row keeps the clean content (no note text); then type mid-turn (steer) in a mode and check the correction row carries the note in api_content and the label in display_metadata.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate (none found; feat: per-turn model note + display-only mode label on prompt.submit / steer / redirect #108241 opened as the feature record)
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I ran the full Python suite locally — scripts/run_tests.sh (the same runner tests.yml uses): 41,629 tests, 1,107 failures on this Windows box. Every failing file was re-run against a clean main worktree: 327 of the 334 fail identically there (environment/OS), and the run caught 3 real regressions that this PR fixes (2 lock-semantics cases — the _ic_* helpers are now module-level and stub-safe — and a stale _prepare_turn_input test double). After those fixes the candidate set is green except one known-flaky log-append race that also fails 5/5 on clean main.
  • I've added tests for my changes
  • I've tested on my platform: Windows 11 (git-bash), desktop app suites + gateway suites green

Documentation & Housekeeping

  • I've updated relevant documentation — apps/desktop/AGENTS.md + tui_gateway/AGENTS.md document the frame contract
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A (no config keys)
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — done (both area guides)
  • I've considered cross-platform impact (Windows, macOS) — pure Python/TS, no platform-specific code paths
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A

Screenshots / Logs

Full-suite A/B (local, Windows 11)

# branch (pre-fix run): 334 failing files, 1107 failing tests
# clean main worktree, same 334 files: 327 failing files, 1098 failing tests  -> environment
# 7 files failed only on the branch:
#   2  lock-semantics            -> fixed (35cd2649, helpers now module-level/stub-safe)
#   1  stale test double         -> fixed (a8806974)
#   4  load-flaky (compression x2, heartbeat, run_tests_parallel) -> green standalone
#   compute_host_phase1: log-append race fails 5/5 on clean main too (known flaky on Windows)
$ python -m pytest tests/tui_gateway/test_turn_note_frame.py -q
7 passed in 0.34s

$ cd apps/desktop && npx vitest run <the 6 touched suites>
 Test Files  6 passed (6)
      Tests  189 passed (189)

# red proofs (mutating back to the old logic):
#   steerQueuedNow `if (!accepted)`      -> 1 failed  (keeps the entry queued when the middleware cancels)
#   steerDraft without the 'canceled' branch -> 1 failed (restores the draft and queues nothing)

@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/tui Terminal UI (ui-tui/ + tui_gateway/) comp/desktop Electron desktop app (apps/desktop/*) comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state labels Sep 11, 2026
A single send can now carry a composer-mode frame: `note` is an instruction for
the model, `mode` an opaque display-only label. The note never mixes into the
user's own `content` — it rides the api_content sidecar on the delivered row.

- `AIAgent.steer`/`redirect` accept `note`/`mode` and queue them next to their
  pending slot (concatenating notes, last label wins); the delivery sites take
  them one-shot so a frame can never leak into a later correction.
- `steer_user_row` puts the note in `api_content` and the label in
  display_metadata; `_apply_active_turn_redirect` prepends the note to the
  correction's api_content and carries the label.
- The redundant-final-turn path consumes a leftover frame instead of letting it
  attach to the next turn; interrupt/clear paths drop it with their slot.
… steer

`prompt.submit`, `session.steer` and `session.redirect` accept an optional
`note`/`mode` pair (`parse_turn_note`: sanitized + capped; mode is display-only).
The note is prepended to the run message at turn assembly — like the other
per-turn notes — so it reaches the model without polluting the durable row.

- Busy arrivals keep the frame in their OWN queue envelope (never merged into a
  plain slot, never dropped by the self-duplicate sanitizer); the drain hands it
  back to the turn, and the compute-host turn frame carries the same keys.
- The server-side queued branch of a build-window redirect keeps the frame too.
The composer middleware is now the single producer of a per-turn frame: it runs
once per send and hands `note`/`mode` to the gateway through submit options /
redirect params, so a mode survives every entry point — typed, voice, queued
drains, and mid-turn steers (including the tile composer).

- `redirectPrompt` runs the chain before the RPC and before the session-not-found
  retry; `steerPrompt` runs it before the optimistic append so a cancel leaves no
  bubble behind.
- `'canceled'` is a cancel, not a rejection: `steerDraft` restores the draft and
  queues nothing, and the queue's steer-now consumes an entry only on a delivered
  redirect (`accepted === true`) — 'canceled' is truthy, so a truthiness test
  would silently eat the queued words.
- Python: parse/sanitize/cap, steer-row api_content vs content, one-shot frame
  consumption, queue envelope isolation, sanitizer exemption (proven red against
  the pre-fix tree).
- Desktop: cancel semantics for `steerDraft` (restore, no enqueue) and
  `steerQueuedNow` (no consumption), both failing under the old truthiness logic.
Area guides now state the contract: the frame is per-send, the middleware is its
only producer, the note rides api_content / the label display_metadata, and
'canceled' never consumes queued words.
The note/label queued beside a pending steer/redirect is now written and read by
module-level helpers (`_ic_queue_correction_note` / `_ic_take_correction_note`)
that mirror `_ic_lock`/`_ic_slot`: `getattr`-based, lock-optional, so the
`__init__`-less stubs the lock-semantics tests build keep working.

`AIAgent.steer()` calling the previous bound method broke
`tests/agent/test_lock_fallback_base_semantics.py` (2 failures vs the baseline:
the slot-reads-fail-loud case and the steer/drain roundtrip). Delivery sites
import the helpers instead of reaching through the agent.

Also covers the stub contract in the frame contract suite.
`_run_prompt_submit` now calls `_prepare_turn_input(..., turn_note=...)`, so the
rebind namespace in the bot-live-owner test must accept keyword arguments. With
the old `lambda *args: None` the refusal path raised inside the turn, the
recovery branch hit a stale name (the template namespace has no
`_recover_turn_exception`) and the failed-mailbox receipt was never committed —
the test asserted `'failed'` but saw `'claimed'`.
A queued send drained with whatever mode was live AT DRAIN TIME: queueCurrentDraft enqueued {text, attachments} and the drain re-ran the middleware chain, so changing modes with entries still queued stamped every drain with the new mode (and an Agent switch erased the queued Plan/Ask/Debug framing entirely).

Run the chain once at enqueue and seal {mode, note} onto the QueuedPromptEntry; every drain (foreground runDrain, background drain, steerQueuedNow) hands the sealed frame back as submit options with fromQueue, and the wrapper passes note/mode/fromQueue INTO the chain so middleware can hand the frame back untouched instead of re-deriving.
Store: the frame persists through enqueue and survives text edits. Hook: the chain runs once at enqueue (seal) and a drain forwards the sealed frame with fromQueue — proven against a registered middleware, never the live mode.
The queue freeze sealed frames only inside queueCurrentDraft; a steer rejected mid-turn (reconnect / settle race) fell into the raw requeue at use-composer-submit — a frameless entry, so its drain arrived note=0 (observed live: ask note lost during a backend recycle). Extract sealQueuedFrame (runs the chain once, cancel => empty frame) and seal on every enqueue path; queueCurrentDraft now reuses it.
…g one-shot copy)

Primacy+recency for instruction compliance: the ask note already led the run message via _prepare_turn_input; also park it in the one-shot slot (gated on the [mode:ask] head) so _merge_gateway_notes appends it — api_content = note + text + note. Ask-only: plan/debug keep their single leading copy (cost). Session titles read the pristine persist override first so the scaffold never leaks into a title.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/desktop Electron desktop app (apps/desktop/*) comp/tui Terminal UI (ui-tui/ + tui_gateway/) P3 Low — cosmetic, nice to have sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: per-turn model note + display-only mode label on prompt.submit / steer / redirect

2 participants