Skip to content

cmux NEXT composer: dictation looks like Claude's: Listening…, a live waveform, a finish key and a lit mic - #22713

Merged
teamleaderleo merged 3 commits into
feat-cmux-nextfrom
feat-cmux-next-composer-dictation-look
Oct 21, 2026
Merged

teamleaderleo merged 3 commits into
feat-cmux-nextfrom
feat-cmux-next-composer-dictation-look

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

While you dictate into the agent chat composer (or the New Tab field), it now looks like Claude's (Leo's Claude Code reference, 2026-10-20). The empty prompt says Listening… in italic, muted. Words stream into the field as they're heard. Before the mic, in the same row, you get a waveform of thin bars that follows the mic level and fades to dots at both ends, then a finish key (the Return glyph) that ends dictation and keeps the text. The mic itself stays a mic, lit as an accent-tinted rounded square. Clicking it again still stops. Before this change, the mic turned into a small red level meter and there was no placeholder or finish control.

Mechanism (webview only; Chief's native composer is another PR):

  • dictation.ts keeps the level history at 11 samples for the wave. It also keeps a page-wide "starting or listening" flag (useDictationListening), fed by the same deliverDictation bridge, and adds Dictation.finish() → dictation.stop (the host's stop keeps the words; Esc's dictation.cancel still drops them).
  • DictationListening.tsx is new: DictationWave (bars whose --level sets their height, a sine taper so the middle rises most, three resting dots each end) and DictationFinishButton (disabled while finalizing). DictationMic renders wave, finish key and mic inside the same HideableChrome / fit slot. So the hidden-mic setting hides all three, and the narrow-row rule that keeps a pressed mic in the row keeps them too. DictationLevelMeter.tsx is gone.
  • Composer.tsx (+5 lines) and NewTabScreen.tsx swap the placeholder for dictation.listening while the flag is on and mark the field (is-dictating / data-dictation-listening) for the italic.
  • dictation.css (new): the lit mic uses --cmux-accent, which falls back to --agent-accent and then the text color. No hard-coded blue; it's neutral in a theme without an accent. The wave's height transition is inside prefers-reduced-motion: no-preference, and the pane's data-agent-motion="off" stops it, so with motion off each level applies at once. The red listening rules in styles.css / composerStates.css are removed, and the two hover/focus pill exclusions in composerLocation.css now also cover starting.
  • Live text: the native side already sends the whole text so far on every update (AgentPaneDictation.payload), and useDictation splices each one. A test now checks that partials land before the session ends; no fix was needed.
  • Strings dictation.listening and dictation.finish are in all 21 locales of the agent pane catalog: English translated, the rest machine translations marked needs_review.

Kept as before: agentPane.showDictation hide/show, Esc cancel, the Toggle Dictation shortcut and hold-to-talk (native), and aria-label / aria-pressed / aria-busy on the mic.

Gallery: agent-pane.dictation (component) gains starting, finish, and a realistic listening level history. The new agent-pane.composer-dictation drives the chat's real composer through cmuxAcpmuxBridge.dictation with these variants: idle, listening-empty, listening-words and finalizing. That entry is in its own file, away from composer.gallery.ts, which #22667 is editing.

Risk: normal (webview UI, no protocol or native change).

Changelog

Changed: Dictation in the agent composer looks like Claude's: "Listening…" in the empty prompt, a live waveform, a finish key that keeps your words, and a lit mic.

Test

  • webviews/test/new-tab-dictation.test.tsx, 3 new tests: the "Listening…" placeholder shows only while waiting with no words, and partial words stream in live; the mic is pressed and lit beside the wave and finish key, and idle shows the mic alone; the finish key calls dictation.stop (never cancel) and the words stay.
  • DictationButton.test.tsx: the mic keeps its glyph while listening; the finish key calls finish() and is disabled while finalizing.
  • Red: commit 1f55ac4 (tests only). Green: 67f86ea. Fleet: cmux-ci run --class isolated --script scripts/ci/run-webviews-tests.sh --ref <sha> (results below).

Observed / Not run

  • Fleet, scripts/ci/run-webviews-tests.sh (whole webviews suite, light class): red 1f55ac4 (step 95f12af47f30af7bd9e9e850) fails the 3 new tests in new-tab-dictation.test.tsx. Green 67f86ea (step ed1107d150f2f73e803d8f41) passes new-tab-dictation.test.tsx and DictationButton.test.tsx. The same 9 other files fail in both runs (gallery-coverage on ChatHeaderInlineStatus, hover-row-sweep, stats-page, tooltip-surface, whats-new-text-selection, agent-pane-column-edges, apps-flow, composer-html-paste-edit, thread-minimap-feel), so those are base failures.

  • The first green run (c145e8b) failed while building the bundles: build-agent-pane-web.sh didn't ship the new dictation.css. Fixed in 67f86ea, which adds it after the composer sheets.

  • Locally: bunx vp check --no-fmt (lint + types: 0 errors), vp fmt on the changed files, prepush-gate.py (no new violations). scripts/web-lint.sh fails only on a base react-hooks/exhaustive-deps warning at Composer.tsx:461, outside this diff, so I pushed with CMUX_WEB_LINT_SKIP_REASON.

  • Not run: no local build or browser; screenshots come from the gallery advisory run. Not checked in the real app: the italic placeholder, the accent tint per theme, the wave's motion, and the narrow-row fit with the wave present.

Agent session: cosmic-gray
🤖 Generated with Claude Code

The field says Listening… until words come and streams partials live; a
waveform, a finish key and the lit mic sit beside Send while listening; the
finish key stops (keeps the words), never cancels.

Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=e8f63fd9-94e8-4878-bcd4-704659aba1f2&host=cosmic-gray
… waveform, a finish key and a lit mic

While a session runs, the empty prompt reads "Listening…" in italic (chat
composer and New Tab field), the words stream in update by update as before,
and before the mic come a waveform of thin bars that follows the mic level and
tapers into dots at both ends, and a finish key (Return glyph) that calls
dictation.stop, keeping the words. The mic keeps its glyph and lights in the
theme accent (an accent-tinted rounded square) instead of turning into a red
level meter. Bars move only under the pane's motion gate.

Strings dictation.listening and dictation.finish in all 21 locales
(machine translations marked needs_review). Gallery: component states
(idle, starting, listening, finalizing, finish) and the chat's real composer
driven through the bridge (idle, listening empty, listening with words,
finalizing).

Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=e8f63fd9-94e8-4878-bcd4-704659aba1f2&host=cosmic-gray
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review 67f86ea: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr22713 -f ref=feat-cmux-next-composer-dictation-look -f platform=macos -f backend_mode=local

Unclear: web / react-apps-check failed: no known signature; failed step: Check webview test shards. This PR merged before its CI finished: fix forward on main.

CI failed on 67f86eab2b (run 39524247268 attempt 1): 1 code, 1 unknown.

Job Verdict Why
web / react-apps-check (1) code a web test failed
web / react-apps-check unknown no known signature; failed step: Check webview test shards

Failures:

  • seen on other PRs agent-pane-background-tasks.test.ts clicking a background task's end opens its terminal beside the chat: failed (job)
Matched log lines
web / react-apps-check (1): (fail) clicking a background task's end opens its terminal beside the chat [926.65ms]

Not re-run automatically: web / react-apps-check (1), web / react-apps-check are not machine failures.

Written by scripts/ci/classify_failures.py (ci-failure-attribution.yml); signatures are its SIGNATURES table. A machine verdict is the runner's fault, not this PR's; yours means the failing file is one this PR changes, also red on main that main's latest full suite fails the same way, seen on other PRs that it failed on another pull request's run lately.

red: web / Web status; NEW: green on the base b8314d7; do not merge until green; see lane-rules
red: web / react-apps-check; no finished base run on b8314d7 to compare; do not merge until green; see lane-rules
red: web / react-apps-check (1); no finished base run on b8314d7 to compare; do not merge until green; see lane-rules

build-agent-pane-web.sh inlines every sheet the pane imports; the new
dictation sheet goes after the composer sheets so its lit-mic rules win.

Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=e8f63fd9-94e8-4878-bcd4-704659aba1f2&host=cosmic-gray
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Bead cx-x3v88 created for this PR by the cmux coordinator.

@blacksmith-sh

blacksmith-sh Bot commented Oct 21, 2026

Copy link
Copy Markdown

Found 2 test failures on Blacksmith runners:

Failures

Test View Logs
test/agent-pane-background-tasks.test.ts/agent-pane-background-tasks.test View Logs
test/agent-pane-background-tasks.test.ts/agent-pane-background-tasks.test View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

teamleaderleo added a commit that referenced this pull request Oct 21, 2026
…ish key returns focus to the prompt (#22719)

* test(next): the dictation finish key hands the keyboard back to the field (red)

The New Tab helper passes `dictating` from the pane's own dictation, as
App.tsx will, instead of reading a page-wide flag.

Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=e8f63fd9-94e8-4878-bcd4-704659aba1f2&host=cosmic-gray

* cmux NEXT composer: dictation's Listening… comes from the pane's own session; the finish key keeps focus

Follow-up to #22713's review: the page-wide listening flag in dictation.ts is
gone; App.tsx passes `dictating` (isWaitingForWords of its dictation state)
to the Composer and the New Tab screen. The finish key puts the keyboard
back in the prompt as it goes away, keeps a 4px gap from the mic in the chat
bar, WAVE_BARS is private, the stale red-mic comment in styles.css is
updated, and the gallery covers 297px and 260px rows.

Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=e8f63fd9-94e8-4878-bcd4-704659aba1f2&host=cosmic-gray

* test(agent pane): the Dictation key starts the pane's dictation, not macOS's overlay (red)

Start Dictation sent up the responder chain from an agent page's web view
goes past the pane to NSApplication, which shows macOS's black dictation
HUD at WebKit's caret over the composer's corner. Adds the
agent.systemDictation Debug toggle (off) and the AgentPaneDictationKey
routes the fix uses; the pane does not take the action yet, so the tests
fail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15ba2-63d6-7533-b232-21f35d99d2b2&host=cosmic-gray

* cmux NEXT agent pane: the Dictation key starts cmux's dictation in the composer

The Dictation key (fn, the 🎤 key) and Edit ▸ Start Dictation send
startDictation: up the responder chain; NSApplication answered it with
macOS's black dictation HUD at WebKit's caret, over the composer's corner.
AgentPaneView now takes the action when the first responder is inside it
and toggles the pane's own dictation, as the composer's mic does, so a
second press finishes it. Terminals, browsers and other panes never reach
an agent pane and keep macOS's dictation. The agent.systemDictation Debug
toggle (off by default) hands the action back to NSApplication.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15ba2-63d6-7533-b232-21f35d99d2b2&host=cosmic-gray

* test: the dictation finish key's focus check starts from the key itself

Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15ba2-63d6-7533-b232-21f35d99d2b2&host=cosmic-gray

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant