feat(serve): voice dictation over the daemon for the Web Shell - #7
Closed
qqqys wants to merge 16 commits into
Closed
feat(serve): voice dictation over the daemon for the Web Shell#7qqqys wants to merge 16 commits into
qqqys wants to merge 16 commits into
Conversation
…4721) (QwenLM#5740) pruneSnapshots derived runId from a snapshot filename and passed it straight to fs.rm(..., { recursive: true, force: true }). The listing is a plain *.json glob, so a file named ...json yields runId '..' and the rm deletes the runs dir's PARENT (the project root, .git and all); notarun.json deletes a sibling dir. A malicious repo could commit such a file under workflows/; once a victim runs more than the retention cap, the prune wipes their project. Gate the recursive delete on the generated wf_<hex> run-id shape (the same pattern workflow.ts already uses to validate resumeFromRunId). The .json unlink stays unconditional — it removes exactly that one file, never a directory. Carries a RED-before-fix test that plants ...json / notarun.json as the oldest snapshots and asserts the parent canary and a sibling dir survive. Follow-up to PR QwenLM#5600 (merged); addresses review thread r3451484367.
A fork (cross-repository) PR whose title is a `refactor` type could be auto-approved by the /triage skill and merged without a maintainer reviewing the structural changes (this happened with QwenLM#5089). Add a deterministic approval guardrail to Stage 3: before approving, check `isCrossRepository && title ~ /^refactor/i`; on a match, skip `gh pr review --approve` and escalate to the maintainer instead. Approval is now a positive condition (the guard must explicitly pass), so a blocked or empty check never approves. Document the rule in the skill's global Rules section as well.
…wenLM#5746) The TUI paints no global background — almost everything relies on the terminal's own background. The input box (since QwenLM#5568) and the user-message band flood themselves with theme.background.primary, which only looks right when the active theme's brightness matches the terminal. Forcing e.g. "Qwen Light" onto a dark terminal painted a bright input box and bright message bands fighting the dark surroundings. Add themeManager.getTerminalBackgroundType() (memoised; prefers the startup OSC 11 result, else a sync COLORFGBG/macOS heuristic) and gate those fills on it: use the theme background when it matches the terminal, otherwise stay transparent and blend in. The software cursor derives its contrast from the same effective background so it stays visible when no fill is painted.
Bring voice input to the `qwen serve` Web Shell. The browser captures the microphone, streams raw 16 kHz mono PCM to a new `/voice/stream` WebSocket, and the daemon transcribes server-side by reusing the existing CLI voice pipeline (realtime streaming + on-stop batch) — provider credentials never reach the browser, and it works whether the daemon is local or remote. Daemon: - `/voice/stream` WebSocket handler (serve/voice) reusing openVoiceStream / openQwenAsrRealtimeStream / transcribeVoiceAudio; resolves the workspace voiceModel from a ModelsConfig built off settings. - Routed through the existing ACP upgrade listener so it shares the loopback / host-allowlist / CSRF / bearer checks; concurrency-capped. - Advertises the `voice_transcribe` capability (unconditional, like auth_device_flow; the WS errors when no voice model is configured). - Relax `Permissions-Policy` to `microphone=(self)` so the same-origin shell can request the mic — an empty `microphone=()` allowlist blocked the prompt entirely. - Allowlist `voiceModel` on `/workspace/settings` so the picker can read it. Web Shell: - Mic button in the composer: click to record, click to stop -> the transcript is inserted into the input for review before sending. - `/model --voice` picker (sourced from `/workspace/providers`, since voice models are hidden from the session model list) and `/model --voice <id>`, persisted via the prompt channel like `/model --fast`. The voiceModel resolver now accepts a structural model lookup so the daemon can resolve it without constructing a full CLI Config. Co-authored-by: Qwen-Coder <noreply@qwen.ai>
qqqys
force-pushed
the
feat/serve-voice-dictation
branch
from
June 23, 2026 07:56
fd2c061 to
459bded
Compare
Owner
Author
|
Superseded by the upstream PR QwenLM#5755 (same branch, now targeting upstream). |
Code Coverage Summary
CLI Package - Full Text ReportCore Package - Full Text ReportFor detailed HTML reports, please see the 'coverage-reports-22.x-ubuntu-latest' artifact from the main CI run. |
qqqys
pushed a commit
that referenced
this pull request
Jul 7, 2026
… model persistence (QwenLM#6060) * feat(cli): add --project and --global flags to /model for per-project model persistence Add scope control to the /model command so users can persist model selections to either project-level or user-level settings independently. - /model --project: persist to workspace .qwen/settings.json - /model --global: persist to user ~/.qwen/settings.json - /model (no flag): unchanged behavior (backward compatible) - Model dialog title shows scope: 'Select Model (this project)' / 'Select Model (global)' - Completion and argumentHint updated with new flags - Full i18n support for zh/en Closes QwenLM#6052 Signed-off-by: Alex <alex.tech.lab@outlook.com> * fix(cli): add missing zh-TW translations for /model scope flags Signed-off-by: Alex <alex.tech.lab@outlook.com> * fix(cli): address PR review — scope flags, subcommand persistScope, titles, tests - parseScopeFlags: use (?:^|\s) instead of \b for --flag matching (\b fails because - is not a word character) - Completion: strip all flags to isolate model prefix, supports any order - Subcommand dialogs (fast/voice/vision) now propagate persistScope - slashCommandProcessor forwards persistScope for all subcommand cases - ModelDialog title combines subcommand mode + scope label e.g. 'Select Fast Model (this project)' - Subcommand confirmations show scope suffix (project/global) - Extract persistScopeSpread() helper to reduce duplication - Add 9 tests covering scope flags, dialog returns, confirmations - Add i18n keys for scope suffix labels in zh/en/zh-TW Signed-off-by: Alex <alex.tech.lab@outlook.com> * fix(cli): use Partial<Config> & {[key:string]:unknown} to fix index signature TS error Replace Record<string,unknown> with Partial<Config> & {[key:string]:unknown} to satisfy TS4111 index signature access rule in the CI build. Signed-off-by: Alex <alex.tech.lab@outlook.com> * fix(cli): add scope suffix to ModelDialog history items Address review comment: historyManager.addItem for voice/fast/vision/main model selections now shows scope indicator like ' (this project)' or ' (global)', consistent with CLI direct-set confirmations. Affected: handleModelSwitchSuccess (main), handleSelect (voice/fast/vision) Signed-off-by: Alex <alex.tech.lab@outlook.com> * fix(cli): wrap scopeSuffix in t() and unify wording with ModelDialog - scopeSuffix in modelCommand.ts now uses t(' (this project)') / t(' (global)') instead of hardcoded English strings, matching ModelDialog.tsx wording - Main model confirmation uses shared scopeSuffix instead of separate i18n keys, eliminating 'Model: {{model}} (project)' duplication - Remove unused i18n keys from en/zh/zh-TW locales - Update tests to expect '(this project)' wording Signed-off-by: Alex <alex.tech.lab@outlook.com> * fix(cli): address code review feedback — scope validation, i18n, tests - Reject inline prompt + scope flag combination with clear error (#1) - Add mutual exclusivity check for --project and --global (#5) - Verify setValue scope parameter in tests + add --global test (#2) - Extract scopeSuffix to shared variable, remove duplication (#3) - Remove dead i18n keys 'Select Model (this project)' / '(global)' (#4) - Fix scopeSuffix placement on model line not API key line (#8) - Add fr.js / ja.js translations for scope keys (#10) - Remove unused export ModelDialogPersistScope (#6) - Wrap non-interactive help text in t() with new flags (#7) - Fix argumentHint grouping to show mode vs scope flags (#11) Signed-off-by: Alex <alex.tech.lab@outlook.com> * fix(cli): reject --project when workspace is untrusted Reject --project scope flag before direct persistence or opening ModelDialog when settings.isTrusted is false. Workspace settings are ignored on merge in that state, so the save would silently not take effect. Also mirrors the guard in ModelDialog.tsx resolvePersistScope() to fall back to user scope when the dialog is opened with --project on an untrusted folder. Default mock settings now includes isTrusted: true. Signed-off-by: Alex <alex.tech.lab@outlook.com> --------- Signed-off-by: Alex <alex.tech.lab@outlook.com> Co-authored-by: qwen-code-dev-bot <qwen-code-dev-bot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
Adds voice dictation to the
qwen serveWeb Shell. The browser captures the microphone, streams raw 16 kHz mono PCM to a new/voice/streamWebSocket on the daemon, and the daemon transcribes server-side by reusing the existing CLI voice pipeline (realtime streaming and on-stop batch). The final transcript is inserted into the composer for the user to review before sending. Also adds a/model --voicepicker so the transcription model can be chosen from the Web Shell.Why it's needed
Voice dictation previously existed only in the terminal UI; the daemon-backed Web Shell had no voice at all. This brings it to the browser without sending provider credentials to the client — capture is client-side, transcription is server-side — and it works whether the daemon runs locally or remotely.
Reviewer Test Plan
How to verify
/model --voice, e.g.fun-asr-realtimefor streaming orqwen3-asr-flashfor batch).npm start -- serve --open(must be loopback with no token — see Risk)./model --voiceto open a picker listing voice-transcribable models; selecting one (or typing/model --voice <id>) switches the transcription model used by the next dictation.Evidence (Before & After)
Before: no microphone or voice affordance in the Web Shell. After: a mic button records and inserts a transcript, and
/model --voicelists/sets the voice model. Verified locally in Chrome against a loopback daemon; the daemon/voice/streamround-trip was confirmed via a fake-mic Playwright run plus curl checks on/capabilities,/voice/stream(real WS handshake), and/workspace/settings.Tested on
Environment (optional)
npm start -- serve(loopback), Chrome. Daemon unit tests via vitest.Risk & Scope
Authorizationheader, so voice works only against a loopback daemon with no token — the same limitation as the existing ACP WebSocket browser transport.Permissions-Policytomicrophone=(self)(wasmicrophone=(), which blocked the permission prompt outright). Same-origin only; framing stays blocked byX-Frame-Options: DENYand CSPframe-ancestors 'none'.ScriptProcessorNodefor capture to avoid a CSPscript-srcblock on blob-URL AudioWorklet modules.Linked Issues
N/A
中文说明
这个 PR 做了什么
为
qwen serve的 Web Shell 增加语音听写。浏览器采集麦克风，将原始 16 kHz 单声道 PCM 流式推送到 daemon 新增的/voice/streamWebSocket，daemon 复用现有的 CLI 语音流水线在服务端转写(实时流式 + 停止后批量)。最终转写结果会填入输入框,供用户确认后再发送。同时新增/model --voice选择器,可在 Web Shell 里选择转写模型。为什么需要
语音听写此前只存在于终端 UI;daemon 背后的 Web Shell 完全没有语音。本 PR 把它带到浏览器,且不把模型凭证下发到客户端——采集在客户端、转写在服务端——无论 daemon 本地还是远程都可用。
评审验证步骤
如何验证
/model --voice,例如流式的fun-asr-realtime或批量的qwen3-asr-flash)。npm start -- serve --open(必须是 loopback 且无 token,见风险说明)。/model --voice打开选择器,列出可转写的语音模型;选择一个(或输入/model --voice <id>)即可切换下次听写使用的转写模型。证据(前后对比)
之前:Web Shell 没有麦克风/语音入口。之后:麦克风按钮可录音并插入转写,
/model --voice可列出/设置语音模型。已在本地 Chrome 对 loopback daemon 验证;daemon/voice/stream往返通过假麦克风的 Playwright 运行 + 对/capabilities、/voice/stream(真实 WS 握手)、/workspace/settings的 curl 验证确认。风险与范围
Authorization头,因此语音仅在 loopback 且无 token 的 daemon 下可用——与现有 ACP WebSocket 浏览器传输的限制一致。Permissions-Policy放宽为microphone=(self)(原为microphone=(),会直接屏蔽权限弹窗)。仅同源;X-Frame-Options: DENY与 CSPframe-ancestors 'none'仍禁止被嵌入。ScriptProcessorNode,以绕开 CSPscript-src对 blob-URL AudioWorklet 模块的拦截。