Skip to content

feat(desktop): add microphone device selector for voice mode - #74372

Open
EdderTalmor wants to merge 2 commits into
NousResearch:mainfrom
EdderTalmor:fix/issue-55697-microphone-selector
Open

feat(desktop): add microphone device selector for voice mode#74372
EdderTalmor wants to merge 2 commits into
NousResearch:mainfrom
EdderTalmor:fix/issue-55697-microphone-selector

Conversation

@EdderTalmor

Copy link
Copy Markdown

Closes #55697

What changed:

  • Desktop composer can now select an audio input device before starting voice mode.
  • Selected device is persisted across composer/session restarts.

Backend:

  • Thread deviceId through the desktop voice recorder hooks and apply it as an exact getUserMedia audio constraint when present.

Frontend:

  • Add useMicDevice for device enumeration, selection, and persistence.
  • Add a mic device dropdown to the composer controls when multiple inputs are available.

Why:
Users on laptops/docking setups frequently have multiple microphones; without a selector they are forced to use the OS default, which may not be the intended input.

Edder Talmor added 2 commits July 1, 2026 10:38
Backend:
- thread selected deviceId through MicRecorderOptions, VoiceRecorderOptions, and VoiceConversation
- honor exact deviceId hints in recorder constraints

Frontend:
- add useMicDevice hook with device enumeration and persistence
- add MicDeviceMenu to composer controls for input-device selection

Idea rationale: users on laptops with multiple inputs can choose the correct mic instead of always defaulting to system default
@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/desktop Electron desktop app (apps/desktop/*) comp/gateway Gateway runner, session dispatch, delivery platform/wecom WeCom / WeChat Work adapter sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages labels Jul 29, 2026
@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 30, 2026
@0xLeathery

Copy link
Copy Markdown
Contributor

Heads-up: deviceId helps selection; continuous voice still has a VAD SPOF

This composer-level device selector is a clean approach for #76866.

One gap we hit in production use (detailed on #75329): after getUserMedia with the right deviceId, continuous conversation can still drop utterances if the WebAudio level meter dies. Submission is gated on heardSpeech from RMS in use-mic-recorder; a dead AudioContext leaves heardSpeech === false and use-voice-conversation never calls STT — even when the WebM blob is fine.

Since this PR already threads deviceId through use-mic-recorder / use-voice-conversation / use-voice-recorder, it may be the cheapest place to also:

  • resume AudioContext / detect unhealthy meter
  • fall back VAD from MediaRecorder timeslice chunks
  • avoid silently discarding non-trivial recordings

I can provide a small patch atop this branch if useful. Otherwise I’ll leave device selection to this PR / #46916 and keep VAD hardening as a separate follow-up after maintainers pick a primary mic-recorder diff — to avoid a merge pile-up on the same files.

RCA comment: #75329 (comment)

@andrexibiza

Copy link
Copy Markdown
Contributor

Vox Lockin lane 09 verification receipt (feature PR for #46337):

  • Not mergeable as-is: git merge-tree against current main (70db671) conflicts in controls.tsx, composer/index.tsx, and tests/gateway/test_wecom.py (the head also carries an unrelated WeCom commit, 2f049a174f, which should be dropped before merge).
  • Desktop-class review: the deviceId plumbing (use-mic-recorderhandle.start({ deviceId })getUserMedia exact constraint) is sound and conflicts only because the hook moved on. It overlaps with Add Desktop microphone picker and STT diagnostics #46916 (also a mic-picker feature, also conflicting, currently marked CONFLICTING by GitHub). Both address feature request [Feature]: Add UI for Custom Local STT/TTS and Local Media Generation Providers in Hermes Desktop #46337.
  • No bug-class gap: this is an additive feature, not a reliability fix. Recommend the two authors coordinate on one rebased implementation; lane 09 will not duplicate.

Note for the rebase: on current main useMicRecorder.start already guards recorderRef.current and handle.start re-arms with the exact same options object each cycle, so a deviceId captured at conversation start stays stable across re-arms — the feature composes cleanly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/desktop Electron desktop app (apps/desktop/*) comp/gateway Gateway runner, session dispatch, delivery P3 Low — cosmetic, nice to have platform/wecom WeCom / WeChat Work adapter sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

read_file undercounts total_lines and can silently drop the last line of a file without a trailing newline

5 participants