Skip to content

feat(vscode): prewarm microphone capture before showing voice input as recording - #12001

Merged
marius-kilocode merged 4 commits into
mainfrom
obtainable-olive
Jul 7, 2026
Merged

feat(vscode): prewarm microphone capture before showing voice input as recording#12001
marius-kilocode merged 4 commits into
mainfrom
obtainable-olive

Conversation

@marius-kilocode

Copy link
Copy Markdown
Collaborator

Problem

When the microphone button is clicked, voice input immediately appears as "recording" even though FFmpeg and the OS may still be opening the capture device. Speech during that opening gap is lost, which is most noticeable on the first recording after startup. The loading indicator also flips to the recording state before capture is actually ready, so users begin speaking too early.

What changed

Introduces an explicit starting state between click and confirmed capture readiness, and prewarms FFmpeg binary discovery so the first recording does not pay the discovery cost at click time.

Capture-readiness state machine (useSpeechToText.ts, SpeechToTextButton.tsx):

  • The webview now transitions idle -> starting -> recording instead of jumping straight to recording. It only enters recording when the extension confirms capture is ready via the existing speechToTextStarted message.
  • While starting, the button shows the spinner, is non-interactive (clicks are ignored), and exposes aria-busy / aria-disabled so assistive tech announces it as in progress. Tooltips distinguish "Starting microphone..." from "Recording. Click to stop."
  • Stale speechToTextStarted messages for a request that is no longer in starting are dropped, preventing an orphaned ready event from falsely marking the button as recording.
  • Active recording is cancelled on component teardown (onCleanup) so a webview re-mount during a recording does not leave a dangling capture.

FFmpeg prewarm (capture.ts, input-tools.ts, SpeechToTextPrewarm.tsx):

  • A new speechToTextPrewarm webview message triggers resolveFFmpeg(), which discovers the FFmpeg binary and caches the resolved path. This resolves the binary only, without opening the microphone or prompting for permission, so signed-out users and users with Kilo speech disabled never run it.
  • startSpeechCapture reuses the cached binary via resolveFFmpeg(), falling back to a fresh discovery if the cached path no longer exists. The cached promise is cleared on failure so a failed prewarm remains retryable when recording actually starts.
  • A SpeechToTextPrewarm component is mounted in the sidebar, Agent Manager, and diff viewer webview roots; it fires the prewarm once speech-to-text is available and the user is authenticated.

Related

Addresses the same capture-readiness gap as #11738 (fix/speech-capture-readiness), which introduces a starting state and prewarms FFmpeg binary discovery. This is an alternate implementation of the same behavior for comparison.

Comment thread packages/kilo-vscode/src/speech-to-text/capture.ts Outdated
Comment thread packages/kilo-vscode/src/speech-to-text/capture.ts
Comment thread packages/kilo-vscode/src/services/input-tools.ts Outdated
Comment thread packages/kilo-vscode/tests/unit/speech-to-text-prewarm.test.ts
@kilo-code-bot

kilo-code-bot Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: 3 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 2
Issue Details (click to expand)

WARNING

File Line Issue
packages/kilo-vscode/webview-ui/src/components/speech-to-text/SpeechToTextButton.tsx 45 onCleanup silently cancels an active recording if the wrapping <Show> condition transiently flips false, with no user feedback

SUGGESTION

File Line Issue
packages/kilo-vscode/src/speech-to-text/capture.ts 241 PATH-resolved FFmpeg binary is cached indefinitely with no revalidation window
packages/kilo-vscode/tests/unit/speech-to-text-prewarm.test.ts 82 Retry-up-to-3-times loop treats any non-PASS/non-FAIL outcome as transient, which could mask a rare intermittent bug

The latest commit (fix(vscode): address speech-to-text review findings) resolved 4 previously flagged items: the unguarded FFmpeg cache reset race, the implicit any on the prewarm .catch parameter, the aria-disabled/disabled mismatch during starting, and the missing --starting CSS rule. Those are no longer listed above.

Files Reviewed (32 files)
  • .changeset/warm-speech-capture.md
  • packages/kilo-vscode/src/services/input-tools.ts
  • packages/kilo-vscode/src/speech-to-text/capture.ts - 1 issue
  • packages/kilo-vscode/tests/unit/speech-to-text-prewarm.test.ts - 1 issue
  • packages/kilo-vscode/tests/unit/use-speech-to-text.test.ts
  • packages/kilo-vscode/webview-ui/agent-manager/AgentManagerApp.tsx
  • packages/kilo-vscode/webview-ui/diff-viewer/DiffViewerApp.tsx
  • packages/kilo-vscode/webview-ui/src/App.tsx
  • packages/kilo-vscode/webview-ui/src/components/speech-to-text/SpeechToTextButton.tsx - 1 issue
  • packages/kilo-vscode/webview-ui/src/components/speech-to-text/SpeechToTextPrewarm.tsx
  • packages/kilo-vscode/webview-ui/src/components/speech-to-text/useSpeechToText.ts
  • packages/kilo-vscode/webview-ui/src/styles/prompt-input.css
  • packages/kilo-vscode/webview-ui/src/i18n/*.ts (26 locale files, one-line additions)
  • packages/kilo-vscode/webview-ui/src/types/messages/webview-messages.ts

Fix these issues in Kilo Cloud


Reviewed by claude-sonnet-5 · Input: 28 · Output: 13.9K · Cached: 817.6K

Review guidance: REVIEW.md from base branch main

@marius-kilocode
marius-kilocode merged commit b6247e8 into main Jul 7, 2026
21 checks passed
@marius-kilocode
marius-kilocode deleted the obtainable-olive branch July 7, 2026 12:30
t7tran pushed a commit to t7tran/kilocode that referenced this pull request Aug 14, 2026
feat(vscode): prewarm microphone capture before showing voice input as recording
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants