Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/remote-speech-to-text.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"kilo-code": patch
---

Hide voice input in remote VS Code windows, where microphone capture runs on the remote machine and fails, and explain the limit in the Speech to Text settings
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@ The feature includes real-time audio level visualization and voice activity dete
**Microphone button not appearing:**

- Enable and sign in to the Kilo provider, or set a custom transcription base URL
- Confirm you are in a local VS Code window. Voice input is unavailable in remote windows such as Remote-SSH, WSL, and dev containers, because audio capture runs on the machine that hosts the extension

**Transcription errors:**

Expand All @@ -118,5 +119,6 @@ The feature includes real-time audio level visualization and voice activity dete
Voice transcription has these requirements:

- Requires an active internet connection, unless your custom base URL points at a local service
- Requires a local VS Code window. In remote windows audio capture runs on the remote machine, which usually has no microphone
- Requires Kilo Gateway access through your Kilo account, or a custom OpenAI-compatible base URL
- Transcription accuracy depends on audio quality and speech clarity
9 changes: 8 additions & 1 deletion packages/kilo-vscode/src/features.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { hasIndexingPlugin } from "@kilocode/kilo-indexing/detect"
import type { KiloClient } from "@kilocode/sdk/v2"
import * as vscode from "vscode"

type PluginSpec = string | [string, Record<string, unknown>]

Expand All @@ -11,13 +12,19 @@ export type Features = {
indexing: boolean
sandboxControls: boolean
backgroundSubagents: boolean
speechToText: boolean
}

export function configFeatures(config?: ConfigLike | null, backgroundSubagents = false): Features {
export function configFeatures(
config?: ConfigLike | null,
backgroundSubagents = false,
remote = !!vscode.env.remoteName,
): Features {
return {
indexing: hasIndexingPlugin(config?.plugin ?? []),
sandboxControls: process.platform !== "win32",
backgroundSubagents,
speechToText: !remote,
}
}

Expand Down
6 changes: 6 additions & 0 deletions packages/kilo-vscode/tests/unit/indexing-utils.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,12 @@ describe("indexing feature detection", () => {
expect(configFeatures({}, true).backgroundSubagents).toBe(true)
})

it("disables speech input capture in remote windows", () => {
expect(configFeatures().speechToText).toBe(true)
expect(configFeatures({}, false, true).speechToText).toBe(false)
expect(configFeatures({}, false, false).speechToText).toBe(true)
})

it("enables indexing settings when the indexing plugin is present", () => {
expect(configFeatures({ plugin: ["kilo-indexing"] }).indexing).toBe(true)
})
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,11 @@ describe("speech-to-text availability", () => {
expect(canUseSpeechToText({ enabled_providers: ["kilo"] }, { kilo: "oauth" })).toBe(true)
})

it("hides speech input when the window cannot capture audio", () => {
expect(canUseSpeechToText({}, { kilo: "oauth" }, false)).toBe(false)
expect(canUseSpeechToText({}, { kilo: "oauth" }, true)).toBe(true)
})

it("normalizes configured and unknown transcription models", () => {
expect(
selectedSpeechToTextModel({ experimental: { speech_to_text_model: "google/chirp-3" } }, [
Expand Down
13 changes: 11 additions & 2 deletions packages/kilo-vscode/tests/unit/speech-to-text-prewarm.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,14 +44,21 @@ const SCRIPT = `

const [config, setConfig] = createSignal({ disabled_providers: ["kilo"] })
const [auth, setAuth] = createSignal({})
const [capture, setCapture] = createSignal(false)
const features = () => ({
indexing: false,
sandboxControls: false,
backgroundSubagents: false,
speechToText: capture(),
})
const root = document.createElement("div")
const dispose = render(
() =>
createComponent(ProviderContext.Provider, {
value: { authStates: auth },
get children() {
return createComponent(ConfigContext.Provider, {
value: { config },
value: { config, features },
get children() {
return createComponent(SpeechToTextPrewarm, {})
},
Expand All @@ -65,8 +72,10 @@ const SCRIPT = `
setAuth({ kilo: "api" })
if (sent.length !== 0) fail("prewarmed while Kilo was disabled")
setConfig({})
if (sent.length !== 0) fail("prewarmed without capture support")
setCapture(true)
if (sent.length !== 1 || sent[0]?.type !== "speechToTextPrewarm") {
fail("did not prewarm after Kilo access became available")
fail("did not prewarm after capture became available")
}
setAuth({ kilo: "oauth" })
if (sent.length !== 1) fail("prewarmed more than once")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ export const NewWorktreeDialog: Component<{
const sandboxVisible = () => features().sandboxControls && globalConfig().sandbox?.enabled === true
const speech = useSpeechToText(vscode, server, { t })
const speechModels = useSpeechToTextModels()
const canUseSpeech = () => canUseSpeechToText(config(), provider.authStates())
const canUseSpeech = () => canUseSpeechToText(config(), provider.authStates(), features().speechToText)
const speechModel = () => selectedSpeechToTextModel(config(), speechModels.models())
let prior: string | null = null
let request: string | undefined
Expand Down
4 changes: 2 additions & 2 deletions packages/kilo-vscode/webview-ui/diff-viewer/review-setup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,12 @@ export function createReviewSpeech(t: T): {
const vscode = useVSCode()
const server = useServer()
const provider = useProvider()
const { config } = useConfig()
const { config, features } = useConfig()
const speech = useSpeechToText(vscode, server, { t })
const models = useSpeechToTextModels()
return {
speech,
enabled: () => canUseSpeechToText(config(), provider.authStates()),
enabled: () => canUseSpeechToText(config(), provider.authStates(), features().speechToText),
model: () => selectedSpeechToTextModel(config(), models.models()),
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -764,7 +764,7 @@ export const PromptInput: Component<PromptInputProps> = (props) => {
globalConfig(),
)
const isDisabled = () => !server.isConnected() || locked() || goal.pending()
const canUseSpeech = () => canUseSpeechToText(config(), provider.authStates())
const canUseSpeech = () => canUseSpeechToText(config(), provider.authStates(), features().speechToText)
const speechModel = () => selectedSpeechToTextModel(config(), speechModels.models())
const hasInput = () =>
text().trim().length > 0 ||
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ import { AUTOCOMPLETE_SELECTOR_MODELS, getAutocompleteSelection } from "./autoco
import { preserveVariant } from "../../context/session-variant-store"

const ModelsTab: Component = () => {
const { config, settings, updateConfig, updateSetting } = useConfig()
const { config, settings, updateConfig, updateSetting, features } = useConfig()
const language = useLanguage()
const provider = useProvider()
const session = useSession()
Expand Down Expand Up @@ -246,11 +246,13 @@ const ModelsTab: Component = () => {
<SettingsRow
title={language.t("settings.models.speechToTextModel.title")}
description={
customSpeech()
? language.t("settings.models.speechToTextModel.customDescription")
: kiloReady()
? language.t("settings.models.speechToTextModel.description")
: language.t("settings.models.speechToText.disabledDescription")
!features().speechToText
? language.t("settings.models.speechToText.remoteDescription")
: customSpeech()
? language.t("settings.models.speechToTextModel.customDescription")
: kiloReady()
? language.t("settings.models.speechToTextModel.description")
: language.t("settings.models.speechToText.disabledDescription")
}
>
<Show
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ import { canUseSpeechToText } from "./availability"
export const SpeechToTextPrewarm: Component = () => {
const vscode = getVSCodeAPI()
const provider = useProvider()
const { config } = useConfig()
const { config, features } = useConfig()
let prepared = false

createEffect(() => {
if (prepared || !canUseSpeechToText(config(), provider.authStates())) return
if (prepared || !canUseSpeechToText(config(), provider.authStates(), features().speechToText)) return
prepared = true
vscode.postMessage({ type: "speechToTextPrewarm" })
})
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ export function hasSpeechToTextAccess(cfg: Cfg, auth: Readonly<Record<string, Au
return enabled && !cfg.disabled_providers?.includes(KILO_PROVIDER_ID) && (type === "api" || type === "oauth")
}

export function canUseSpeechToText(cfg: Cfg, auth: Readonly<Record<string, AuthState>>): boolean {
export function canUseSpeechToText(cfg: Cfg, auth: Readonly<Record<string, AuthState>>, capture = true): boolean {
if (!capture) return false
if (!hasSpeechToTextAccess(cfg, auth)) return false
// A custom endpoint needs an explicit model. Never fall back to a Gateway model ID,
// which the endpoint would reject or misinterpret.
Expand Down
1 change: 1 addition & 0 deletions packages/kilo-vscode/webview-ui/src/context/config.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,7 @@ export const ConfigProvider: ParentComponent = (props) => {
indexing: false,
sandboxControls: false,
backgroundSubagents: false,
speechToText: false,
})
const [loading, setLoading] = createSignal(true)
const [draft, setDraft] = createSignal<Partial<Config>>({})
Expand Down
2 changes: 2 additions & 0 deletions packages/kilo-vscode/webview-ui/src/i18n/ar.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions packages/kilo-vscode/webview-ui/src/i18n/br.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions packages/kilo-vscode/webview-ui/src/i18n/bs.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions packages/kilo-vscode/webview-ui/src/i18n/da.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions packages/kilo-vscode/webview-ui/src/i18n/de.ts
Original file line number Diff line number Diff line change
Expand Up @@ -938,6 +938,8 @@ export const dict = {
"settings.models.speechToTextApiKey.placeholder": "sk-...",
"settings.models.speechToText.disabledDescription":
"Aktivieren Sie den Kilo-Anbieter und melden Sie sich an, um Speech to Text zu verwenden, oder legen Sie unten eine eigene Transkriptions-Basis-URL fest.",
"settings.models.speechToText.remoteDescription":
"Spracheingabe ist in Remote-Fenstern nicht verfügbar. Öffnen Sie Kilo in einem lokalen Fenster, um das Mikrofon zu verwenden.",
"settings.models.speechToTextModel.title": "Sprache-zu-Text-Modell",
"settings.models.speechToTextModel.description":
"Wählen Sie das Kilo Gateway-Transkriptionsmodell für die Spracheingabe.",
Expand Down
2 changes: 2 additions & 0 deletions packages/kilo-vscode/webview-ui/src/i18n/en.ts
Original file line number Diff line number Diff line change
Expand Up @@ -887,6 +887,8 @@ export const dict = {

"settings.models.speechToText.disabledDescription":
"Kilo Gateway is selected. Enable and sign in to the Kilo provider to choose a supported model, or enter a custom transcription base URL above.",
"settings.models.speechToText.remoteDescription":
"Voice input is unavailable in remote windows. Open Kilo in a local window to use the microphone.",
"settings.models.speechToTextModel.title": "Speech to Text Model",
"settings.models.speechToTextModel.description":
"Kilo Gateway is the active speech-to-text source. Choose its transcription model for voice input.",
Expand Down
2 changes: 2 additions & 0 deletions packages/kilo-vscode/webview-ui/src/i18n/es.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions packages/kilo-vscode/webview-ui/src/i18n/fa.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions packages/kilo-vscode/webview-ui/src/i18n/fr.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions packages/kilo-vscode/webview-ui/src/i18n/it.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions packages/kilo-vscode/webview-ui/src/i18n/ja.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions packages/kilo-vscode/webview-ui/src/i18n/ko.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions packages/kilo-vscode/webview-ui/src/i18n/nl.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions packages/kilo-vscode/webview-ui/src/i18n/no.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading