Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/restore-authenticated-speech-input.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
"@kilocode/cli": patch
"@kilocode/sdk": patch
"kilo-code": patch
---

Restore speech input when profile details are unavailable, move transcription model selection to the Models tab, and default transcription to Whisper Large V3 Turbo.
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ Enable and sign in to the Kilo provider to use voice input in prompt fields. Req

## Choose a model

You can optionally choose a transcription model in **Settings** > **Experimental** > **Speech to Text Model**. Kilo stores this choice as `experimental.speech_to_text_model` in your global Kilo CLI config (`~/.config/kilo/kilo.jsonc`).
You can optionally choose a transcription model in **Settings** > **Models** > **Speech to Text Model**. Kilo stores this choice as `experimental.speech_to_text_model` in your global Kilo CLI config (`~/.config/kilo/kilo.jsonc`).

---

Expand Down
23 changes: 12 additions & 11 deletions packages/kilo-docs/pages/getting-started/settings/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,18 @@ Use **Local Config** or **Global Config** in the Settings header to open the mat
If you check config files into version control, make sure they do not contain API keys or other secrets (e.g., `provider.*.options.apiKey`). Use environment variables for credentials instead.
{% /callout %}

### Voice Transcription Model

When the Kilo provider is enabled and you are signed in, choose the transcription model under **Models** > **Speech to Text Model**. This stores `experimental.speech_to_text_model` in your global Kilo CLI config:

```json
{
"experimental": {
"speech_to_text_model": "openai/whisper-large-v3-turbo"
}
}
```

### Reasoning Blocks

Reasoning blocks stay expanded by default in the VS Code chat UI. Enable **Auto-Collapse Reasoning** in the Display tab, or set `auto_collapse_reasoning` in `kilo.jsonc`, to collapse them after the agent finishes writing them:
Expand Down Expand Up @@ -198,20 +210,9 @@ Available experimental settings include:
- **Share mode** - `manual`, `auto`, or `disabled` session sharing
- **LSP integration** - expose language server diagnostics to the agent
- **Paste summary** - summarize large clipboard pastes before including them
- **Speech to Text Model** - optionally select the transcription model
- **Batch tool** - allow the agent to batch multiple tool calls in one step
- **OpenTelemetry** - enable Kilo telemetry and optional OTLP export when configured

Voice input appears automatically when the Kilo provider is enabled and you are signed in. Choosing **Speech to Text Model** stores `experimental.speech_to_text_model` in your global Kilo CLI config (`~/.config/kilo/kilo.jsonc`):

```json
{
"experimental": {
"speech_to_text_model": "openai/gpt-4o-mini-transcribe"
}
}
```

Advanced options not exposed in the UI can be configured via the `experimental` key in `kilo.jsonc`:

```json
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 0 additions & 4 deletions packages/kilo-vscode/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -902,10 +902,6 @@
"default": false,
"description": "Enable chat textarea autocomplete"
},
"kilo-code.new.speechToText.model": {
"type": "string",
"description": "Model to use for experimental speech-to-text voice input. Requires Kilo Gateway."
},
"kilo-code.new.claudeCodeCompat": {
"type": "boolean",
"default": false,
Expand Down
13 changes: 10 additions & 3 deletions packages/kilo-vscode/src/provider-actions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import {
sanitizeCustomProviderConfig,
withCustomProviderDeletions,
} from "./shared/custom-provider"
import { isCustomProviderPackage, KILO_AUTO, parseModelString } from "./shared/provider-model"
import { isCustomProviderPackage, KILO_AUTO, KILO_PROVIDER_ID, parseModelString } from "./shared/provider-model"
import { configFeatures } from "./features"

/**
Expand Down Expand Up @@ -50,7 +50,7 @@ function same(a: unknown, b: unknown): boolean {
return akeys.every((key, index) => key === bkeys[index] && same(a[key], b[key]))
}

/** Fetch auth methods alongside the provider list. Auth states default to empty (endpoint not yet available). */
/** Fetch provider availability and authentication state without exposing stored credentials. */
export async function fetchProviderData(client: KiloClient, dir: string) {
const authRequest =
typeof client.provider.auth === "function"
Expand All @@ -59,10 +59,15 @@ export async function fetchProviderData(client: KiloClient, dir: string) {
.then((r) => r.data ?? {})
.catch(() => ({}))
: Promise.resolve({})
const kiloRequest = client.kilo
.authStatus({ directory: dir }, { throwOnError: true })
.then((r) => (r.data?.authenticated ? (r.data.type ?? null) : null))
.catch(() => null)

const [{ data: response }, authMethods] = await Promise.all([
const [{ data: response }, authMethods, kiloAuth] = await Promise.all([
client.provider.list({ directory: dir }, { throwOnError: true }),
authRequest,
kiloRequest,
])
const authStates: Record<string, AuthState> = {}
const storedKeys: Record<string, StoredProviderKey> = {}
Expand All @@ -83,6 +88,8 @@ export async function fetchProviderData(client: KiloClient, dir: string) {
delete next.key
return next as (typeof response.all)[number]
})
delete authStates[KILO_PROVIDER_ID]
if (kiloAuth) authStates[KILO_PROVIDER_ID] = kiloAuth
return { response: { ...response, all }, authMethods, authStates, storedKeys }
}

Expand Down
10 changes: 5 additions & 5 deletions packages/kilo-vscode/src/speech-to-text/models.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@ export interface SpeechToTextModelDef {
}

const models: SpeechToTextModelDef[] = [
{
id: "openai/whisper-large-v3-turbo",
label: "Whisper Large V3 Turbo",
provider: "OpenAI-compatible",
},
{
id: "openai/gpt-4o-mini-transcribe",
label: "GPT-4o Mini Transcribe",
Expand All @@ -23,11 +28,6 @@ const models: SpeechToTextModelDef[] = [
label: "Whisper 1",
provider: "OpenAI",
},
{
id: "openai/whisper-large-v3-turbo",
label: "Whisper Large V3 Turbo",
provider: "OpenAI-compatible",
},
{
id: "openai/whisper-large-v3",
label: "Whisper Large V3",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,13 @@ test("settings and mode editing expose distinct model field purposes", async ({
/Override the default model for specific modes/,
)

await load(page, "settings--models-speech-to-text")
const speech = page.getByRole("button", { name: "Speech to Text Model: Chirp 3" })
await expect(speech).toBeEnabled()
await speech.click()
await page.getByRole("option", { name: "GPT-4o Mini Transcribe (OpenAI)" }).click()
await expect(page.getByRole("button", { name: "Speech to Text Model: GPT-4o Mini Transcribe" })).toBeVisible()

await load(page, "settings--mode-edit-export")
await expect(page.getByRole("button", { name: /Model Override:/ })).toHaveAccessibleDescription(
"Override the default model for this agent",
Expand Down
53 changes: 53 additions & 0 deletions packages/kilo-vscode/tests/unit/provider-actions-save.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,9 @@ function createCtx(existing: ExistingGlobal = { disabled_providers: [] }, merged
}),
auth: async () => ({ data: {} }),
},
kilo: {
authStatus: async () => ({ data: { authenticated: false } }),
},
global: {
config: {
get: async () => ({ data: existing }),
Expand Down Expand Up @@ -435,6 +438,9 @@ describe("fetchProviderData", () => {
}),
auth: async () => ({ data: {} }),
},
kilo: {
authStatus: async () => ({ data: { authenticated: false } }),
},
} as unknown as Parameters<typeof fetchProviderData>[0]

const result = await fetchProviderData(client, "/tmp")
Expand All @@ -444,6 +450,50 @@ describe("fetchProviderData", () => {
expect("key" in item).toBe(false)
})

it("uses local Kilo auth status instead of profile availability", async () => {
const client = {
provider: {
list: async () => ({
data: {
all: [{ id: "kilo", name: "Kilo Gateway", source: "custom", env: [], models: {} }],
connected: ["kilo"],
default: { kilo: "kilo-auto/frontier" },
},
}),
auth: async () => ({ data: {} }),
},
kilo: {
authStatus: async () => ({ data: { authenticated: true, type: "oauth" } }),
},
} as unknown as Parameters<typeof fetchProviderData>[0]

const result = await fetchProviderData(client, "/tmp")

expect(result.authStates).toEqual({ kilo: "oauth" })
})

it("does not infer Kilo speech access without stored Gateway auth", async () => {
const client = {
provider: {
list: async () => ({
data: {
all: [{ id: "kilo", name: "Kilo Gateway", source: "config", key: "configured", env: [], models: {} }],
connected: ["kilo"],
default: { kilo: "kilo-auto/frontier" },
},
}),
auth: async () => ({ data: {} }),
},
kilo: {
authStatus: async () => ({ data: { authenticated: false } }),
},
} as unknown as Parameters<typeof fetchProviderData>[0]

const result = await fetchProviderData(client, "/tmp")

expect(result.authStates).toEqual({})
})

it("retains stripped keys for providers with a configured baseURL", async () => {
const client = {
provider: {
Expand Down Expand Up @@ -474,6 +524,9 @@ describe("fetchProviderData", () => {
}),
auth: async () => ({ data: {} }),
},
kilo: {
authStatus: async () => ({ data: { authenticated: false } }),
},
} as unknown as Parameters<typeof fetchProviderData>[0]

const result = await fetchProviderData(client, "/tmp")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,20 @@ import {
import { DEFAULT_SPEECH_TO_TEXT_MODEL } from "../../src/speech-to-text/models"

describe("speech-to-text availability", () => {
const providers = ["kilo"]
const profile = {}
it("shows speech input for stored Kilo credentials", () => {
expect(canUseSpeechToText({}, { kilo: "oauth" })).toBe(true)
expect(canUseSpeechToText({}, { kilo: "api" })).toBe(true)
})

it("shows speech input by default when Kilo access exists", () => {
expect(canUseSpeechToText({}, providers, profile)).toBe(true)
it("hides speech input without usable Kilo credentials", () => {
expect(canUseSpeechToText({}, {})).toBe(false)
expect(canUseSpeechToText({}, { kilo: "wellknown" })).toBe(false)
})

it("hides speech input without a signed-in, enabled Kilo provider", () => {
expect(canUseSpeechToText({}, [], profile)).toBe(false)
expect(canUseSpeechToText({}, providers, null)).toBe(false)
expect(canUseSpeechToText({ disabled_providers: ["kilo"] }, providers, profile)).toBe(false)
it("honors enabled and disabled provider configuration", () => {
expect(canUseSpeechToText({ disabled_providers: ["kilo"] }, { kilo: "oauth" })).toBe(false)
expect(canUseSpeechToText({ enabled_providers: ["openai"] }, { kilo: "oauth" })).toBe(false)
expect(canUseSpeechToText({ enabled_providers: ["kilo"] }, { kilo: "oauth" })).toBe(true)
})

it("normalizes configured and unknown transcription models", () => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ import {
} from "../../src/speech-to-text/models"

describe("speech-to-text model catalog", () => {
it("keeps the first speech model as the fallback default", () => {
it("uses Whisper Large V3 Turbo as the fallback default", () => {
expect(DEFAULT_SPEECH_TO_TEXT_MODEL.id).toBe("openai/whisper-large-v3-turbo")
expect(DEFAULT_SPEECH_TO_TEXT_MODEL.id).toBe(SPEECH_TO_TEXT_MODELS[0]?.id)
})

Expand Down
34 changes: 31 additions & 3 deletions packages/kilo-vscode/tests/unit/use-speech-to-text.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,22 @@ import { describe, expect, it, mock } from "bun:test"
import { createRoot } from "solid-js"
import type { ExtensionMessage, WebviewMessage } from "../../webview-ui/src/types/messages"

type Toast = {
actions?: Array<{ onClick: string | (() => void) }>
}

const toasts: Toast[] = []
mock.module("@kilocode/kilo-ui/toast", () => ({
showToast: () => undefined,
showToast: (toast: Toast) => toasts.push(toast),
}))

const { useSpeechToText } = await import("../../webview-ui/src/components/speech-to-text/useSpeechToText")

function setup() {
const sent: WebviewMessage[] = []
let handler: ((message: ExtensionMessage) => void) | undefined
let logins = 0
toasts.length = 0

const root = createRoot((dispose) => ({
dispose,
Expand All @@ -24,16 +31,37 @@ function setup() {
}
},
},
{ profileData: () => ({}), goToLogin: () => {} },
{ goToLogin: () => logins++ },
{ t: (key) => key },
),
}))

const fire = (message: ExtensionMessage) => handler?.(message)
return { ...root, fire, sent }
return { ...root, fire, sent, logins: () => logins }
}

describe("useSpeechToText", () => {
it("offers sign-in when stored credentials stop authenticating", () => {
const ctx = setup()

ctx.speech.start({ model: "scribe", insert: () => {} })
const start = ctx.sent[0]
if (start?.type !== "speechToTextStart") throw new Error("speech start message missing")

ctx.fire({
type: "speechToTextError",
requestId: start.requestId,
error: "Unauthorized",
code: "not_authenticated",
})
const action = toasts[0]?.actions?.find((item) => typeof item.onClick === "function")
if (typeof action?.onClick === "function") action.onClick()

expect(ctx.logins()).toBe(1)
expect(ctx.speech.error()).toBe("speechToText.error.loginRequired")
ctx.dispose()
})

it("runs the stop completion after inserting a transcript", () => {
const ctx = setup()
const text: string[] = []
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ export const DiffPanel: Component<DiffPanelProps> = (props) => {
const provider = useProvider()
const { config } = useConfig()
const speech = useSpeechToText(vscode, server, { t })
const canUseSpeech = () => canUseSpeechToText(config(), provider.connected(), server.profileData())
const canUseSpeech = () => canUseSpeechToText(config(), provider.authStates())
const speechModel = () => selectedSpeechToTextModel(config())
const isMac = typeof navigator !== "undefined" && /Mac|iPhone|iPad/.test(navigator.userAgent)
const sendAllKeybind = () =>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ export const NewWorktreeDialog: Component<{ onClose: () => void; defaultBaseBran
const [highlightedIndex, setHighlightedIndex] = createSignal(0)
const [variant, setVariant] = createSignal<string | undefined>(session.currentVariant())
const speech = useSpeechToText(vscode, server, { t })
const canUseSpeech = () => canUseSpeechToText(config(), provider.connected(), server.profileData())
const canUseSpeech = () => canUseSpeechToText(config(), provider.authStates())
const speechModel = () => selectedSpeechToTextModel(config())

// Variant list for the currently selected model
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ export const FullScreenDiffView: Component<FullScreenDiffViewProps> = (props) =>
const provider = useProvider()
const { config } = useConfig()
const speech = useSpeechToText(vscode, server, { t })
const canUseSpeech = () => canUseSpeechToText(config(), provider.connected(), server.profileData())
const canUseSpeech = () => canUseSpeechToText(config(), provider.authStates())
const speechModel = () => selectedSpeechToTextModel(config())
const isMac = typeof navigator !== "undefined" && /Mac|iPhone|iPad/.test(navigator.userAgent)
const sendAllKeybind = () =>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -295,7 +295,7 @@ export const PromptInput: Component<PromptInputProps> = (props) => {
const isBusy = () =>
isPromptBusy(session.status(), !!props.suggesting?.(), !!props.questioning?.(), session.submitting())
const isDisabled = () => !server.isConnected()
const canUseSpeech = () => canUseSpeechToText(config(), provider.connected(), server.profileData())
const canUseSpeech = () => canUseSpeechToText(config(), provider.authStates())
const speechModel = () => selectedSpeechToTextModel(config())
const hasInput = () => text().trim().length > 0 || imageAttach.images().length > 0 || reviewComments().length > 0
const canSend = () =>
Expand Down
Loading
Loading