Support backend-routed autocomplete FIM providers by markijbema · Pull Request #10559 · Kilo-Org/kilocode

markijbema · 2026-05-25T14:38:57Z

Summary

Add autocomplete provider selection for Kilo Gateway, Mistral, and Inception while storing provider/model separately for compatibility.
Route /kilo/fim through the shared backend so VS Code and future clients can use Kilo Gateway or direct BYOK provider credentials.
Preserve existing Kilo Gateway FIM pass-through behavior, support env-backed direct provider keys, and cache the working Mistral/Codestral FIM endpoint in-process.
Replace noisy prompt/response FIM logs with compact provider/model/endpoint request logs.

Tests

bun test test/fim.test.ts test/mistral-fim-endpoint.test.ts
bun run typecheck in packages/kilo-gateway
bun run typecheck in packages/opencode
bun test src/services/autocomplete/tests/settings.spec.ts tests/unit/autocomplete-model-selector.test.ts tests/unit/autocomplete-models-sync.test.ts
bun run check-types:extension
bun run compile in packages/kilo-vscode
bun run script/extract-source-links.ts

markijbema · 2026-05-25T14:41:24Z

-
-    logtime("snippets")
-
-    console.log("[FIM] formattedPrefix:", formattedPrefix)


remvoed this because it spammed the logs

…ete-fim-providers # Conflicts: # packages/kilo-gateway/src/server/routes.ts # packages/sdk/js/src/v2/gen/sdk.gen.ts # packages/sdk/js/src/v2/gen/types.gen.ts

kilo-code-bot · 2026-05-26T10:32:54Z

Code Review Summary

Status: No Issues Found | Recommendation: Merge

What Changed Since Last Review

The only new commit since the previous review is 21b5607bc8 chore: trigger CI — an empty commit with no file changes. All previously reviewed code is unchanged.

Previous Issues — All Carried Forward (no changes)

All 5 existing inline comments remain active on unchanged files. No new issues found.

Files Reviewed (incremental diff — 0 changed files)

No file changes in this increment. All previous review findings carry forward as-is.

Previously reviewed files with active comments:

packages/kilo-vscode/src/services/autocomplete/classic-auto-complete/FillInTheMiddle.ts — existing comment at line 80
packages/kilo-gateway/src/server/fim.ts — 2 existing comments
packages/opencode/src/kilocode/server/httpapi/handlers/kilo-gateway.ts — 1 existing comment
packages/kilo-vscode/src/services/autocomplete/settings.ts — 1 existing comment

_{Reviewed by claude-4.6-sonnet-20260217 · 232,916 tokens}

_{Review guidance: REVIEW.md from base branch main}

Previously `getAutocompleteModel` inferred the provider from a bare model id like "codestral-2508" or "mercury-edit-2" when no provider was set. That meant a user with a legacy `model` setting could be silently routed to a direct BYOK provider without an explicit opt-in. Default to Kilo Gateway whenever provider is unset; direct providers now require an explicit `provider` setting.

Replace `urls: string[]` with provider-specific shapes: kilo and inception each carry a single `url`, mistral carries no urls (its dual-endpoint fallback is encapsulated in requestMistralFim). Removes the non-null assertion on `urls[0]` and the dead "No FIM endpoint configured" branch. Also tightens `getProviderKey` to take `DirectAutocompleteProviderID`, which now follows naturally from the narrowed union and prevents the `DIRECT_FIM_ENV["kilo"]` undefined-lookup footgun.

@markijbema

Addresses @markijbema's architectural feedback on Kilo-Org#10536: moves the HTTP edit-completion call to the gateway, removes the standalone API-key setting, and aligns with the provider/model selection design introduced in Kilo-Org#10559. Wire-level changes ------------------ * New `/kilo/edit` endpoint added to the opencode HttpApi contract (`packages/opencode/src/kilocode/server/httpapi/groups/kilo-gateway.ts`) and mirrored into `packages/kilo-gateway/src/server/routes.ts` for the hono surface. SDK regenerated; `client.kilo.edit({ content, provider, model, maxTokens })` is now available. * `packages/kilo-gateway/src/edit.ts` — `EditTarget` resolver mirroring the FIM pattern. Only the Inception provider is wired today (Mistral doesn't expose a comparable surface); Kilo Gateway has a placeholder branch that returns 400 until a server-side proxy exists. * `packages/kilo-gateway/src/server/edit.ts` — `createEditHandler` reads the Inception BYOK key from `Auth.get("inception")` and falls back to `INCEPTION_API_KEY` from env, exactly like the FIM handler. * The gateway unwraps Mercury's triple-backtick fence (and any `<|code_to_edit|>` sentinels) server-side so the VSCode response is just the rewritten code. * `AutocompleteModelDef` gains an optional `kind: "fim" | "edit"` discriminator; new entry `inception/mercury-next-edit` (label "Mercury Next Edit") sets `kind: "edit"` and shares the wire model `mercury-edit-2`. VSCode-side ----------- * `MercuryEditProvider` no longer does its own HTTP — it's now a thin wrapper around `client.kilo.edit(...)` via `KiloConnectionService`. * Dropped `kilo-code.new.autocomplete.nextEdit.apiKey` and `.nextEdit.baseUrl` settings. Auth and routing live in the gateway. * The `AutocompleteServiceManager` dispatch now switches on the model's `kind` field (set by `getAutocompleteModel(provider, model)`) instead of string-comparing a model id, matching Mark's provider+model split. * The `NextEditInlineCompletionProvider`, `NextEditSuggestionManager`, prompt template, parser, editable-region selector, edit-history tracker, recently-viewed-snippets adapter, and decoration-based jump-to-edit UX remain in the VSCode extension since they need editor-specific APIs (`InlineCompletionItem`, `TextEditorDecorationType`, keybinding context keys). Bot review nits resolved ------------------------ * `INLINE_COMPLETION_ACCEPTED_COMMAND` renamed `kilocode.*` → `kilo-code.*` to match the project convention. * `kilo-code.next-edit.acceptOrJump` and `.dismiss` now declared in `contributes.commands` so VS Code can resolve them in the palette. * `disposeLog()` wired into `AutocompleteServiceManager.dispose()` so the dedicated "Kilo Code · Next Edit" OutputChannel doesn't leak. * Per-keystroke "skip — no API key resolved" log removed (the entire API-key code path is gone). Tests ----- * `bun run check-types:extension` clean * `bun run lint src` clean * `bun test src/services/autocomplete/next-edit/__tests__/` — 23/23 pass Docs ---- * The partner walkthrough at `packages/kilo-vscode/docs/mercury-next-edit-testing.html` and the 20-test playground under `packages/kilo-vscode/docs/nes-examples/` survive from the prior iteration. The walkthrough's "Install the PR locally" section still applies (the model dropdown choice is now "Mercury Next Edit (Inception)" — the API-key step is gone since BYOK is plumbed through the gateway's Auth store). Known follow-ups (not in this commit) ------------------------------------- * `FileIgnoreController` plumbing through the NES context builder so `.env`-style files don't get sent. Hook point identified in `NextEditInlineCompletionProvider.buildRequestContext`. * Settings UI changes in the webview to expose Mercury Next Edit as a selectable provider/model pair alongside the FIM entries. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…ete-fim-providers Support backend-routed autocomplete FIM providers

@markijbema

Addresses @markijbema's architectural feedback on Kilo-Org#10536: moves the HTTP edit-completion call to the gateway, removes the standalone API-key setting, and aligns with the provider/model selection design introduced in Kilo-Org#10559. Wire-level changes ------------------ * New `/kilo/edit` endpoint added to the opencode HttpApi contract (`packages/opencode/src/kilocode/server/httpapi/groups/kilo-gateway.ts`) and mirrored into `packages/kilo-gateway/src/server/routes.ts` for the hono surface. SDK regenerated; `client.kilo.edit({ content, provider, model, maxTokens })` is now available. * `packages/kilo-gateway/src/edit.ts` — `EditTarget` resolver mirroring the FIM pattern. Only the Inception provider is wired today (Mistral doesn't expose a comparable surface); Kilo Gateway has a placeholder branch that returns 400 until a server-side proxy exists. * `packages/kilo-gateway/src/server/edit.ts` — `createEditHandler` reads the Inception BYOK key from `Auth.get("inception")` and falls back to `INCEPTION_API_KEY` from env, exactly like the FIM handler. * The gateway unwraps Mercury's triple-backtick fence (and any `<|code_to_edit|>` sentinels) server-side so the VSCode response is just the rewritten code. * `AutocompleteModelDef` gains an optional `kind: "fim" | "edit"` discriminator; new entry `inception/mercury-next-edit` (label "Mercury Next Edit") sets `kind: "edit"` and shares the wire model `mercury-edit-2`. VSCode-side ----------- * `MercuryEditProvider` no longer does its own HTTP — it's now a thin wrapper around `client.kilo.edit(...)` via `KiloConnectionService`. * Dropped `kilo-code.new.autocomplete.nextEdit.apiKey` and `.nextEdit.baseUrl` settings. Auth and routing live in the gateway. * The `AutocompleteServiceManager` dispatch now switches on the model's `kind` field (set by `getAutocompleteModel(provider, model)`) instead of string-comparing a model id, matching Mark's provider+model split. * The `NextEditInlineCompletionProvider`, `NextEditSuggestionManager`, prompt template, parser, editable-region selector, edit-history tracker, recently-viewed-snippets adapter, and decoration-based jump-to-edit UX remain in the VSCode extension since they need editor-specific APIs (`InlineCompletionItem`, `TextEditorDecorationType`, keybinding context keys). Bot review nits resolved ------------------------ * `INLINE_COMPLETION_ACCEPTED_COMMAND` renamed `kilocode.*` → `kilo-code.*` to match the project convention. * `kilo-code.next-edit.acceptOrJump` and `.dismiss` now declared in `contributes.commands` so VS Code can resolve them in the palette. * `disposeLog()` wired into `AutocompleteServiceManager.dispose()` so the dedicated "Kilo Code · Next Edit" OutputChannel doesn't leak. * Per-keystroke "skip — no API key resolved" log removed (the entire API-key code path is gone). Tests ----- * `bun run check-types:extension` clean * `bun run lint src` clean * `bun test src/services/autocomplete/next-edit/__tests__/` — 23/23 pass Docs ---- * The partner walkthrough at `packages/kilo-vscode/docs/mercury-next-edit-testing.html` and the 20-test playground under `packages/kilo-vscode/docs/nes-examples/` survive from the prior iteration. The walkthrough's "Install the PR locally" section still applies (the model dropdown choice is now "Mercury Next Edit (Inception)" — the API-key step is gone since BYOK is plumbed through the gateway's Auth store). Known follow-ups (not in this commit) ------------------------------------- * `FileIgnoreController` plumbing through the NES context builder so `.env`-style files don't get sent. Hook point identified in `NextEditInlineCompletionProvider.buildRequestContext`. * Settings UI changes in the webview to expose Mercury Next Edit as a selectable provider/model pair alongside the FIM entries. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Mark IJbema added 2 commits May 25, 2026 16:38

fix(vscode): support direct autocomplete FIM providers

33eb706

docs(vscode): clarify autocomplete FIM changeset

900b1fc

markijbema commented May 25, 2026

View reviewed changes

kilo-code-bot Bot mentioned this pull request May 25, 2026

chore: remove URL count from source-links.md to reduce merge conflicts #10560

Merged

Mark IJbema added 16 commits May 25, 2026 16:47

refactor(vscode): route direct FIM through backend

c09c5c4

test(vscode): update autocomplete selector expectations

d635a15

refactor(gateway): extract FIM routing

2bf49bd

refactor(gateway): own autocomplete model catalog

25e89db

fix(gateway): preserve legacy autocomplete model ids

afde386

refactor(gateway): split autocomplete provider and model

4cd463d

fix(vscode): import browser-safe autocomplete catalog

bfda8fe

Merge remote-tracking branch 'origin/main' into mark/direct-autocompl…

08124be

…ete-fim-providers # Conflicts: # packages/kilo-gateway/src/server/routes.ts # packages/sdk/js/src/v2/gen/sdk.gen.ts # packages/sdk/js/src/v2/gen/types.gen.ts

fix(kilo-docs): exclude FIM API endpoints from link check

cc50d76

fix(vscode): use autocomplete provider for chat completions

cef6164

fix(gateway): use env keys for direct FIM providers

19719ba

fix(gateway): preserve FIM gateway model passthrough

e5b2bb8

fix(vscode): infer autocomplete provider from direct models

6fb50a8

fix(gateway): cache working Mistral FIM endpoint

4ca226c

refactor(gateway): clarify Mistral FIM endpoint helper

c073dac

refactor(gateway): simplify Mistral FIM endpoint cache

d02145a

markijbema changed the title ~~Support direct autocomplete FIM providers~~ Support backend-routed autocomplete FIM providers May 26, 2026

Mark IJbema added 3 commits May 26, 2026 11:59

refactor(gateway): remove unshipped autocomplete aliases

9b1407e

refactor(gateway): simplify autocomplete model lookup

5590b43

refactor(gateway): share autocomplete provider types

762eb22

markijbema marked this pull request as ready for review May 26, 2026 10:29

Merge branch 'main' into mark/direct-autocomplete-fim-providers

bbc57f6

kilo-code-bot Bot reviewed May 26, 2026

View reviewed changes

Mark IJbema added 2 commits May 26, 2026 13:08

fix: address autocomplete FIM review findings

923d955

Merge branch 'main' into mark/direct-autocomplete-fim-providers

80e39c2

Mark IJbema added 2 commits May 26, 2026 13:39

chrarnoldus approved these changes May 26, 2026

View reviewed changes

markijbema enabled auto-merge May 26, 2026 11:45

markijbema mentioned this pull request May 26, 2026

Add Mercury Next Edit (NES) as an opt-in autocomplete mode #10536

Merged

6 tasks

markijbema disabled auto-merge May 26, 2026 12:26

markijbema closed this May 26, 2026

markijbema reopened this May 26, 2026

chore: trigger CI

21b5607

markijbema enabled auto-merge May 26, 2026 13:18

markijbema disabled auto-merge May 26, 2026 13:21

markijbema enabled auto-merge May 26, 2026 13:21

markijbema merged commit 022be20 into main May 26, 2026
17 of 20 checks passed

markijbema deleted the mark/direct-autocomplete-fim-providers branch May 26, 2026 13:48

t7tran pushed a commit to t7tran/kilocode that referenced this pull request Aug 14, 2026

Merge pull request Kilo-Org#10559 from Kilo-Org/mark/direct-autocompl…

31f4e29

…ete-fim-providers Support backend-routed autocomplete FIM providers

Navigation Menu

Uh oh!

Support backend-routed autocomplete FIM providers - #10559

Support backend-routed autocomplete FIM providers#10559
markijbema merged 27 commits into
mainfrom
mark/direct-autocomplete-fim-providers

markijbema commented May 25, 2026 •

edited

Loading

Uh oh!

markijbema May 25, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kilo-code-bot Bot commented May 26, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		logtime("snippets")

		console.log("[FIM] formattedPrefix:", formattedPrefix)

Uh oh!

Conversation

markijbema commented May 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Tests

Uh oh!

markijbema May 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kilo-code-bot Bot commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review Summary

What Changed Since Last Review

Previous Issues — All Carried Forward (no changes)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

markijbema commented May 25, 2026 •

edited

Loading

kilo-code-bot Bot commented May 26, 2026 •

edited

Loading