JetBrains IDEs - Add Autocomplete Acceptance Telemetry

[catrielmuller]

Context

The JetBrains plugin's autocomplete service was successfully generating and displaying AI-powered code suggestions using the VSCode extension's Ghost service via RPC. However, we had no visibility into whether users were actually accepting these suggestions. While all other telemetry events (suggestion requests, LLM completions, cache hits) were being captured, the critical AUTOCOMPLETE_ACCEPT_SUGGESTION event was never triggered when JetBrains users accepted completions.

This missing telemetry prevented us from:

• Measuring autocomplete acceptance rates for JetBrains users
• Comparing JetBrains vs VSCode user engagement
• Understanding the effectiveness of our autocomplete feature across platforms
• Making data-driven decisions about autocomplete improvements

Implementation

Architecture

The solution leverages IntelliJ Platform's InlineCompletionInsertHandler API, which provides an afterInsertion() callback that's invoked when users accept inline completions. This mirrors the VSCode implementation pattern where InlineCompletionItem has a command property that executes automatically on acceptance.

Key Components

1. Custom Insert Handler (KiloCodeInlineCompletionInsertHandler.kt)

• Extends DefaultInlineCompletionInsertHandler from IntelliJ Platform
• Overrides afterInsertion() to trigger telemetry via RPC
• Executes the VSCode extension's INLINE_COMPLETION_ACCEPTED_COMMAND asynchronously
• Fire-and-forget pattern ensures telemetry doesn't block user experience
• Robust error handling prevents telemetry failures from affecting completion insertion

2. Provider Integration (KiloCodeInlineCompletionProvider.kt)

• Added insertHandler property override to use custom handler
• IntelliJ Platform automatically invokes this handler when completions are accepted
• Minimal changes to existing code (single property addition)

Program Flow

User accepts completion in JetBrains
    ↓
IntelliJ Platform calls afterInsertion()
    ↓
KiloCodeInlineCompletionInsertHandler.afterInsertion()
    ↓
RPC call to VSCode extension
    ↓
INLINE_COMPLETION_ACCEPTED_COMMAND handler
    ↓
AutocompleteTelemetry.captureAcceptSuggestion()
    ↓
TelemetryService.captureEvent()
    ↓
Event logged and sent to analytics

Screenshots

❯ cat /home/catrielmuller/Dev/Kilo-Org/kilocode/jetbrains/plugin/build/idea-sandbox/IC-2024.3/log/idea.log | grep "Autocomplete Telemetry event"

2025-12-19 16:11:01,993 [  72020]   INFO - #ai.kilocode.jetbrains.actors.MainThreadConsole - [Extension Host] [Autocomplete Telemetry event: Autocomplete Suggestion Requested",{"languageId":"typescript","modelId":"mistralai/codestral-2508","provider":"Kilo Gateway"}]
2025-12-19 16:11:02,831 [  72858]   INFO - #ai.kilocode.jetbrains.actors.MainThreadConsole - [Extension Host] [Autocomplete Telemetry event: Autocomplete LLM Request Completed",{"latencyMs":785.7598150000122,"cost":0,"inputTokens":301,"outputTokens":16,"languageId":"typescript","modelId":"mistralai/codestral-2508","provider":"Kilo Gateway","strategy":"fim"}]
2025-12-19 16:11:02,831 [  72858]   INFO - #ai.kilocode.jetbrains.actors.MainThreadConsole - [Extension Host] [Autocomplete Telemetry event: Autocomplete LLM Suggestion Returned",{"languageId":"typescript","modelId":"mistralai/codestral-2508","provider":"Kilo Gateway","strategy":"fim","suggestionLength":29}]
2025-12-19 16:11:05,707 [  75734]   INFO - #ai.kilocode.jetbrains.actors.MainThreadConsole - [Extension Host] [Autocomplete Telemetry event: Autocomplete Accept Suggestion"]
2025-12-19 16:11:05,759 [  75786]   INFO - #ai.kilocode.jetbrains.actors.MainThreadConsole - [Extension Host] [Autocomplete Telemetry event: Autocomplete Suggestion Requested",{"languageId":"typescript","modelId":"mistralai/codestral-2508","provider":"Kilo Gateway"}]
2025-12-19 16:11:05,760 [  75787]   INFO - #ai.kilocode.jetbrains.actors.MainThreadConsole - [Extension Host] [Autocomplete Telemetry event: Autocomplete Suggestion Cache Hit",{"matchType":"partial_typing","languageId":"typescript","modelId":"mistralai/codestral-2508","provider":"Kilo Gateway","suggestionLength":30}]

How to Test

Prerequisites

• JetBrains IDE with Kilo Code plugin installed
• Autocomplete enabled in settings
• Valid API credentials configured

Test Steps

1. Enable Debug Logging (optional)

   • Open JetBrains IDE
   • Go to Help → Diagnostic Tools → Debug Log Settings
   • Add: #ai.kilocode.jetbrains.inline.KiloCodeInlineCompletionInsertHandler

2. Trigger Autocomplete

   • Open a TypeScript/JavaScript file
   • Start typing code
   • Wait for autocomplete suggestion to appear (gray text)

3. Accept Suggestion

   • Press Tab or Enter to accept the suggestion
   • The completion text should be inserted

4. Verify Telemetry

   • Check the IDE log file:

     cat ~/Dev/Kilo-Org/kilocode/jetbrains/plugin/build/idea-sandbox/IC-2024.3/log/idea.log | grep "Autocomplete Telemetry event: Autocomplete Accept Suggestion"

   • You should see ONE log entry per acceptance (not duplicates)

5. Verify Debug Logs (if enabled)

   • Check for insert handler execution:

     cat ~/Dev/Kilo-Org/kilocode/jetbrains/plugin/build/idea-sandbox/IC-2024.3/log/idea.log | grep "Triggered inline completion acceptance telemetry"

Expected Results

• ✅ Autocomplete suggestions appear as before
• ✅ Accepting suggestions works normally
• ✅ One "Autocomplete Accept Suggestion" telemetry event per acceptance
• ✅ No performance degradation
• ✅ No errors in logs

[changeset-bot]

🦋 Changeset detected

Latest commit: 7caa96e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
kilo-code	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[kilo-code-bot]

✅ No Issues Found

16 files reviewed | Confidence: 92% | Recommendation: Merge

Review Details

Key Files Reviewed:

• KiloCodeInlineCompletionInsertHandler.kt (new) - Custom insert handler for telemetry
• KiloCodeInlineCompletionProvider.kt - Integration of insert handler
• GhostServiceManager.ts - Removal of JetBrains conditional for telemetry

Other Files: Formatting/style changes only (trailing commas, whitespace normalization)

Checked:

• ✅ Security: No vulnerabilities found
• ✅ Bug detection: Logic is sound, error handling is appropriate
• ✅ Performance: Fire-and-forget telemetry pattern is correct
• ✅ Architecture: RPC-based telemetry flow is well-designed

Architecture Notes:

• JetBrains plugin now triggers telemetry via RPC command kilocode.ghost.inline-completion.accepted
• VSCode extension always creates AutocompleteTelemetry (conditional removed)
• Command ID is consistent between Kotlin and TypeScript implementations
• Error handling properly isolates telemetry failures from user experience

[markijbema]

Sweet, that's really clean

[kilo-code-bot]

✅ No New Issues

This is an incremental review following the previous bot review. The changes since the last review (commit 3de2547 → 4ee5f0c) appear to be minor and the core implementation remains sound.

Files Reviewed	Confidence
8 files	90%

Review Details

Key Implementation:

• KiloCodeInlineCompletionInsertHandler.kt correctly extends DefaultInlineCompletionInsertHandler and triggers telemetry via RPC
• Command ID kilocode.ghost.inline-completion.accepted matches the registered command in GhostInlineCompletionProvider.ts
• GhostServiceManager.ts change ensures AutocompleteTelemetry is always instantiated (previously null for JetBrains)

Verified:

• ✅ Command ID consistency between Kotlin and TypeScript
• ✅ Error handling isolates telemetry failures from user experience
• ✅ Fire-and-forget pattern is appropriate for telemetry
• ✅ RPC proxy retrieval follows established patterns in the codebase

Other Changes: Formatting only (trailing commas, whitespace normalization in build.gradle.kts)

[kilo-code-bot]

✅ No New Issues

Changes since last review (4ee5f0c → 7caa96e) are a clean refactor moving the command constant to InlineCompletionConstants.kt.

Files Reviewed	Confidence
3 files	95%

Review Details

Latest Commit: Refactored command ID to shared constants file

Verified:

• ✅ Command ID kilocode.ghost.inline-completion.accepted matches TypeScript constant in GhostInlineCompletionProvider.ts:137
• ✅ No functional changes in latest commit
• ✅ Error handling properly isolates telemetry failures
• ✅ RPC-based telemetry flow is well-designed