[DATA-8187] fix(vscode): sample repetitive autocomplete failure telemetry

[pedroheyerdahl]

Issue

No linked issue. This change reduces the number of autocomplete failure events sent to PostHog.

Context

Autocomplete sends an event for each failed request. Keep a random 10% of those failure events to reduce telemetry volume while keeping a sample for investigation. Successful completions and other events are still sent in full.

Implementation

The VS Code extension drops unselected Autocomplete LLM Request Failed events before sending them to the CLI. Selected events keep their existing properties and add autocomplete_failure_sample_rate: 0.1.

This applies to inline autocomplete, chat-textarea autocomplete, and next-edit suggestions. Reports must multiply sampled failure counts by 10 when comparing them with completion counts. Events from older clients without the sample-rate property still count as 1 each. Dropped events cannot be inspected individually.

Screenshots / Video

Output from a local HTTP receiver using the actual extension telemetry proxy. The check used synthetic events, simulated VS Code telemetry consent, and fixed random values to make the result repeatable. The screenshot shows that output in Chrome.

How to Test

Local verification

Checked on September 25 at 69c72baf:

• The local receiver received 10 of 100 failures and all 100 completions. Retained failures had the correct sample-rate property. Events stayed local.
• Existing telemetry tests: 9 passed. Extension, CLI and SDK builds, typechecks, lint and git diff --check passed.
• The extension loaded in an isolated VS Code window. Autocomplete was not triggered through the editor.
• Full extension suite: 6,600 passed, 2 skipped, 18 failed. The failures occurred in the Chromium browser suite after a timeout. All 37 browser tests passed twice when run separately with Playwright's bundled Chromium. The failure during the full run remains unresolved.

Reviewer test steps

1. From packages/kilo-vscode, run bun test tests/unit/telemetry-proxy-utils.test.ts.
2. Check that only failure events are sampled and retained failures include the sample-rate property.
3. Update any failure-rate reports to account for the 10% sample before release.

Checklist

• Issue linked above, or exception explained
• Tests/verification described
• Screenshots/video included for visual changes, or marked N/A
• Changeset considered for user-facing changes
• I personally reviewed the diff and can explain the changes, including any AI-assisted work.

Get in Touch

Follow up in this PR. Prepared with Codex; human review is pending.

[kilo-code-bot]

Code Review Summary

Status: 1 Issue Found | Recommendation: Address before merge

Overview

Severity	Count
CRITICAL	0
WARNING	1
SUGGESTION	0

Issue Details (click to expand)

WARNING

File	Line	Issue
packages/kilo-vscode/src/services/telemetry/telemetry-proxy-utils.ts	17	New 10% failure-sampling behavior and the autocomplete_failure_sample_rate property have no test coverage; PR test changes only add optional chaining for the nullable return, and the sampling tests from earlier revisions were removed.

Files Reviewed (4 files)

• .changeset/calm-hogs-sample.md - no issues
• packages/kilo-vscode/src/services/telemetry/telemetry-proxy-utils.ts - 1 issue
• packages/kilo-vscode/src/services/telemetry/telemetry-proxy.ts - no issues (null return correctly short-circuits before fetch)
• packages/kilo-vscode/tests/unit/telemetry-proxy-utils.test.ts - no issues

Fix these issues in Kilo Cloud

---

_{Reviewed by deepseek-v4.1-flash · Input: 0 · Output: 0 · Cached: 0}

_{Review guidance: REVIEW.md from base branch main}