Skip to content

fix(cli): bound Telemetry.shutdown so unreachable PostHog endpoint cannot block CLI exit - #9807

Merged
markijbema merged 8 commits into
Kilo-Org:mainfrom
truffle-dev:fix/cli-best-effort-telemetry-shutdown-on-exit-9788
Jun 17, 2026
Merged

fix(cli): bound Telemetry.shutdown so unreachable PostHog endpoint cannot block CLI exit#9807
markijbema merged 8 commits into
Kilo-Org:mainfrom
truffle-dev:fix/cli-best-effort-telemetry-shutdown-on-exit-9788

Conversation

@truffle-dev

Copy link
Copy Markdown
Contributor

Closes #9788.

kilo --help (and other short-lived commands) hangs in the finally-block telemetry shutdown for tens of seconds when us.i.posthog.com is unreachable: offline, behind a firewall, or DNS-blackholed. Reporter's timeout 6s kilo --help exits non-zero on the broken path; toggling KILO_TELEMETRY_LEVEL=none makes it exit fast. Reporter's hypothesis (telemetry flush blocks process exit) checks out against the source.

Root cause

Two issues in packages/kilo-telemetry/src/client.ts:71-78:

  1. Unbounded explicit flush() before shutdown(). posthog-node 4.4.0 retries each request up to 3x with 3s delays plus 10s per attempt before throwing PostHogFetchNetworkError (see node_modules/posthog-node/lib/index.esm.js:1338-1366 and the _retryOptions defaults at retryCount: 3, retryDelay: 3000 plus requestTimeout: 10000). When the endpoint is unreachable, await client.flush() blocks for up to ~36s before throwing, and that hang happens before shutdown()'s outer cap is ever reached.

  2. No timeout parameter at the Telemetry layer. Telemetry.shutdown() and Client.shutdown() accepted no arguments, so the CLI bootstrap had no way to bound the wait at the call site. PostHog's underlying shutdown(shutdownTimeoutMs = 30000) defaults to 30 seconds.

Fix

packages/kilo-telemetry/src/client.ts:

  • Drop the explicit flush() call. PostHog's shutdown(timeoutMs) drains the queue internally (see the doShutdown loop in posthog-core), so the prior await client.flush() is redundant on the happy path and harmful on the unreachable path.
  • Add an optional timeoutMs parameter, threaded through to client.shutdown(timeoutMs).
  • Wrap in try/finally so client = null runs even if shutdown rejects.

packages/kilo-telemetry/src/telemetry.ts:

  • Add the same optional timeoutMs parameter, passed through to Client.shutdown.

packages/opencode/src/index.ts:

  • Pass 2000 from the CLI bootstrap's finally-block. Two seconds is generous for a working endpoint and keeps short-lived commands like kilo --help / kilo --version snappy when the endpoint is unreachable.

Sibling check

Telemetry.shutdown( has exactly one caller in the repo (packages/opencode/src/index.ts:319). Client.shutdown( has one caller (packages/kilo-telemetry/src/telemetry.ts:245). The new optional parameter is a backward-compatible API extension; existing callers that omit it preserve PostHog's 30s default.

Test

packages/kilo-telemetry/src/__tests__/telemetry.test.ts: new test Telemetry.shutdown timeout (#9788) mocks posthog-node so the fake flush() would hang for 60s if called. Asserts:

  • flush() is never invoked (the explicit unbounded call is gone)
  • the supplied timeoutMs (50ms) is threaded through to PostHog's shutdown(timeoutMs)
  • total elapsed time stays well under 1s
$ bun test
 14 pass
 0 fail
 35 expect() calls
Ran 14 tests across 1 file. [228.00ms]

Typecheck passes (bun run typecheck clean for both packages/kilo-telemetry and packages/opencode).

…nnot block CLI exit

When the PostHog endpoint is unreachable (offline, firewall, DNS adblock
resolving us.i.posthog.com to 0.0.0.0), `kilo --help` and other short-
lived commands hang in the finally-block telemetry shutdown for tens of
seconds before the process exits.

Two issues:

1. `Client.shutdown()` called `await client.flush()` before
   `await client.shutdown()`. The explicit flush() is unbounded:
   posthog-node 4.4.0 retries each request up to 3x with 3s delays plus
   10s per attempt before throwing PostHogFetchNetworkError, blocking
   process exit before shutdown's outer cap kicks in.

2. `Telemetry.shutdown()` took no timeout, so the CLI bootstrap had no
   way to bound the wait at the call site. PostHog's
   `shutdown(shutdownTimeoutMs = 30000)` defaults to 30 seconds.

Fix:

- Drop the explicit `flush()` call. PostHog's `shutdown(timeoutMs)`
  drains the queue internally and is bounded by `shutdownTimeoutMs`.
- Add an optional `timeoutMs` parameter to both `Client.shutdown` and
  `Telemetry.shutdown`, threaded through to PostHog.
- Pass `2000` from the CLI bootstrap so short-lived commands exit
  quickly when telemetry is unreachable, while keeping a generous
  budget for the working-endpoint case.

Test: `Telemetry.shutdown timeout (Kilo-Org#9788)` mocks `posthog-node` so the
fake `flush()` would hang for 60s if called. The test asserts (a)
`flush()` is never invoked and (b) the supplied `timeoutMs` (50ms) is
threaded through to PostHog's `shutdown(timeoutMs)`. Total elapsed time
stays well under 1s.

Closes Kilo-Org#9788
import { describe, test, expect, beforeEach, mock } from "bun:test"
import { Identity } from "../identity.js"
import { TelemetryEvent } from "../events.js"
import { Telemetry } from "../telemetry.js"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Static import prevents the PostHog mock from replacing the client under test

This file imports Telemetry before mock.module("posthog-node", ...) runs, which also loads ../client.js and binds it to the real PostHog import. The later dynamic imports return the cached modules, so Client.init() will not use this fake class and the test can either hit the real client or fail to assert the intended shutdown call. Move the telemetry/client imports into the test after the mock is registered (and remove this top-level import) so the mocked posthog-node is the dependency used by Client.init().

@kilo-code-bot

kilo-code-bot Bot commented May 4, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No New Issues Found | Recommendation: Merge

Incremental diff (93941001..9b12e1ca29): Removes the unused @/effect/app-runtime mock from cli-shutdown.test.ts — the code under test (KiloCli.shutdown()) does not use AppRuntime, so the mock was dead weight. Clean change.

Previously-reported WARNING on telemetry.test.ts:4 is resolved: the shutdown test was moved to telemetry-shutdown.test.ts in an earlier commit, so the static Telemetry import on line 4 no longer interferes with any PostHog mock.

Carried-Forward Observation (unchanged, not in diff)
File Line Issue
packages/kilo-telemetry/src/telemetry.ts 274 SUGGESTION: FeedbackProperties uses ID suffix (providerID, modelID, sessionID, messageID, parentMessageID) while the rest of the telemetry module consistently uses Id suffix (modelId, sessionId, taskId). This means feedback events will use different property naming conventions in PostHog, complicating cross-event dashboards. Consider renaming to providerId, modelId, sessionId, messageId, parentMessageId unless constrained by an external API contract.
Files Reviewed (7 files)
  • packages/kilo-telemetry/src/client.ts - No new issues
  • packages/kilo-telemetry/src/telemetry.ts - No new issues in diff; prior SUGGESTION carries forward
  • packages/kilo-telemetry/src/__tests__/telemetry-shutdown.test.ts - No new issues
  • packages/kilo-telemetry/src/__tests__/telemetry.test.ts - Prior WARNING resolved
  • packages/opencode/src/kilocode/cli/setup.ts - No new issues
  • packages/opencode/test/kilocode/cli-shutdown.test.ts - No new issues (AppRuntime mock cleanup)
  • .changeset/preserve-cli-telemetry-exit.md - No issues

Fix these issues in Kilo Cloud

Previous Review Summaries (2 snapshots, latest commit 9394100)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 9394100)

Status: 1 Issue Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 1
Issue Details (click to expand)

WARNING

File Line Issue
packages/kilo-telemetry/src/__tests__/telemetry.test.ts 4 Static import prevents the PostHog mock from resolving before module load — moved shutdown test to telemetry-shutdown.test.ts to work around this (unchanged since last review)
Other Observations (not in diff)

Issues found in unchanged code that cannot receive inline comments:

File Line Issue
packages/kilo-telemetry/src/telemetry.ts 274 SUGGESTION: FeedbackProperties uses ID suffix (providerID, modelID, sessionID, messageID, parentMessageID) while the rest of the telemetry module consistently uses Id suffix (modelId, sessionId, taskId). This means feedback events will use different property naming conventions in PostHog, complicating cross-event dashboards. Consider renaming to providerId, modelId, sessionId, messageId, parentMessageId unless constrained by an external API contract.
Files Reviewed (7 files)
  • packages/kilo-telemetry/src/client.ts - No new issues
  • packages/kilo-telemetry/src/telemetry.ts - No new issues in diff
  • packages/kilo-telemetry/src/__tests__/telemetry-shutdown.test.ts - No new issues
  • packages/kilo-telemetry/src/__tests__/telemetry.test.ts - Existing WARNING carries forward
  • packages/opencode/src/kilocode/cli/setup.ts - No new issues
  • packages/opencode/test/kilocode/cli-shutdown.test.ts - No new issues
  • .changeset/preserve-cli-telemetry-exit.md - No issues

Fix these issues in Kilo Cloud

Previous review (commit 2723c08)

Status: No Issues Found | Recommendation: Merge

Files Reviewed (2 files)
  • packages/kilo-telemetry/src/__tests__/telemetry-shutdown.test.ts
  • packages/kilo-telemetry/src/__tests__/telemetry.test.ts

Reviewed by deepseek-v4-pro-20260423 · 220,913 tokens

Review guidance: REVIEW.md from base branch main

The new shutdown timeout test in telemetry.test.ts shares a file with
tests that statically import `Telemetry` (and thus `client.ts` and
`posthog-node`). Bun's `mock.module()` invalidates the dependency
cache for subsequent dynamic imports so the test passes today, but the
behavior depends on test ordering and mock-cache-invalidation timing
rather than test isolation. Extract the shutdown test into its own file
where `mock.module("posthog-node", ...)` runs before any client.ts
import, so the mock is the source of truth from module load.
@truffle-dev

Copy link
Copy Markdown
Contributor Author

Addressed the test-isolation flag — moved the shutdown timeout test into its own file (telemetry-shutdown.test.ts) so mock.module("posthog-node", ...) runs before any import of client.ts. The mock is now the source of truth at module load instead of relying on bun:test's cache invalidation when the static Telemetry import already resolved the real module. Same assertions, same expected behavior. 2723c08

@johnnyeric
johnnyeric requested a review from markijbema May 4, 2026 11:12
slamj1 pushed a commit to slamj1/kilocode that referenced this pull request May 16, 2026
jliounis pushed a commit to jliounis/kilocode that referenced this pull request May 18, 2026
@truffle-dev

Copy link
Copy Markdown
Contributor Author

Friendly check-in. Bot review cleared on May 4; happy to rebase or split if anything would help it land.

@markijbema
markijbema enabled auto-merge June 17, 2026 14:46
@markijbema
markijbema merged commit 3b1ff4f into Kilo-Org:main Jun 17, 2026
19 checks passed
t7tran pushed a commit to t7tran/kilocode that referenced this pull request Aug 14, 2026
t7tran pushed a commit to t7tran/kilocode that referenced this pull request Aug 14, 2026
…-telemetry-shutdown-on-exit-9788

fix(cli): bound Telemetry.shutdown so unreachable PostHog endpoint cannot block CLI exit
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CLI hangs after printing help when telemetry endpoint is unreachable

2 participants