Skip to content

feat(cloud-agent): attribute setup and admission failures by cause - #6057

Merged
eshurakov merged 4 commits into
mainfrom
eshurakov/snappy-lantern
Sep 10, 2026
Merged

feat(cloud-agent): attribute setup and admission failures by cause#6057
eshurakov merged 4 commits into
mainfrom
eshurakov/snappy-lantern

Conversation

@eshurakov

Copy link
Copy Markdown
Contributor

Problem

Cloud Agent session/run telemetry (cloud_agent_sessions, cloud_agent_session_runs) grouped several distinct failure causes into coarse or unknown buckets, so the admin health views could not show why sessions struggled:

  • Workspace setup subtypes collapsed: clone/checkout timeout, network, corrupt pack all became source_control_network; import timeout/failure and unknown setup became workspace_unknown; setup-command timeout and failure became setup_command.
  • initial_queue_full classified as unknown/initial_admission_unknown even though it is a platform capacity signal, not user error.
  • initial_admission_rejected is a catch-all: the boundary discarded the underlying admission code, so platform/user/not-found causes were indistinguishable and all landed in unknown.

Changes

  • worker-utils: add bounded admission failure codes and split the collapsed workspace reasons into distinct failure_reason values (clone/checkout timeout, corrupt pack, setup-command timeout, session import timeout/failure). initial_queue_full now classifies as platform/admission_capacity; initial_admission_rejected classifies from the preserved admission code.
  • cloud-agent-next: preserve the admission code across the setup boundary and pass it to the classifier (classifier input only, not a new column). Deduplicate the admission mapping into one helper shared by the grouped start path and the legacy prepared-admission path.
  • db schema: add the new reason values to the local CloudAgentFailureReason union.
  • web admin health: replace the mixed allFailureRate with separate runFailureRate (failed runs / run outcomes) and setupFailureRate (setup failures / sessions observed); add labels for every new reason.

Notes

  • No migration: failure_reason/failure_responsibility are unconstrained text columns; only enum/type values changed.
  • Known follow-up: packages/db/src/schema.ts keeps its own copy of the reason union (it cannot import worker-utils), so the two must stay in sync; a compile-time parity assertion would prevent drift.
  • Follow-ups from the same discussion (not in this PR): derive provider/sandbox class from sandbox_id, and join cli_sessions_v2 for origin/worktree grouping.

Verification

  • pnpm --filter @kilocode/worker-utils typecheck + test: 668 pass.
  • services/cloud-agent-next pnpm run typecheck clean; pnpm run test: 205 files, 5943 pass / 3 skipped.
  • apps/web pnpm --filter web typecheck clean; jest health-summary.test.ts + admin-cloud-agent-next-router.test.ts: 34 pass.
  • oxlint clean (worker-utils, cloud-agent-next, web); pnpm format:changed and git diff --check clean.

Unit/integration only; no live Worker or end-to-end run was performed.

Cloud Agent session/run telemetry collapsed several distinct failure
causes into coarse or unknown buckets, which hid their real cause from
the admin health views.

- Split previously collapsed workspace reasons into distinct values
  (clone/checkout timeout, corrupt pack, setup-command timeout, session
  import timeout/failure) so the classifier keeps the detail.
- Preserve the bounded admission failure code across the setup boundary
  and classify it, so initial_admission_rejected is attributed instead
  of landing in initial_admission_unknown. Map initial_queue_full to
  platform/admission_capacity rather than unknown.
- Deduplicate the admission failure mapping into one helper shared by
  the grouped start path and the legacy prepared-admission path.
- Separate the health summary into run failure rate (failed runs over
  run outcomes) and setup failure rate (setup failures over sessions
  observed) instead of mixing runs and sessions in one rate.

No migration: failure_reason/failure_responsibility are unconstrained
text columns; only the shared enum values and schema type changed.
Comment thread apps/web/src/routers/admin-cloud-agent-next-router.ts Outdated
@kilo-code-bot

kilo-code-bot Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Incremental review of the two follow-up commits (the setup failure-rate proportion fix and assistant/wrapper failure attribution) found no new issues in changed code; the prior setupFailureRate cohort warning is resolved by the updated denominator.

Files Reviewed (12 files)
  • apps/web/src/app/admin/components/CloudAgentNextTelemetry/health-summary.test.ts
  • apps/web/src/app/admin/components/CloudAgentNextTelemetry/health-summary.ts
  • apps/web/src/lib/code-reviews/terminal-reason-from-failure.test.ts
  • apps/web/src/lib/code-reviews/terminal-reason-from-failure.ts
  • apps/web/src/routers/admin-cloud-agent-next-router.test.ts
  • apps/web/src/routers/admin-cloud-agent-next-router.ts
  • apps/web/src/routers/admin-code-reviews-router.ts
  • packages/db/src/schema-types.ts
  • packages/db/src/schema.ts
  • packages/worker-utils/src/cloud-agent-failure.test.ts
  • packages/worker-utils/src/cloud-agent-failure.ts
  • services/cloud-agent-next/src/telemetry/queue-reports.test.ts
Previous Review Summary (commit 66d2b2b)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 66d2b2b)

Status: 1 Issue Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
apps/web/src/routers/admin-cloud-agent-next-router.ts 194 setupFailureRate mixes created_at and failure_at cohorts
Files Reviewed (15 files)
  • apps/web/src/app/admin/components/CloudAgentNextTelemetry/health-summary.test.ts
  • apps/web/src/app/admin/components/CloudAgentNextTelemetry/health-summary.ts
  • apps/web/src/routers/admin-cloud-agent-next-router.test.ts
  • apps/web/src/routers/admin-cloud-agent-next-router.ts - 1 issue
  • packages/db/src/schema.ts
  • packages/worker-utils/src/cloud-agent-failure.test.ts
  • packages/worker-utils/src/cloud-agent-failure.ts
  • services/cloud-agent-next/src/router.test.ts
  • services/cloud-agent-next/src/session-prepare.test.ts
  • services/cloud-agent-next/src/session/admission-failure.ts
  • services/cloud-agent-next/src/session/legacy-prepared-admission.ts
  • services/cloud-agent-next/src/session/session-registration.ts
  • services/cloud-agent-next/src/telemetry/report-store.test.ts
  • services/cloud-agent-next/src/telemetry/report-store.ts
  • services/cloud-agent-next/src/telemetry/session-reports.ts

Fix these issues in Kilo Cloud


Reviewed by deepseek-v4.1-flash · Input: 0 · Output: 0 · Cached: 0

Review guidance: REVIEW.md from base branch main

The rate divided sessions created in the window by setup failures
counted by failure_at, so a session created just before the window
that failed inside it was in the numerator but not the denominator,
letting the rate exceed 100%. Count sessions exposed to setup in the
window (created in it, or failed in it) so failures stay a subset.
… failures

Split the remaining undifferentiated failure buckets by cause and
ownership so the admin health view attributes them correctly:

- assistant limits: assistant_invalid_request/context_limit/output_limit
  by provider ownership (byok user, managed platform, unknown unknown),
  assistant_content_filter user, assistant_structured_output platform
- source-control clone/checkout/network/corrupt and session-import
  failures are platform infrastructure
- user_interrupt user; container_shutdown and system_interrupt platform
- wrapper_disconnected, wrapper_startup, wrapper_crash, assistant_no_reply
- provider_ownership_unknown when an ownership-dependent assistant
  failure has no known provider ownership

Adds the matching code-review terminal reasons, labels, and tests. The
old generic reason values stay as deprecated union entries so historical
rows keep a label during retention.
# Conflicts:
#	services/cloud-agent-next/src/session/session-registration.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants