Skip to content

perf: parallelize local environment sync - #5256

Merged
RSO merged 1 commit into
mainfrom
perf/parallel-env-sync
Aug 13, 2026
Merged

perf: parallelize local environment sync#5256
RSO merged 1 commit into
mainfrom
perf/parallel-env-sync

Conversation

@RSO

@RSO RSO commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Summary

  • parallelize local Secrets Store discovery across independent Wrangler persistence directories
  • parallelize Secrets Store creation across independent persistence directories while locking shared SQLite state
  • add --missing-secrets-only for fast worktree setup without refreshing existing source-backed secrets
  • add a configurable KILO_ENV_SYNC_CONCURRENCY limit and document the workflow

Performance

  • warm all-service dev:env --check: 20.9s → 7.1s locally

Safety

  • secret values are passed to Wrangler over stdin and never placed in argv
  • operations sharing one resolved .wrangler directory remain serialized and cross-process locked; Wrangler 4.112.0 produced SQLITE_READONLY failures when tested with concurrent writers to one local store
  • missing-only mode rechecks each secret under the lock before creating it

Testing

  • node --import tsx --test dev/local/*.test.ts dev/local/env-sync/*.test.ts dev/local/scripts/*.test.ts (212 passed)
  • focused TypeScript check for changed env-sync files
  • focused oxlint and oxfmt checks
  • git diff --check

@RSO
RSO marked this pull request as ready for review August 13, 2026 16:45
child.stderr.on('data', chunk => (stderr += chunk));
child.once('error', reject);
child.once('close', code => resolve({ status: code ?? 1, stderr, stdout }));
child.stdin.end(input);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Unhandled 'error' event on child.stdin can crash the process and mask the real error

If the spawned wrangler exits before consuming stdin (e.g. it fails fast on an invalid store ID) — or the spawn itself fails — writing/closing stdin emits an EPIPE/stream error on child.stdin. With no error listener attached, Node raises it as an unhandled exception, killing the sync instead of surfacing wrangler's stderr via the result.status !== 0 path in createSecretsStoreSecret. The previous spawnSync implementation handled this internally, so this is a regression. Attach a no-op error handler before ending stdin, e.g.:

Suggested change
child.stdin.end(input);
child.stdin.on('error', () => {});
child.stdin.end(input);

Reply with @kilocode-bot fix it to have Kilo Code address this issue.

@kilo-code-bot

kilo-code-bot Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: 1 Issue Found | Recommendation: Address before merge

Executive Summary

The new async runWrangler spawn helper in dev/local/env-sync/output.ts lacks an error handler on child.stdin, so an early-exiting or failed wrangler process crashes the sync with an unhandled EPIPE instead of surfacing the real stderr error.

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
dev/local/env-sync/output.ts 299 Unhandled 'error' on child.stdin crashes the process (EPIPE) and masks wrangler's real stderr error
Files Reviewed (9 files)
  • DEVELOPMENT.md - 0 issues
  • dev/local/cli.ts - 0 issues
  • dev/local/env-sync/concurrency.ts - 0 issues
  • dev/local/env-sync/index.ts - 0 issues
  • dev/local/env-sync/output.test.ts - 0 issues
  • dev/local/env-sync/output.ts - 1 issue
  • dev/local/env-sync/plan.test.ts - 0 issues
  • dev/local/env-sync/plan.ts - 0 issues
  • dev/local/mobile-workflow.test.ts - 0 issues

The concurrency limiting, per-persistence-dir grouping, cross-process locking, and --missing-secrets-only recheck-under-lock logic are otherwise sound; secret values stay on stdin and out of argv as before.

Fix these issues in Kilo Cloud


Reviewed by kimi-k3 · Input: 168.8K · Output: 6K · Cached: 259.7K

Review guidance: REVIEW.md from base branch main

@RSO
RSO merged commit e18f45c into main Aug 13, 2026
15 checks passed
@RSO
RSO deleted the perf/parallel-env-sync branch August 13, 2026 19:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants