Skip to content

Split monolithic CI job into parallel jobs - #198

Merged
brianramseyau merged 5 commits into
mainfrom
fix/slow-ci-pipelines
Sep 4, 2026
Merged

brianramseyau merged 5 commits into
mainfrom
fix/slow-ci-pipelines

Conversation

@brianramseyau

Copy link
Copy Markdown
Owner

Summary

  • Splits the single "Lint, typecheck, test" job (test.yml) into parallel jobs — lint, build-shared, typecheck, and a test matrix over [api, web, desktop, shared] — so independent work runs concurrently instead of serialized in one job.
  • Extracts the repeated checkout+pnpm+node+install preamble in ci.yml into a composite action (.github/actions/setup) to stop it drifting across jobs.
  • Adds an actions/cache for Playwright's browser binaries, shared between the web test leg and the e2e job, and has e2e reuse the shared-dist build artifact instead of rebuilding packages/shared from scratch.

Test plan

  • Watch this PR's own Actions run: confirm lint, build-shared, typecheck, and the 4 test matrix legs show as separate jobs running in parallel
  • Confirm e2e and docker-smoke still gate correctly on the full test.yml fan-out
  • Compare overall PR wall-clock time against a recent main run

🤖 Generated with Claude Code

brianramseyau and others added 2 commits September 4, 2026 20:28
The PR gate ran lint, typecheck, and every workspace's tests as
sequential steps in one job, serializing independent work and making
every PR wait on the slowest step before e2e/docker-smoke could even
start. Split test.yml into lint / build-shared / typecheck / a test
matrix (api, web, desktop, shared) so independent work runs in
parallel, extracted the repeated checkout+pnpm+node+install steps in
ci.yml into a composite action to stop them drifting, and added a
Playwright browser cache shared between the web test leg and e2e.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The runner reads a local composite action's own action.yml off disk
before running any of its steps, so a checkout step can't be the
first thing inside the action — nothing's checked out yet to read it
from. Moved checkout back to being an explicit first step in every
job that calls .github/actions/setup.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
test:
name: Lint, typecheck, test
lint:
name: Lint

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Splitting the single test job into seven named jobs changes the required status-check names that branch protection on main must reference

The old gate exposed one check, test / Lint, typecheck, test. After this change the reusable workflow produces test / Lint, test / Build packages/shared, test / Typecheck, test / Test (api), test / Test (web), test / Test (desktop), and test / Test (shared). Branch protection is configured in GitHub repo settings (ci.yml already notes it isn't expressible in the workflow file), so the required-check list has to be updated in the same merge — otherwise the gate either blocks every future PR (the old required name no longer matches any produced check) or, if the old entry is just removed, silently stops enforcing this gate on main. native-build.yml calls the same test.yml and is affected the same way (its needs: test jobs keep working, but the check names it fans out to change too).


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup
- run: pnpm -r lint

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: lint now runs without build-shared having produced packages/shared/dist

This is fine today — the workspaces' ESLint configs only enable non-type-aware rules (apps/web/eslint.config.js sets projectService: true for .svelte* files but no type-checked rules are enabled, so a missing dist doesn't surface as a lint error). It is a latent coupling though: apps/api and apps/web import @everylist/shared via exports → dist/, so if a type-aware rule (e.g. recommended-type-checked) is ever enabled, the now-independent lint job would start failing on the missing build. Worth a one-line note so nobody either re-serializes lint on build-shared or breaks it accidentally later.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

@kilo-code-bot

kilo-code-bot Bot commented Sep 4, 2026 •

Copy link
Copy Markdown

Code Review Summary

Status: 1 Issue Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 1
Issue Details (click to expand)

SUGGESTION

File Line Issue
apps/api/tests/bootstrap.ts 74 Swallowing a now-deterministic ace.boot() failure masks a root cause that should be investigated rather than silenced
Files Reviewed (1 file)
  • apps/api/tests/bootstrap.ts - 1 issue

Fix these issues in Kilo Cloud

Previous Review Summaries (3 snapshots, latest commit 93c7b13)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 93c7b13)

Status: 3 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 2
SUGGESTION 1
Issue Details (click to expand)

WARNING

File Line Issue
.github/workflows/test.yml 49 Splitting the single test job into seven named jobs changes the branch-protection required-check names; repo settings must be updated in the same merge or the main gate breaks/weakens
.github/workflows/ci.yml 66 docker-smoke builds the full production image but is capped at 15 minutes; a cold-cache run (gha layer-cache miss) can exceed it and fail spuriously

SUGGESTION

File Line Issue
.github/workflows/ci.yml 30 e2e's 15-minute cap may be tight on a cold Playwright cache (playwright install --with-deps + web build + full E2E suite)
Files Reviewed (2 files)
  • .github/workflows/ci.yml - 2 issues
  • .github/workflows/test.yml - 1 issue

Fix these issues in Kilo Cloud

Previous review (commit 08971f1)

Status: 1 Issue Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
.github/workflows/test.yml 49 Splitting the single test job into seven named jobs changes the branch-protection required-check names; repo settings must be updated in the same merge or the main gate breaks/weakens
Files Reviewed (1 file)
  • .github/workflows/test.yml - 1 issue

Fix these issues in Kilo Cloud

Previous review (commit af47090)

Status: 2 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 1
Issue Details (click to expand)

WARNING

File Line Issue
.github/workflows/test.yml 49 Splitting the single test job into seven named jobs changes the branch-protection required-check names; repo settings must be updated in the same merge or the main gate breaks/weakens

SUGGESTION

File Line Issue
.github/workflows/test.yml 54 lint now runs independent of build-shared; fine today (non-type-aware rules), but a latent coupling if type-checked lint rules are ever enabled
Files Reviewed (3 files)
  • .github/actions/setup/action.yml - 0 issues
  • .github/workflows/ci.yml - 0 issues
  • .github/workflows/test.yml - 2 issues

Fix these issues in Kilo Cloud


Reviewed by deepseek-v4-pro-0813 · Input: 31.8K · Output: 15.8K · Cached: 399.2K

Review guidance: REVIEW.md from base branch main

brianramseyau and others added 2 commits September 4, 2026 20:42
Per Kilo review: today's ESLint configs enable no type-aware rules, so
a missing packages/shared/dist doesn't surface as a lint error — but
it's a latent coupling worth flagging for whoever adds one later.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
None of these jobs had an explicit timeout, so a hang (like the one
apps/api's test job is currently hitting in CI, despite passing
cleanly and fast locally — 531/531 in ~20s) would otherwise run
against GitHub's 360-minute default before failing, with no logs
available to diagnose it while in progress. A tight bound makes a
hang fail fast with a real log instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Comment thread .github/workflows/ci.yml
Comment thread .github/workflows/ci.yml
apps/api's test bootstrap eagerly boots the shared ace kernel in a
global Japa setup hook (added in #193) to close a race between the
unit/functional suites' concurrent first exec() calls. In CI it's now
throwing the exact "Invalid command exported from demo_seed.js file.
Invalid URL" error #193 was meant to prevent — deterministically, on
every run, not intermittently — and since nothing catches it, Japa's
global setup hook just stalls the whole job for the rest of its
timeout instead of failing.

Locally this boot call succeeds every time (531/531 tests passing,
~20s) — so whatever's throwing in CI is environment-specific (Linux
runner vs local macOS), not a genuine bug in demo_seed.ts's command
metadata. Rather than chase that down blind, wrap the warm-up in
try/catch: it exists purely to avoid a race, not to make the suites
work at all, so a failure here should fall back to the pre-#193
behavior (ace boots lazily on first real exec(), same as before) —
not take the whole run down.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
try {
const ace = await app.container.make('ace')
await ace.boot()
} catch (error) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: Swallowing a now-deterministic ace.boot() failure masks a root cause that should be investigated rather than silenced

The comment notes the "Invalid command exported... Invalid URL" validation error is now surfacing deterministically (not intermittently) "for reasons that look environment-specific" — but no root cause is pinned down and nothing links to an issue tracking it. The catch also swallows every boot error, not just the transient race. Because migration:run's loader registers ahead of the throwing commands/ FsLoader, the suite still goes green while this error keeps firing, so a real command-metadata bug in apps/api/commands/* (e.g. demo_seed.ts / openapi_generate.ts) would be invisible here. That same FsLoader scan runs on production boot (docker/root/etc/cont-init.d/30-migrate / 35-demo-seed), where there's no catch — so a real registration bug would still surface (or silently no-op) in prod while tests pass. Worth narrowing the catch to the known transient error and/or filing a follow-up to root-cause the deterministic failure, rather than swallowing it indefinitely.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

@brianramseyau
brianramseyau merged commit d2dad89 into main Sep 4, 2026
10 checks passed
@brianramseyau
brianramseyau deleted the fix/slow-ci-pipelines branch September 4, 2026 12:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant