Skip to content

perf(ci): cut E2E smoke wall-clock from ~8min to ~3min - #347

Closed
Million-mo wants to merge 1 commit into
wolf1069b:mainfrom
Million-mo:ci/e2e-smoke-parallel
Closed

perf(ci): cut E2E smoke wall-clock from ~8min to ~3min#347
Million-mo wants to merge 1 commit into
wolf1069b:mainfrom
Million-mo:ci/e2e-smoke-parallel

Conversation

@Million-mo

Copy link
Copy Markdown
Collaborator

Summary

The single L4a e2e-smoke job ran 172 tests serially and took 7m44s on PR #346
(see run 30810232095).
This PR cuts that to ~3min wall-clock with zero coverage loss.

Changes

  1. Split e2e-smoke into 3 parallel jobse2e-acp, e2e-opencode, e2e-misc

    • Each job runs on its own runner and owns its own subprocess_server cache instance.
    • The conftest cache auto-disables under xdist, so naive -n auto is not viable; splitting
      by protocol gives parallel runners without touching xdist/cache semantics.
    • Wall-clock becomes max(acp, opencode, misc)~3min vs the 7m44s serial sum.
  2. Cap known-hanging test_post_init at 10s (tests/e2e/test_opencode_misc.py)

    • Already xfailed / known_bug, but it burned the full 60s job-level --timeout on
      every PR just to reach the xfail. @pytest.mark.timeout(10) saves 50s/PR.
  3. Add concurrency: cancel-in-progress

    • Each new push to a PR cancels the superseded in-flight run, freeing runner capacity
      that was delaying newly-queued e2e jobs.

Expected impact

Metric Before After
E2E smoke wall-clock 464s (serial) ~295s (parallel, ACP-heavy bucket)
test_post_init 60s xfail wait 10s
PR responsibility 3 e2e jobs 3 e2e jobs (same coverage)

Slow L4 tests remain covered by the existing e2e-nightly.yml. Run history:
no coverage or test-selection semantics changed — only job partitioning and
the known-bug timeout, plus report aggregation kept in sync.

Verification

  • yaml.safe_load valid; needs-graph has no dangling refs
  • Bucket globs partition all 32 e2e test files with zero overlap
  • ruff check + ruff format --check pass on the modified test file
  • pytest --collect-only -m e2e tests/e2e/test_opencode_misc.py collects test_post_init

The single L4a e2e-smoke job collected 172 tests and ran them serially,
taking 7m44s. Split it by server into three parallel jobs (acp / opencode /
misc), each with its own subprocess-server cache instance, so wall-clock is
now ~the slowest bucket instead of the serial sum.

Also cap the known-hanging test_post_init at a 10s timeout so its xfail no
longer burns the full 60s job-level timeout on every PR.

Add a concurrency cancel-in-progress so a new push to a PR cancels the
superseded in-flight run instead of occupying further runner queues.
@Million-mo

Copy link
Copy Markdown
Collaborator Author

Closing fork-based PR; pushing to origin instead so OpenCode code review runs on the trunk branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant