Skip to content

fix(ci): e2e gates always emit a result so auto-promote can read it - #2203

Merged
hongmingwang-moleculeai merged 1 commit into
stagingfrom
fix/e2e-gates-always-emit-result
Apr 28, 2026
Merged

fix(ci): e2e gates always emit a result so auto-promote can read it#2203
hongmingwang-moleculeai merged 1 commit into
stagingfrom
fix/e2e-gates-always-emit-result

Conversation

@HongmingWang-Rabbit

Copy link
Copy Markdown
Contributor

Summary

auto-promote-staging.yml's gate-check at line 99 treats "workflow didn't run" the same as "workflow failed":

if [ "$RESULT" != "completed/success" ]; then
  ALL_GREEN=false
fi

E2E Staging Canvas (Playwright) and E2E API Smoke Test both have paths: filters on their triggers. A push that doesn't touch the watched paths never triggers the workflow, so auto-promote sees missing/none, marks all_green=false, and aborts. Every platform-only or test-only push dead-locks here. This bit us today on PR #2201 (touched only tests/e2e/test_staging_full_saas.sh) — required a manual staging → main bridge (PR #2202) to unstick.

Design B — always-run + fast-skip

For each gate workflow:

  • Drop paths: from the push: and pull_request: triggers
  • Add a detect-changes job using dorny/paths-filter@v3, scoped to the same paths the trigger filter used to watch
  • Gate the real-work job on needs.detect-changes.outputs.<filter> matching true
  • Add a sibling no-op job that runs when the filter output is false and exits success, so the workflow run's conclusion is success either way

workflow_dispatch and the weekly canvas schedule short-circuit detect-changes to always-run — those triggers exist to exercise the suite, not to be silently no-op'd.

Why this over Design A (smarter auto-promote)

Design A (considered + rejected): teach auto-promote-staging.yml to read each gate's paths: filter and treat "no run because filter excluded the commit" as conditional pass. Couples auto-promote to other workflows' YAML schema; breaks silently if a gate is renamed or its filter changes. Design B keeps the auto-promote contract simple ("each gate emits success") and makes each gate self-describing.

Aligns with the user-stated principle: long-term, robust, fully automated, eliminate human error.

Cost

~10-30s of runner time per gate per push for the no-op job when paths don't match. Negligible vs dead-locked promotion chains.

Test plan

  • yaml syntax validates
  • After merge: a non-canvas, non-workspace-server push to staging should produce green runs for both gates (no-op pass), and auto-promote should fast-forward main
  • A canvas-touching push: e2e-staging-canvas runs the playwright job (real work)
  • A workspace-server-touching push: e2e-api runs the e2e-api job (real work)

Follow-ups (separate PRs)

  • e2e-staging-saas.yml has the same path-filter pattern but isn't currently in the auto-promote gate list — patch it preventively for consistency, or wait until it becomes a gate
  • auto-promote-on-e2e.yml for :staging-<sha>:latest retag (Design B unblocks this — the auto-promote chain becomes deterministic)

🤖 Generated with Claude Code

The auto-promote-staging.yml gate-check (line 99) treats "workflow
didn't run" as failure. Path-filtered triggers on E2E API Smoke Test
and E2E Staging Canvas meant a platform-only or test-only push to
staging — say, the prior PR #2201 which only touched
tests/e2e/test_staging_full_saas.sh — never triggered the canvas
workflow, and auto-promote saw `missing/none`, marked all_green=false,
and aborted. Same class for any push that doesn't touch the gate's
watched paths. Dead-lock by design, never noticed because the gate
was new.

Fix per Design B (always-run + fast-skip):

- Drop `paths:` from the push/pull_request triggers on both gate
  workflows. The workflow now always fires on every staging+main
  push/PR.
- Add a `detect-changes` job using `dorny/paths-filter@v3` that
  decides whether to do real work, scoped to the same paths the
  trigger filter used to watch.
- Real work job (e2e-api / playwright) gates on
  `needs: detect-changes; if: needs.detect-changes.outputs.X == 'true'`.
- Add a sibling `no-op` job that runs when the filter output is
  false, emitting `::notice::… no-op pass`. The workflow run's
  conclusion is `success` either way — auto-promote sees green and
  proceeds.

manual `workflow_dispatch` and the weekly canvas `schedule` short-
circuit detect-changes to always-run — those triggers exist precisely
to exercise the suite and shouldn't be silently no-op'd.

Why this approach over making auto-promote-staging smarter:

The alternative (Design A, considered + rejected) was to teach
auto-promote-staging to read each gate's `paths:` filter and treat
"no run because filter excluded the commit" as conditional pass.
That couples auto-promote to other workflows' YAML schema and breaks
silently if a gate is renamed or its filter changes. Design B keeps
the auto-promote contract simple ("each gate emits success") and
makes each gate self-describing — adding a new gate doesn't require
touching auto-promote.

Cost: ~10-30s of runner overhead per gate per push for the no-op when
paths don't match. Negligible vs the alternative of dead-locked
auto-promote chains.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Merged via the queue into staging with commit 96955f7 Apr 28, 2026
21 checks passed
HongmingWang-Rabbit added a commit that referenced this pull request Apr 28, 2026
staging → main: pipeline self-healing fixes (#2203 + #2204) — final manual bridge
HongmingWang-Rabbit pushed a commit that referenced this pull request Jun 12, 2026
…class in staging SaaS canary' (#2203) from devops/saas-a2a-empty-completion-diagnostic into main
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant