Skip to content

[CI] Fix random job cancellations in combined publish workflows#11703

Merged
BDevParity merged 10 commits intomasterfrom
bd-fix-combined-publish-sync-check-cancellation
Apr 15, 2026
Merged

[CI] Fix random job cancellations in combined publish workflows#11703
BDevParity merged 10 commits intomasterfrom
bd-fix-combined-publish-sync-check-cancellation

Conversation

@BDevParity
Copy link
Copy Markdown
Contributor

@BDevParity BDevParity commented Apr 9, 2026

Summary

  • The combined publish workflows (release-70, release-22) call multiple sub-workflows that each invoke check-synchronization.yml. That reusable workflow has a concurrency group (${{ github.repository }}-${{ github.workflow }}-${{ github.ref }}) that evaluates to the same value for all callers within a single run, causing GitHub to randomly cancel competing jobs.
  • PR [Release|CI/CD] Trigger check-sync flow in callable flows only if they are triggered manually and not with the workflow_call trigger #11631 attempted to fix this with github.event_name == 'workflow_dispatch' guards, but github.event_name propagates the original trigger through workflow_call chains, so the guard doesn't reliably skip the check.
  • This PR replaces that guard with an explicit skip_sync_check boolean input. Combined workflows pass skip_sync_check: true since they already run the check at the top level. Standalone dispatches default to false, preserving existing behavior.
  • Also removes the redundant check-synchronization job from release-reusable-publish-packages.yml (only ever called from workflows that already perform the check).

Evidence: 4 consecutive failed attempts with different random jobs cancelled each time.

Issue

Replace unreliable `github.event_name == 'workflow_dispatch'` guard
with explicit `skip_sync_check` input to prevent redundant
check-synchronization calls from colliding on the same concurrency
group when called from combined workflows.
@BDevParity BDevParity requested review from a team as code owners April 9, 2026 12:55
@BDevParity BDevParity added the R0-no-crate-publish-required The change does not require any crates to be re-published. label Apr 9, 2026
@paritytech-review-bot paritytech-review-bot Bot requested a review from a team April 9, 2026 13:19
Copy link
Copy Markdown
Contributor

@EgorPopelyaev EgorPopelyaev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to find another way to skip this check, otherwise, the jobs can be triggered separatelly without it

… release safeguards

Even when skip_sync_check skips the full check-synchronization workflow,
downstream jobs now independently verify repository_owner == RELEASE_ORG
and RELEASES_ON == true before proceeding.
Each caller now passes a unique concurrency_suffix so parallel
check-synchronization invocations within the same workflow run
get distinct concurrency groups and don't cancel each other.

Also restores check-synchronization in release-reusable-publish-packages
with the concurrency suffix and inline org/killswitch guard.

Depends on: paritytech-release/sync-workflows#12
@EgorPopelyaev
Copy link
Copy Markdown
Contributor

LGTM, but let's test it before merge

@BDevParity BDevParity added A4-backport-stable2506 Pull request must be backported to the stable2506 release branch A4-backport-stable2509 Pull request must be backported to the stable2509 release branch A4-backport-stable2512 Pull request must be backported to the stable2512 release branch A4-backport-stable2603 Pull request must be backported to the stable2603 release branch labels Apr 14, 2026
EgorPopelyaev and others added 3 commits April 15, 2026 13:00
…sions

Skipping check-synchronization via skip_sync_check caused GitHub to
propagate the skipped-ancestor status through the dependency chain,
making all downstream build jobs skip silently.

With unique concurrency_suffix values on every check-synchronization
call, parallel invocations no longer collide. No need to skip anything.
@github-actions
Copy link
Copy Markdown
Contributor

Review required! Latest push from author must always be reviewed

@BDevParity BDevParity enabled auto-merge April 15, 2026 17:00
@BDevParity BDevParity added this pull request to the merge queue Apr 15, 2026
Merged via the queue into master with commit 9f6182f Apr 15, 2026
234 of 236 checks passed
@BDevParity BDevParity deleted the bd-fix-combined-publish-sync-check-cancellation branch April 15, 2026 19:14
paritytech-release-backport-bot Bot pushed a commit that referenced this pull request Apr 15, 2026
## Summary

- The combined publish workflows (`release-70`, `release-22`) call
multiple sub-workflows that each invoke `check-synchronization.yml`.
That reusable workflow has a concurrency group (`${{ github.repository
}}-${{ github.workflow }}-${{ github.ref }}`) that evaluates to the
**same value** for all callers within a single run, causing GitHub to
randomly cancel competing jobs.
- PR #11631 attempted to fix this with `github.event_name ==
'workflow_dispatch'` guards, but `github.event_name` propagates the
original trigger through `workflow_call` chains, so the guard doesn't
reliably skip the check.
- This PR replaces that guard with an explicit `skip_sync_check` boolean
input. Combined workflows pass `skip_sync_check: true` since they
already run the check at the top level. Standalone dispatches default to
`false`, preserving existing behavior.
- Also removes the redundant `check-synchronization` job from
`release-reusable-publish-packages.yml` (only ever called from workflows
that already perform the check).

Evidence: [4 consecutive failed
attempts](https://github.com/paritytech-release/polkadot-sdk/actions/runs/23787944175)
with different random jobs cancelled each time.

## Issue

- paritytech/release-engineering#291

---------

Co-authored-by: Egor_P <egor@parity.io>
(cherry picked from commit 9f6182f)
@paritytech-release-backport-bot
Copy link
Copy Markdown

Successfully created backport PR for stable2506:

@paritytech-release-backport-bot
Copy link
Copy Markdown

Created backport PR for stable2509:

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin backport-11703-to-stable2509
git worktree add --checkout .worktree/backport-11703-to-stable2509 backport-11703-to-stable2509
cd .worktree/backport-11703-to-stable2509
git reset --hard HEAD^
git cherry-pick -x 9f6182fe1ef2a4bd2013571396999ddd4044a8ef
git push --force-with-lease

@paritytech-release-backport-bot
Copy link
Copy Markdown

Successfully created backport PR for stable2512:

paritytech-release-backport-bot Bot pushed a commit that referenced this pull request Apr 15, 2026
## Summary

- The combined publish workflows (`release-70`, `release-22`) call
multiple sub-workflows that each invoke `check-synchronization.yml`.
That reusable workflow has a concurrency group (`${{ github.repository
}}-${{ github.workflow }}-${{ github.ref }}`) that evaluates to the
**same value** for all callers within a single run, causing GitHub to
randomly cancel competing jobs.
- PR #11631 attempted to fix this with `github.event_name ==
'workflow_dispatch'` guards, but `github.event_name` propagates the
original trigger through `workflow_call` chains, so the guard doesn't
reliably skip the check.
- This PR replaces that guard with an explicit `skip_sync_check` boolean
input. Combined workflows pass `skip_sync_check: true` since they
already run the check at the top level. Standalone dispatches default to
`false`, preserving existing behavior.
- Also removes the redundant `check-synchronization` job from
`release-reusable-publish-packages.yml` (only ever called from workflows
that already perform the check).

Evidence: [4 consecutive failed
attempts](https://github.com/paritytech-release/polkadot-sdk/actions/runs/23787944175)
with different random jobs cancelled each time.

## Issue

- paritytech/release-engineering#291

---------

Co-authored-by: Egor_P <egor@parity.io>
(cherry picked from commit 9f6182f)
paritytech-release-backport-bot Bot pushed a commit that referenced this pull request Apr 15, 2026
## Summary

- The combined publish workflows (`release-70`, `release-22`) call
multiple sub-workflows that each invoke `check-synchronization.yml`.
That reusable workflow has a concurrency group (`${{ github.repository
}}-${{ github.workflow }}-${{ github.ref }}`) that evaluates to the
**same value** for all callers within a single run, causing GitHub to
randomly cancel competing jobs.
- PR #11631 attempted to fix this with `github.event_name ==
'workflow_dispatch'` guards, but `github.event_name` propagates the
original trigger through `workflow_call` chains, so the guard doesn't
reliably skip the check.
- This PR replaces that guard with an explicit `skip_sync_check` boolean
input. Combined workflows pass `skip_sync_check: true` since they
already run the check at the top level. Standalone dispatches default to
`false`, preserving existing behavior.
- Also removes the redundant `check-synchronization` job from
`release-reusable-publish-packages.yml` (only ever called from workflows
that already perform the check).

Evidence: [4 consecutive failed
attempts](https://github.com/paritytech-release/polkadot-sdk/actions/runs/23787944175)
with different random jobs cancelled each time.

## Issue

- paritytech/release-engineering#291

---------

Co-authored-by: Egor_P <egor@parity.io>
(cherry picked from commit 9f6182f)
@paritytech-release-backport-bot
Copy link
Copy Markdown

Successfully created backport PR for stable2603:

EgorPopelyaev added a commit that referenced this pull request Apr 16, 2026
Backport #11703 into `stable2506` from BDevParity.

See the
[documentation](https://github.com/paritytech/polkadot-sdk/blob/master/docs/BACKPORT.md)
on how to use this bot.

<!--
  # To be used by other automation, do not modify:
  original-pr-number: #${pull_number}
-->

Co-authored-by: BDevParity <bruno.devic@parity.io>
Co-authored-by: Egor_P <egor@parity.io>
EgorPopelyaev pushed a commit that referenced this pull request Apr 16, 2026
Backport #11703 into `stable2509` from BDevParity.

See the
[documentation](https://github.com/paritytech/polkadot-sdk/blob/master/docs/BACKPORT.md)
on how to use this bot.

<!--
  # To be used by other automation, do not modify:
  original-pr-number: #${pull_number}
-->

Co-authored-by: BDevParity <bruno.devic@parity.io>
EgorPopelyaev added a commit that referenced this pull request Apr 16, 2026
Backport #11703 into `stable2512` from BDevParity.

See the
[documentation](https://github.com/paritytech/polkadot-sdk/blob/master/docs/BACKPORT.md)
on how to use this bot.

<!--
  # To be used by other automation, do not modify:
  original-pr-number: #${pull_number}
-->

Co-authored-by: BDevParity <bruno.devic@parity.io>
Co-authored-by: Egor_P <egor@parity.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A4-backport-stable2506 Pull request must be backported to the stable2506 release branch A4-backport-stable2509 Pull request must be backported to the stable2509 release branch A4-backport-stable2512 Pull request must be backported to the stable2512 release branch A4-backport-stable2603 Pull request must be backported to the stable2603 release branch R0-no-crate-publish-required The change does not require any crates to be re-published.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants