Skip to content

perf(ci): shard release quality checks #10619

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yiliang114 merged 17 commits into from
Sep 1, 2026
Merged

perf(ci): shard release quality checks #10619

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
0934492
perf(ci): shard release quality checks
yiliang114 Aug 31, 2026
1182c78
fix(ci): include generated templates in release artifact
yiliang114 Aug 31, 2026
bb857c8
test(web-shell): wait for paginated loads in CI
yiliang114 Aug 31, 2026
3aa90fa
test(web-shell): allow slow boot imports in CI
yiliang114 Aug 31, 2026
1ca54b2
Merge branch 'main' into codex/optimize-release-quality-10422
yiliang114 Aug 31, 2026
7122a3a
test(release): ratchet test-file discovery for every test:ci workspace
yiliang114 Aug 31, 2026
f453f62
fix(release): keep cancelled runs out of the quality failure gate
yiliang114 Aug 31, 2026
e10628e
refactor(release): reuse release_test_env anchor in integration jobs
yiliang114 Aug 31, 2026
187422f
fix(release): prune node_modules when packing build outputs
yiliang114 Aug 31, 2026
7a1ab67
refactor(release-test): reuse getWorkspacePackageJsonPaths in shard test
yiliang114 Aug 31, 2026
751cc4e
refactor(release-test): extract shared test:ci workspace selection he…
yiliang114 Aug 31, 2026
0c08597
test(release): pin vitest as the last test:ci command in shard lanes
yiliang114 Aug 31, 2026
ae44527
test(release): pin needs edge, five-result mapping, and per-job skip …
yiliang114 Sep 1, 2026
114479b
fix(release): log packed outputs and fail closed on silent under-pack
yiliang114 Sep 1, 2026
0090c8d
fix(release): pin consumer artifact contract, dist origin, and retention
yiliang114 Sep 1, 2026
0a47496
Merge remote-tracking branch 'origin/main' into codex/optimize-releas…
yiliang114 Sep 1, 2026
c42ee9a
fix(release): skip quality aggregate when prepare never ran
yiliang114 Sep 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/.size-baseline
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@
22037 release-sdk-python.yml
19094 release-sdk.yml
14546 release-vscode-companion.yml
53022 release.yml
59855 release.yml
43717 repo-hygiene.yml
1079 scorecard-monthly.yml
10691 sdk-java.yml
Expand Down
275 changes: 187 additions & 88 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ jobs:
permissions:
contents: 'read'
outputs:
release_sha: '${{ steps.source.outputs.release_sha }}'
release_tag: '${{ steps.version.outputs.RELEASE_TAG }}'
release_version: '${{ steps.version.outputs.RELEASE_VERSION }}'
npm_tag: '${{ steps.version.outputs.NPM_TAG }}'
Expand All @@ -73,7 +74,7 @@ jobs:
# wipe — rather than keeping and scrubbing .git like serve-ab.yml —
# is deliberate: these checkouts run with CI_BOT_PAT and the npm
# OIDC id-token, so no pre-existing repo state may survive into
# them; the accepted cost is re-fetching full history each run.
# them; the accepted cost is fetching a fresh ref each run.
#
# Guards ported from serve-ab.yml's wipe (#9220, #9265): under a
# mangled env even `/home` or an empty string reached the rm. A
Expand Down Expand Up @@ -248,6 +249,10 @@ jobs:
fi
echo "is_dry_run=${is_dry_run}" >> "${GITHUB_OUTPUT}"

- name: 'Resolve release commit'
id: 'source'
run: 'echo "release_sha=$(git rev-parse HEAD)" >> "${GITHUB_OUTPUT}"'

- name: 'Set up Node.js (hosted)'
if: "${{ runner.environment != 'self-hosted' }}"
uses: 'actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e' # v6.4.0
Expand Down Expand Up @@ -312,16 +317,16 @@ jobs:
echo "NPM_TAG=$(echo "$VERSION_JSON" | jq -r .npmTag)" >> "$GITHUB_OUTPUT"
echo "PREVIOUS_RELEASE_TAG=$(echo "$VERSION_JSON" | jq -r .previousReleaseTag)" >> "$GITHUB_OUTPUT"

quality:
name: 'Quality Checks'
quality_static:
name: 'Quality Checks (Static)'
runs-on: '${{ (github.repository == ''QwenLM/qwen-code'' && vars.MAINTAINER_ECS_RUNNER_DISABLED != ''true'') && fromJSON(''["self-hosted", "linux", "x64", "ecs-qwen"]'') || fromJSON(''["ubuntu-latest"]'') }}'
timeout-minutes: 120
timeout-minutes: 30
needs: 'prepare'
if: |-
${{ github.event.inputs.force_skip_tests != 'true' }}
permissions:
contents: 'read'
env:
env: &release_test_env
Comment thread
yiliang114 marked this conversation as resolved.
Resolved
OPENAI_API_KEY: '${{ secrets.OPENAI_API_KEY }}'
OPENAI_BASE_URL: '${{ secrets.OPENAI_BASE_URL }}'
OPENAI_MODEL: '${{ secrets.OPENAI_MODEL }}'
Expand All @@ -331,13 +336,15 @@ jobs:
# containerized job. Restore the reusable workspace before checkout.
- *restore_release_workspace

- name: 'Checkout'
- &checkout_release_validation
name: 'Checkout'
uses: 'actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10' # v6.0.3
with:
ref: '${{ github.event.inputs.ref || github.sha }}'
fetch-depth: 0
ref: '${{ needs.prepare.outputs.release_sha }}'
fetch-depth: 1

- name: 'Set up Node.js (hosted)'
- &setup_release_node_hosted
name: 'Set up Node.js (hosted)'
if: "${{ runner.environment != 'self-hosted' }}"
uses: 'actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e' # v6.4.0
with:
Expand All @@ -349,11 +356,13 @@ jobs:
# Avoid setup-node downloads on ECS, where nodejs.org may be
# unreachable through the egress proxy; reuse the machine's Node
# instead.
- name: 'Use pre-installed Node.js (self-hosted)'
- &setup_release_node_self_hosted
name: 'Use pre-installed Node.js (self-hosted)'
if: "${{ runner.environment == 'self-hosted' }}"
uses: './.github/actions/self-hosted-node'

- name: 'Install Dependencies'
- &install_release_dependencies
name: 'Install Dependencies'
env:
NPM_CONFIG_PREFER_OFFLINE: 'true'
run: |-
Expand All @@ -372,6 +381,24 @@ jobs:
run: |-
npm run lint:ci

quality_build:
name: 'Quality Checks (Build)'
runs-on: '${{ (github.repository == ''QwenLM/qwen-code'' && vars.MAINTAINER_ECS_RUNNER_DISABLED != ''true'') && fromJSON(''["self-hosted", "linux", "x64", "ecs-qwen"]'') || fromJSON(''["ubuntu-latest"]'') }}'
timeout-minutes: 45
needs: 'prepare'
if: |-
${{ github.event.inputs.force_skip_tests != 'true' }}
permissions:
contents: 'read'
env: *release_test_env

steps:
- *restore_release_workspace
- *checkout_release_validation
- *setup_release_node_hosted
- *setup_release_node_self_hosted
- *install_release_dependencies

- name: 'Check Serve Fast Path Bundle'
run: |-
npm run check:serve-fast-path-bundle
Expand All @@ -380,13 +407,150 @@ jobs:
run: |-
npm run build

- name: 'Pack Build Outputs'
run: |-
build_paths=('dist' 'packages/web-templates/src/generated')
Comment thread
yiliang114 marked this conversation as resolved.
Resolved
Comment thread
yiliang114 marked this conversation as resolved.
Resolved
Comment thread
yiliang114 marked this conversation as resolved.
Resolved
while IFS= read -r -d '' path; do
build_paths+=("${path}")
done < <(find packages integrations -type d -name dist -prune -print0)
Comment thread
yiliang114 marked this conversation as resolved.
Resolved
Comment thread
yiliang114 marked this conversation as resolved.
Resolved
tar -czf "${RUNNER_TEMP}/release-build.tgz" "${build_paths[@]}"

- name: 'Upload Build Outputs'
uses: 'actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a' # v7.0.1
with:
name: 'release-quality-build'
path: '${{ runner.temp }}/release-build.tgz'
retention-days: 1
Comment thread
yiliang114 marked this conversation as resolved.
Resolved
Comment thread
yiliang114 marked this conversation as resolved.
Resolved
Comment thread
yiliang114 marked this conversation as resolved.
Resolved
if-no-files-found: 'error'
overwrite: true

quality_typecheck:
name: 'Quality Checks (Typecheck)'
runs-on: '${{ (github.repository == ''QwenLM/qwen-code'' && vars.MAINTAINER_ECS_RUNNER_DISABLED != ''true'') && fromJSON(''["self-hosted", "linux", "x64", "ecs-qwen"]'') || fromJSON(''["ubuntu-latest"]'') }}'
timeout-minutes: 30
needs:
- 'prepare'
- 'quality_build'
if: |-
${{ github.event.inputs.force_skip_tests != 'true' }}
permissions:
contents: 'read'
env: *release_test_env

steps:
- *restore_release_workspace
- *checkout_release_validation
- *setup_release_node_hosted
- *setup_release_node_self_hosted
- *install_release_dependencies

- &download_release_build
name: 'Download Build Outputs'
uses: 'actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c' # v8.0.1
with:
name: 'release-quality-build'
path: '${{ runner.temp }}/release-quality-build'

- &unpack_release_build
name: 'Unpack Build Outputs'
run: 'tar -xzf "${RUNNER_TEMP}/release-quality-build/release-build.tgz"'

- name: 'Typecheck Project'
run: |-
npm run typecheck

workspace_tests:
name: 'Workspace Tests (${{ matrix.shard }}/3)'
Comment thread
yiliang114 marked this conversation as resolved.
Resolved
runs-on: '${{ (github.repository == ''QwenLM/qwen-code'' && vars.MAINTAINER_ECS_RUNNER_DISABLED != ''true'') && fromJSON(''["self-hosted", "linux", "x64", "ecs-qwen"]'') || fromJSON(''["ubuntu-latest"]'') }}'
timeout-minutes: 45
needs:
- 'prepare'
- 'quality_build'
if: |-
${{ github.event.inputs.force_skip_tests != 'true' }}
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3]
permissions:
contents: 'read'
env: *release_test_env

steps:
- *restore_release_workspace
- *checkout_release_validation
- *setup_release_node_hosted
- *setup_release_node_self_hosted
- *install_release_dependencies
- *download_release_build
- *unpack_release_build

- name: 'Run Workspace Tests'
run: 'npm run test:release:workspaces -- --shard=${{ matrix.shard }}/3 --passWithNoTests'
Comment thread
yiliang114 marked this conversation as resolved.
Resolved

quality_scripts:
name: 'Quality Checks (Scripts)'
runs-on: '${{ (github.repository == ''QwenLM/qwen-code'' && vars.MAINTAINER_ECS_RUNNER_DISABLED != ''true'') && fromJSON(''["self-hosted", "linux", "x64", "ecs-qwen"]'') || fromJSON(''["ubuntu-latest"]'') }}'
timeout-minutes: 30
needs:
- 'prepare'
- 'quality_build'
if: |-
${{ github.event.inputs.force_skip_tests != 'true' }}
permissions:
contents: 'read'
env: *release_test_env

steps:
- *restore_release_workspace
- *checkout_release_validation
- *setup_release_node_hosted
- *setup_release_node_self_hosted
- *install_release_dependencies
- *download_release_build
- *unpack_release_build

- name: 'Run Script Tests'
run: 'npm run test:scripts'

quality:
name: 'Quality Checks'
runs-on: 'ubuntu-latest'
timeout-minutes: 5
needs:
- 'quality_static'
- 'quality_build'
- 'quality_typecheck'
- 'workspace_tests'
- 'quality_scripts'
# !cancelled() rather than always(): a cancelled run must leave this
# aggregate skipped, not turn component 'cancelled' results into a
# quality 'failure' that opens notify_failure's "Release Failed" issue
# for a run the operator stopped on purpose. Failed components still
# run this job and fail it closed.
if: |-
${{ !cancelled() && github.event.inputs.force_skip_tests != 'true' }}
Comment thread
yiliang114 marked this conversation as resolved.
Resolved
permissions: {}
steps:
- name: 'Verify quality results'
env:
STATIC_RESULT: '${{ needs.quality_static.result }}'
Comment thread
yiliang114 marked this conversation as resolved.
Resolved
Comment thread
yiliang114 marked this conversation as resolved.
Resolved
BUILD_RESULT: '${{ needs.quality_build.result }}'
TYPECHECK_RESULT: '${{ needs.quality_typecheck.result }}'
WORKSPACE_TEST_RESULT: '${{ needs.workspace_tests.result }}'
SCRIPT_TEST_RESULT: '${{ needs.quality_scripts.result }}'
run: |-
npm run test:release
for result in \
"${STATIC_RESULT}" \
"${BUILD_RESULT}" \
"${TYPECHECK_RESULT}" \
"${WORKSPACE_TEST_RESULT}" \
"${SCRIPT_TEST_RESULT}"; do
if [[ "${result}" != 'success' ]]; then
echo "::error::A release quality check did not succeed."
exit 1
fi
done

integration_none:
name: 'Integration Tests (No Sandbox)'
Expand All @@ -397,48 +561,14 @@ jobs:
${{ github.event.inputs.force_skip_tests != 'true' }}
permissions:
contents: 'read'
env:
OPENAI_API_KEY: '${{ secrets.OPENAI_API_KEY }}'
OPENAI_BASE_URL: '${{ secrets.OPENAI_BASE_URL }}'
OPENAI_MODEL: '${{ secrets.OPENAI_MODEL }}'
env: *release_test_env

steps:
# Shared ECS runners can retain root-owned files from an earlier
# containerized job. Restore the reusable workspace before checkout.
- *restore_release_workspace

- name: 'Checkout'
uses: 'actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10' # v6.0.3
with:
ref: '${{ github.event.inputs.ref || github.sha }}'
fetch-depth: 0

- name: 'Set up Node.js (hosted)'
if: "${{ runner.environment != 'self-hosted' }}"
uses: 'actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e' # v6.4.0
with:
node-version-file: '.nvmrc'
cache: 'npm'
package-manager-cache: false
cache-dependency-path: 'package-lock.json'

# Avoid setup-node downloads on ECS, where nodejs.org may be
# unreachable through the egress proxy; reuse the machine's Node
# instead.
- name: 'Use pre-installed Node.js (self-hosted)'
if: "${{ runner.environment == 'self-hosted' }}"
uses: './.github/actions/self-hosted-node'

- name: 'Install Dependencies'
env:
NPM_CONFIG_PREFER_OFFLINE: 'true'
run: |-
npm ci --ignore-scripts --no-audit --progress=false
# Replay the root postinstall (patch-package) and commit-info
# generation; dependency and workspace lifecycle scripts remain
# disabled.
npm run postinstall
npm run generate
- *checkout_release_validation
- *setup_release_node_hosted
- *setup_release_node_self_hosted
- *install_release_dependencies

- name: 'Build Bundle'
run: |-
Expand Down Expand Up @@ -466,10 +596,7 @@ jobs:
${{ github.event.inputs.force_skip_tests != 'true' }}
permissions:
contents: 'read'
env:
OPENAI_API_KEY: '${{ secrets.OPENAI_API_KEY }}'
OPENAI_BASE_URL: '${{ secrets.OPENAI_BASE_URL }}'
OPENAI_MODEL: '${{ secrets.OPENAI_MODEL }}'
env: *release_test_env

steps:
# Shared ECS runners can retain root-owned files from an earlier
Expand All @@ -484,38 +611,10 @@ jobs:
exit 1
fi

- name: 'Checkout'
uses: 'actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10' # v6.0.3
with:
ref: '${{ github.event.inputs.ref || github.sha }}'
fetch-depth: 0

- name: 'Set up Node.js (hosted)'
if: "${{ runner.environment != 'self-hosted' }}"
uses: 'actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e' # v6.4.0
with:
node-version-file: '.nvmrc'
cache: 'npm'
package-manager-cache: false
cache-dependency-path: 'package-lock.json'

# Avoid setup-node downloads on ECS, where nodejs.org may be
# unreachable through the egress proxy; reuse the machine's Node
# instead.
- name: 'Use pre-installed Node.js (self-hosted)'
if: "${{ runner.environment == 'self-hosted' }}"
uses: './.github/actions/self-hosted-node'

- name: 'Install Dependencies'
env:
NPM_CONFIG_PREFER_OFFLINE: 'true'
run: |-
npm ci --ignore-scripts --no-audit --progress=false
# Replay the root postinstall (patch-package) and commit-info
# generation; dependency and workspace lifecycle scripts remain
# disabled.
npm run postinstall
npm run generate
- *checkout_release_validation
- *setup_release_node_hosted
- *setup_release_node_self_hosted
- *install_release_dependencies

- name: 'Build Bundle'
run: |-
Expand Down Expand Up @@ -657,7 +756,7 @@ jobs:
uses: 'actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10' # v6.0.3
with:
token: '${{ secrets.CI_BOT_PAT }}'
ref: '${{ github.event.inputs.ref || github.sha }}'
ref: '${{ needs.prepare.outputs.release_sha }}'
fetch-depth: 0
persist-credentials: false

Expand Down
Loading
Loading