Skip to content

Refactor Upgrade downgrade tests #14782

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
GuptaManan100 merged 9 commits into from
Dec 20, 2023
Merged

Refactor Upgrade downgrade tests #14782

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
a552531
test: add all subpackages in vtgate/queries to upgrade downgrade tests
GuptaManan100 Dec 14, 2023
bc7636e
test: add a test for time function calls with precision
GuptaManan100 Dec 14, 2023
a10d51a
test: rename upgrade downgrade tests to have upgrade downgrade as a s…
GuptaManan100 Dec 14, 2023
41ac243
refactor backups e2e test to only use a single job
GuptaManan100 Dec 14, 2023
17ade29
refactor other upgrade downgrade tests to only use a single job
GuptaManan100 Dec 14, 2023
26731bd
minor fix to workflow file
GuptaManan100 Dec 14, 2023
320295e
test: fix upgrade downgrade tests
GuptaManan100 Dec 15, 2023
b9e48a5
Merge remote-tracking branch 'upstream/main' into enable-upgrade-down…
GuptaManan100 Dec 15, 2023
be71a16
Merge remote-tracking branch 'upstream/main' into enable-upgrade-down…
GuptaManan100 Dec 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 13 additions & 26 deletions .github/workflows/upgrade_downgrade_test_backups_e2e.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Upgrade Downgrade Testing - Backups - E2E
name: Backups - E2E - Upgrade Downgrade Testing
on:
push:
pull_request:
Expand All @@ -10,33 +10,10 @@ concurrency:
permissions: read-all

jobs:
get_previous_release:
if: always()
name: Get Previous Release - Backups - E2E
runs-on: gh-hosted-runners-16cores-1
outputs:
previous_release: ${{ steps.output-previous-release-ref.outputs.previous_release_ref }}

steps:
- name: Check out to HEAD
uses: actions/checkout@v3
with:
fetch-depth: 0

- name: Set output with latest release branch
id: output-previous-release-ref
run: |
previous_release_ref=$(./tools/get_previous_release.sh ${{github.base_ref}} ${{github.ref}})
echo $previous_release_ref
echo "previous_release_ref=${previous_release_ref}" >> $GITHUB_OUTPUT

upgrade_downgrade_test_e2e:
timeout-minutes: 60
if: always() && needs.get_previous_release.result == 'success'
name: Run Upgrade Downgrade Test - Backups - E2E
runs-on: gh-hosted-runners-16cores-1
needs:
- get_previous_release

steps:
- name: Skip CI
Expand All @@ -59,6 +36,16 @@ jobs:
- name: Check out commit's code
if: steps.skip-workflow.outputs.skip-workflow == 'false'
uses: actions/checkout@v3
with:
fetch-depth: 0

- name: Set output with latest release branch
if: steps.skip-workflow.outputs.skip-workflow == 'false'
id: output-previous-release-ref
run: |
previous_release_ref=$(./tools/get_previous_release.sh ${{github.base_ref}} ${{github.ref}})
echo $previous_release_ref
echo "previous_release_ref=${previous_release_ref}" >> $GITHUB_OUTPUT

- name: Check for changes in relevant files
if: steps.skip-workflow.outputs.skip-workflow == 'false'
Expand Down Expand Up @@ -117,11 +104,11 @@ jobs:
sudo apt-get install -y percona-xtrabackup-24

# Checkout to the last release of Vitess
- name: Check out other version's code (${{ needs.get_previous_release.outputs.previous_release }})
- name: Check out other version's code (${{ steps.output-previous-release-ref.outputs.previous_release_ref }})
if: steps.skip-workflow.outputs.skip-workflow == 'false' && steps.changes.outputs.end_to_end == 'true'
uses: actions/checkout@v3
with:
ref: ${{ needs.get_previous_release.outputs.previous_release }}
ref: ${{ steps.output-previous-release-ref.outputs.previous_release_ref }}

- name: Get dependencies for the last release
if: steps.skip-workflow.outputs.skip-workflow == 'false' && steps.changes.outputs.end_to_end == 'true'
Expand Down
46 changes: 16 additions & 30 deletions .github/workflows/upgrade_downgrade_test_backups_e2e_next_release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Upgrade Downgrade Testing - Backups - E2E - Next Release
name: Backups - E2E - Next Release - Upgrade Downgrade Testing
on:
push:
pull_request:
Expand All @@ -10,33 +10,11 @@ concurrency:
permissions: read-all

jobs:
get_next_release:
if: always()
name: Get Latest Release - Backups - E2E - Next Release
runs-on: gh-hosted-runners-16cores-1
outputs:
next_release: ${{ steps.output-next-release-ref.outputs.next_release_ref }}

steps:
- name: Check out to HEAD
uses: actions/checkout@v3
with:
fetch-depth: 0

- name: Set output with latest release branch
id: output-next-release-ref
run: |
next_release_ref=$(./tools/get_next_release.sh ${{github.base_ref}} ${{github.ref}})
echo $next_release_ref
echo "next_release_ref=${next_release_ref}" >> $GITHUB_OUTPUT

upgrade_downgrade_test_e2e:
timeout-minutes: 60
if: always() && needs.get_next_release.result == 'success'
name: Run Upgrade Downgrade Test - Backups - E2E - Next Release
runs-on: gh-hosted-runners-16cores-1
needs:
- get_next_release

steps:
- name: Skip CI
Expand All @@ -46,23 +24,31 @@ jobs:
exit 1
fi

- name: Check out commit's code
uses: actions/checkout@v3
with:
fetch-depth: 0

- name: Set output with latest release branch
id: output-next-release-ref
run: |
next_release_ref=$(./tools/get_next_release.sh ${{github.base_ref}} ${{github.ref}})
echo $next_release_ref
echo "next_release_ref=${next_release_ref}" >> $GITHUB_OUTPUT

- name: Check if workflow needs to be skipped
id: skip-workflow
run: |
skip='false'
if [[ "${{github.event.pull_request}}" == "" ]] && [[ "${{github.ref}}" != "refs/heads/main" ]] && [[ ! "${{github.ref}}" =~ ^refs/heads/release-[0-9]+\.[0-9]$ ]] && [[ ! "${{github.ref}}" =~ "refs/tags/.*" ]]; then
skip='true'
fi
if [[ "${{needs.get_next_release.outputs.next_release}}" == "" ]]; then
if [[ "${{steps.output-next-release-ref.outputs.next_release_ref}}" == "" ]]; then
skip='true'
fi
echo Skip ${skip}
echo "skip-workflow=${skip}" >> $GITHUB_OUTPUT

- name: Check out commit's code
if: steps.skip-workflow.outputs.skip-workflow == 'false'
uses: actions/checkout@v3

- name: Check for changes in relevant files
if: steps.skip-workflow.outputs.skip-workflow == 'false'
uses: frouioui/paths-filter@main
Expand Down Expand Up @@ -120,11 +106,11 @@ jobs:
sudo apt-get install -y percona-xtrabackup-24

# Checkout to the next release of Vitess
- name: Check out other version's code (${{ needs.get_next_release.outputs.next_release }})
- name: Check out other version's code (${{ steps.output-next-release-ref.outputs.next_release_ref }})
if: steps.skip-workflow.outputs.skip-workflow == 'false' && steps.changes.outputs.end_to_end == 'true'
uses: actions/checkout@v3
with:
ref: ${{ needs.get_next_release.outputs.next_release }}
ref: ${{ steps.output-next-release-ref.outputs.next_release_ref }}

- name: Get dependencies for the next release
if: steps.skip-workflow.outputs.skip-workflow == 'false' && steps.changes.outputs.end_to_end == 'true'
Expand Down
38 changes: 13 additions & 25 deletions .github/workflows/upgrade_downgrade_test_backups_manual.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Upgrade Downgrade Testing - Backups - Manual
name: Backups - Manual - Upgrade Downgrade Testing
on:
push:
pull_request:
Expand All @@ -10,34 +10,12 @@ concurrency:
permissions: read-all

jobs:
get_previous_release:
if: always()
name: Get Previous Release - Backups - Manual
runs-on: gh-hosted-runners-16cores-1
outputs:
previous_release: ${{ steps.output-previous-release-ref.outputs.previous_release_ref }}

steps:
- name: Check out to HEAD
uses: actions/checkout@v3
with:
fetch-depth: 0

- name: Set output with latest release branch
id: output-previous-release-ref
run: |
previous_release_ref=$(./tools/get_previous_release.sh ${{github.base_ref}} ${{github.ref}})
echo $previous_release_ref
echo "previous_release_ref=${previous_release_ref}" >> $GITHUB_OUTPUT

# This job usually execute in ± 20 minutes
upgrade_downgrade_test_manual:
timeout-minutes: 40
if: always() && (needs.get_previous_release.result == 'success')
name: Run Upgrade Downgrade Test - Backups - Manual
runs-on: gh-hosted-runners-16cores-1
needs:
- get_previous_release

steps:
- name: Skip CI
Expand All @@ -61,6 +39,16 @@ jobs:
- name: Checkout to commit's code
if: steps.skip-workflow.outputs.skip-workflow == 'false'
uses: actions/checkout@v3
with:
fetch-depth: 0

- name: Set output with latest release branch
id: output-previous-release-ref
if: steps.skip-workflow.outputs.skip-workflow == 'false'
run: |
previous_release_ref=$(./tools/get_previous_release.sh ${{github.base_ref}} ${{github.ref}})
echo $previous_release_ref
echo "previous_release_ref=${previous_release_ref}" >> $GITHUB_OUTPUT

- name: Check for changes in relevant files
if: steps.skip-workflow.outputs.skip-workflow == 'false'
Expand Down Expand Up @@ -138,11 +126,11 @@ jobs:
sudo apt-get install -y percona-xtrabackup-24

# Checkout to the last release of Vitess
- name: Checkout to the other version's code (${{ needs.get_previous_release.outputs.previous_release }})
- name: Checkout to the other version's code (${{ steps.output-previous-release-ref.outputs.previous_release_ref }})
if: steps.skip-workflow.outputs.skip-workflow == 'false' && steps.changes.outputs.end_to_end == 'true'
uses: actions/checkout@v3
with:
ref: ${{ needs.get_previous_release.outputs.previous_release }}
ref: ${{ steps.output-previous-release-ref.outputs.previous_release_ref }}

- name: Get dependencies for the last release
if: steps.skip-workflow.outputs.skip-workflow == 'false' && steps.changes.outputs.end_to_end == 'true'
Expand Down
48 changes: 17 additions & 31 deletions .github/workflows/upgrade_downgrade_test_backups_manual_next_release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Upgrade Downgrade Testing - Backups - Manual - Next Release
name: Backups - Manual - Next Release - Upgrade Downgrade Testing
on:
push:
pull_request:
Expand All @@ -10,34 +10,12 @@ concurrency:
permissions: read-all

jobs:
get_next_release:
if: always()
name: Get Previous Release - Backups - Manual - Next Release
runs-on: gh-hosted-runners-16cores-1
outputs:
next_release: ${{ steps.output-next-release-ref.outputs.next_release_ref }}

steps:
- name: Check out to HEAD
uses: actions/checkout@v3
with:
fetch-depth: 0

- name: Set output with latest release branch
id: output-next-release-ref
run: |
next_release_ref=$(./tools/get_next_release.sh ${{github.base_ref}} ${{github.ref}})
echo $next_release_ref
echo "next_release_ref=${next_release_ref}" >> $GITHUB_OUTPUT

# This job usually execute in ± 20 minutes
upgrade_downgrade_test_manual:
timeout-minutes: 40
if: always() && (needs.get_next_release.result == 'success')
name: Run Upgrade Downgrade Test - Backups - Manual - Next Release
runs-on: gh-hosted-runners-16cores-1
needs:
- get_next_release

steps:
- name: Skip CI
Expand All @@ -47,24 +25,32 @@ jobs:
exit 1
fi

# Checkout to this build's commit
- name: Checkout to commit's code
uses: actions/checkout@v3
with:
fetch-depth: 0

- name: Set output with latest release branch
id: output-next-release-ref
run: |
next_release_ref=$(./tools/get_next_release.sh ${{github.base_ref}} ${{github.ref}})
echo $next_release_ref
echo "next_release_ref=${next_release_ref}" >> $GITHUB_OUTPUT

- name: Check if workflow needs to be skipped
id: skip-workflow
run: |
skip='false'
if [[ "${{github.event.pull_request}}" == "" ]] && [[ "${{github.ref}}" != "refs/heads/main" ]] && [[ ! "${{github.ref}}" =~ ^refs/heads/release-[0-9]+\.[0-9]$ ]] && [[ ! "${{github.ref}}" =~ "refs/tags/.*" ]]; then
skip='true'
fi
if [[ "${{needs.get_next_release.outputs.next_release}}" == "" ]]; then
if [[ "${{steps.output-next-release-ref.outputs.next_release_ref}}" == "" ]]; then
skip='true'
fi
echo Skip ${skip}
echo "skip-workflow=${skip}" >> $GITHUB_OUTPUT

# Checkout to this build's commit
- name: Checkout to commit's code
if: steps.skip-workflow.outputs.skip-workflow == 'false'
uses: actions/checkout@v3

- name: Check for changes in relevant files
if: steps.skip-workflow.outputs.skip-workflow == 'false'
uses: frouioui/paths-filter@main
Expand Down Expand Up @@ -141,11 +127,11 @@ jobs:
sudo apt-get install -y percona-xtrabackup-24

# Checkout to the next release of Vitess
- name: Checkout to the other version's code (${{ needs.get_next_release.outputs.next_release }})
- name: Checkout to the other version's code (${{ steps.output-next-release-ref.outputs.next_release_ref }})
if: steps.skip-workflow.outputs.skip-workflow == 'false' && steps.changes.outputs.end_to_end == 'true'
uses: actions/checkout@v3
with:
ref: ${{ needs.get_next_release.outputs.next_release }}
ref: ${{ steps.output-next-release-ref.outputs.next_release_ref }}

- name: Get dependencies for the next release
if: steps.skip-workflow.outputs.skip-workflow == 'false' && steps.changes.outputs.end_to_end == 'true'
Expand Down
38 changes: 13 additions & 25 deletions .github/workflows/upgrade_downgrade_test_query_serving_queries.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Upgrade Downgrade Testing Query Serving (Queries)
name: Query Serving (Queries) - Upgrade Downgrade Testing
on:
push:
pull_request:
Expand All @@ -13,32 +13,10 @@ permissions: read-all
# (vtgate, vttablet, etc) built on different versions.

jobs:
get_previous_release:
if: always()
name: Get Previous Release - Query Serving (Queries)
runs-on: gh-hosted-runners-16cores-1
outputs:
previous_release: ${{ steps.output-previous-release-ref.outputs.previous_release_ref }}

steps:
- name: Check out to HEAD
uses: actions/checkout@v3
with:
fetch-depth: 0

- name: Set output with latest release branch
id: output-previous-release-ref
run: |
previous_release_ref=$(./tools/get_previous_release.sh ${{github.base_ref}} ${{github.ref}})
echo $previous_release_ref
echo "previous_release_ref=${previous_release_ref}" >> $GITHUB_OUTPUT

upgrade_downgrade_test:
if: always() && (needs.get_previous_release.result == 'success')
name: Run Upgrade Downgrade Test - Query Serving (Queries)
runs-on: gh-hosted-runners-16cores-1
needs:
- get_previous_release

steps:
- name: Skip CI
Expand All @@ -61,6 +39,16 @@ jobs:
- name: Check out commit's code
if: steps.skip-workflow.outputs.skip-workflow == 'false'
uses: actions/checkout@v3
with:
fetch-depth: 0

- name: Set output with latest release branch
id: output-previous-release-ref
if: steps.skip-workflow.outputs.skip-workflow == 'false'
run: |
previous_release_ref=$(./tools/get_previous_release.sh ${{github.base_ref}} ${{github.ref}})
echo $previous_release_ref
echo "previous_release_ref=${previous_release_ref}" >> $GITHUB_OUTPUT

- name: Check for changes in relevant files
if: steps.skip-workflow.outputs.skip-workflow == 'false'
Expand Down Expand Up @@ -135,11 +123,11 @@ jobs:
sudo apt-get install -y percona-xtrabackup-24

# Checkout to the last release of Vitess
- name: Check out other version's code (${{ needs.get_previous_release.outputs.previous_release }})
- name: Check out other version's code (${{ steps.output-previous-release-ref.outputs.previous_release_ref }})
if: steps.skip-workflow.outputs.skip-workflow == 'false' && steps.changes.outputs.end_to_end == 'true'
uses: actions/checkout@v3
with:
ref: ${{ needs.get_previous_release.outputs.previous_release }}
ref: ${{ steps.output-previous-release-ref.outputs.previous_release_ref }}

- name: Get dependencies for the last release
if: steps.skip-workflow.outputs.skip-workflow == 'false' && steps.changes.outputs.end_to_end == 'true'
Expand Down
Loading