Skip to content

ci: Update screenshots with CI pipeline #38243

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
sagar-qa007 wants to merge 4 commits into from
Closed

ci: Update screenshots with CI pipeline #38243

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
fb551da
First cut for record
sagar-qa007 Dec 18, 2024
3e1b532
Update server logs path for better logging
sagar-qa007 Dec 19, 2024
8319e66
Merge branch 'release' into chore/runupdatescreenshot
sagar-qa007 Dec 19, 2024
d77812a
Merge branch 'release' into chore/runupdatescreenshot
sagar-qa007 Dec 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 23 additions & 1 deletion .github/workflows/build-client-server-count.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ jobs:
matrix_count: ${{steps.matrix.outputs.matrix_count}}
run_count: ${{ steps.countArgs.outputs.run_count }}
is-pg-build: ${{steps.args.outputs.is-pg-build}}
update_snapshot: ${{steps.args.outputs.update_snapshot}}
specs_to_run: ${{steps.args.outputs.specs_to_run}}
steps:
- name: Checkout the head commit of the branch
uses: actions/checkout@v4
Expand All @@ -38,6 +40,22 @@ jobs:
runId=$((checkArg + 0))
echo "runId=$runId" >> $GITHUB_OUTPUT
fi

# Check for update_snapshot
checkArg=${{ github.event.client_payload.slash_command.args.named.update_snapshot }}
if [[ -z "$checkArg" ]]; then
echo "update_snapshot=false" >> $GITHUB_OUTPUT
else
echo "update_snapshot=$checkArg" >> $GITHUB_OUTPUT
fi

# Check for spec file
checkArg=${{ github.event.client_payload.slash_command.args.named.specs_to_run }}
if [[ -z "$checkArg" ]]; then
echo "specs_to_run=''" >> $GITHUB_OUTPUT
else
echo "specs_to_run=$checkArg" >> $GITHUB_OUTPUT
fi

- name: Set run count
id: countArgs
Expand Down Expand Up @@ -97,7 +115,7 @@ jobs:
body: |
Tests running at: <${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}>.
[Cypress dashboard](https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=${{ github.run_id }}&attempt=${{ github.run_attempt }}&selectiontype=test&testsstatus=failed&specsstatus=fail)
PR: #${{ fromJson(steps.args.outputs.pr) }}.
PR: #${{ fromJson(steps.args.outputs.pr) }} with spec: ${{steps.args.outputs.specs_to_run}}.

server-build:
name: server-build
Expand Down Expand Up @@ -149,6 +167,8 @@ jobs:
with:
pr: ${{fromJson(needs.file-check.outputs.pr)}}
run_count: ${{fromJson(needs.file-check.outputs.run_count)}}
update_snapshot: ${{fromJson(needs.file-check.outputs.update_snapshot)}}
specs_to_run: ${{needs.file-check.outputs.specs_to_run}}


ci-test-limited-existing-docker-image:
Expand All @@ -162,6 +182,8 @@ jobs:
pr: ${{fromJson(needs.file-check.outputs.pr)}}
previous-workflow-run-id: ${{ fromJson(needs.file-check.outputs.runId) }}
run_count: ${{fromJson(needs.file-check.outputs.run_count)}}
update_snapshot: ${{fromJson(needs.file-check.outputs.update_snapshot)}}
specs_to_run: ${{needs.file-check.outputs.specs_to_run}}

ci-test-limited-result:
needs: [file-check, ci-test-limited]
Expand Down
54 changes: 49 additions & 5 deletions .github/workflows/ci-test-limited-with-count.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,16 @@ on:
required: false
type: number
default: 1
update_snapshot:
description: 'Give option to update snapshot (true/false)'
required: false
type: boolean
default: false
specs_to_run:
description: 'Cypress spec file(s) to run'
required: false
type: string

workflow_call:
inputs:
pr:
Expand All @@ -35,6 +45,15 @@ on:
required: false
type: number
default: 1
update_snapshot:
description: 'Give option to update snapshot (true/false)'
required: false
type: boolean
default: false
specs_to_run:
description: 'Cypress spec file(s) to run'
required: false
type: string

jobs:
ci-test-limited:
Expand Down Expand Up @@ -111,10 +130,18 @@ jobs:
else
echo "rerun=false" >> $GITHUB_OUTPUT
fi

- name: Upload existing snapshot data
if: always()
uses: actions/upload-artifact@v4
with:
name: cypress-existing-compare-snapshots
path: ${{ github.workspace }}/app/client/cypress/snapshots
overwrite: true

# Get specs to run
- name: Get specs to run
if: steps.run_result.outputs.run_result != 'success' && steps.run_result.outputs.run_result != 'failedtest'
if: ${{ (inputs.specs_to_run == '' || inputs.specs_to_run == null) && steps.run_result.outputs.run_result != 'success' && steps.run_result.outputs.run_result != 'failedtest' }}
run: |
specs_to_run=""
while IFS= read -r line
Expand Down Expand Up @@ -349,10 +376,19 @@ jobs:
NODE_ENV: development
run: |
cd app/client
npx cypress-repeat-pro run -n ${{ inputs.run_count }} --force \
--spec ${{ env.specs_to_run }} \
if [[ "${{ inputs.update_snapshot }}" == "true" ]]; then
echo "Running Cypress with snapshot updates..."
CYPRESS_updateSnapshots=true npx cypress-repeat-pro run -n "${{ inputs.run_count }}" --force \
--spec "${{ inputs.specs_to_run }}" \
--config-file "cypress_ci_custom.config.ts" \
--browser ${{ env.BROWSER_PATH }}
--browser "${{ env.BROWSER_PATH }}"
else
echo "Running Cypress tests without snapshot updates..."
npx cypress-repeat-pro run -n "${{ inputs.run_count }}" --force \
--spec "${{ env.specs_to_run }}" \
--config-file "cypress_ci_custom.config.ts" \
--browser "${{ env.BROWSER_PATH }}"
fi
Comment on lines +379 to +391
Copy link
Contributor

@coderabbitai coderabbitai bot Dec 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Inconsistent spec path usage needs to be fixed

The workflow shows that specs_to_run is set as an environment variable when no specific specs are provided via input. The inconsistency in using inputs.specs_to_run vs env.specs_to_run could lead to test execution issues.

  • Line 380: Update mode uses ${{ inputs.specs_to_run }}
  • Line 388: Non-update mode uses ${{ env.specs_to_run }}

Both modes should consistently use either the input parameter or environment variable to ensure the same specs are run regardless of snapshot update mode.

🔗 Analysis chain

LGTM: Flexible snapshot update handling.

Good implementation of conditional snapshot updates based on the input parameter. The code clearly distinguishes between update and non-update modes.

However, there's a minor inconsistency in spec selection:

  • Update mode uses ${{ inputs.specs_to_run }}
  • Non-update mode uses ${{ env.specs_to_run }}

Let's verify if this difference is intentional:

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check if specs_to_run is used consistently across the workflow
rg -A 2 'specs_to_run' .github/workflows/ci-test-limited-with-count.yml

Length of output: 1359

cat cy-repeat-summary.txt
# Define the path for the failure flag file
FAILURE_FLAG_FILE="ci_test_status.txt"
Expand Down Expand Up @@ -453,8 +489,16 @@ jobs:
if: always()
with:
name: server-logs
path: app/server/server-logs.log
path: ${{ github.workspace }}/app/server/server-logs.log
overwrite: true

- name: Upload new compared snapshot data
if: always()
uses: actions/upload-artifact@v4
with:
name: cypress-new-compare-snapshots
path: ${{ github.workspace }}/app/client/cypress/snapshots
overwrite: true

# Set status = success
- name: Save the status of the run
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci-test-limited.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -418,7 +418,7 @@ jobs:
if: failure()
with:
name: server-logs
path: app/server/server-logs.log
path: ${{ github.workspace }}/app/server/server-logs.log
overwrite: true

# Set status = success
Expand Down