-
Notifications
You must be signed in to change notification settings - Fork 127
CSPL-3584: Split run-tests.sh into multiple files #1507
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Pull Request Test Coverage Report for Build 15283563414Details
💛 - Coveralls |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR refactors the test workflow by splitting the monolithic run-tests.sh script into multiple modular scripts that handle operator deployment, image preparation, and test execution.
- Introduces new scripts: deploy-operator.sh, get-private-registry-operator.sh, get-private-registry-enterprise.sh, and trigger-tests.sh.
- Refactors run-tests.sh to delegate specific tasks to the new scripts, improving maintainability and clarity.
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| test/trigger-tests.sh | New script to set up environment and execute tests using ginkgo. |
| test/run-tests.sh | Refactored to call dedicated scripts for deployment, image handling, and tests. |
| test/get-private-registry-operator.sh | New script to pull/tag operator image from a private registry. |
| test/get-private-registry-enterprise.sh | New script to pull/tag enterprise image from a private registry. |
| test/deploy-operator.sh | New script to deploy the Splunk Operator using Helm or CRDs. |
| # Running only smoke test cases by default or value passed through TEST_FOCUS env variable. To run different test packages add/remove path from focus argument or TEST_FOCUS variable | ||
| echo "ginkgo --junit-report=inttest.xml -vv --keep-going --trace -r --timeout=7h -nodes=${CLUSTER_NODES} --focus="${TEST_TO_RUN}" --skip="${TEST_TO_SKIP}" --output-interceptor-mode=none --cover ${topdir}/test/ -- -commit-hash=${COMMIT_HASH} -operator-image=${PRIVATE_SPLUNK_OPERATOR_IMAGE} -splunk-image=${PRIVATE_SPLUNK_ENTERPRISE_IMAGE} -cluster-wide=${CLUSTER_WIDE}" | ||
| ginkgo --junit-report=inttest-junit.xml --output-dir=`pwd` -vv --keep-going --trace -r --timeout=7h -nodes=${CLUSTER_NODES} --focus="${TEST_TO_RUN}" --skip="${TEST_TO_SKIP}" --output-interceptor-mode=none --cover ${topdir}/test/ -- -commit-hash=${COMMIT_HASH} -operator-image=${PRIVATE_SPLUNK_OPERATOR_IMAGE} -splunk-image=${PRIVATE_SPLUNK_ENTERPRISE_IMAGE} -cluster-wide=${CLUSTER_WIDE} | ||
| PRIVATE_SPLUNK_OPERATOR_IMAGE=$(bash ${scriptdir}/get-private-registry-operator.sh | tail -1) |
Copilot
AI
May 13, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using 'tail -1' to extract the final output of the get-private-registry-operator.sh script can be brittle if extra logging output is printed. Consider modifying the script to send log messages to stderr and reserve stdout for the image value only.
| echo "ginkgo --junit-report=inttest.xml -vv --keep-going --trace -r --timeout=7h -nodes=${CLUSTER_NODES} --focus="${TEST_TO_RUN}" --skip="${TEST_TO_SKIP}" --output-interceptor-mode=none --cover ${topdir}/test/ -- -commit-hash=${COMMIT_HASH} -operator-image=${PRIVATE_SPLUNK_OPERATOR_IMAGE} -splunk-image=${PRIVATE_SPLUNK_ENTERPRISE_IMAGE} -cluster-wide=${CLUSTER_WIDE}" | ||
| ginkgo --junit-report=inttest-junit.xml --output-dir=`pwd` -vv --keep-going --trace -r --timeout=7h -nodes=${CLUSTER_NODES} --focus="${TEST_TO_RUN}" --skip="${TEST_TO_SKIP}" --output-interceptor-mode=none --cover ${topdir}/test/ -- -commit-hash=${COMMIT_HASH} -operator-image=${PRIVATE_SPLUNK_OPERATOR_IMAGE} -splunk-image=${PRIVATE_SPLUNK_ENTERPRISE_IMAGE} -cluster-wide=${CLUSTER_WIDE} | ||
| PRIVATE_SPLUNK_OPERATOR_IMAGE=$(bash ${scriptdir}/get-private-registry-operator.sh | tail -1) | ||
| PRIVATE_SPLUNK_ENTERPRISE_IMAGE=$(bash ${scriptdir}/get-private-registry-enterprise.sh | tail -1) |
Copilot
AI
May 13, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using 'tail -1' to capture the final output from get-private-registry-enterprise.sh may lead to errors if additional output is produced. Consider isolating the image output to stdout (logging to stderr) to ensure reliable retrieval.
| fi | ||
|
|
||
| # Return the value of PRIVATE_SPLUNK_OPERATOR_IMAGE | ||
| echo "${PRIVATE_SPLUNK_OPERATOR_IMAGE}" |
Copilot
AI
May 13, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since the script outputs extra logging (e.g., 'Docker images'), the final echo might be mixed with those logs. Redirect log messages to stderr so that stdout contains only the intended value for callers.
| fi | ||
|
|
||
| # Return the value of PRIVATE_SPLUNK_ENTERPRISE_IMAGE | ||
| echo "${PRIVATE_SPLUNK_ENTERPRISE_IMAGE}" |
Copilot
AI
May 13, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Similar to the operator script, ensure that all logging is directed to stderr so that the echoed image value is not obscured by extra output. This will make the 'tail -1' extraction in the calling script more reliable.
vivekr-splunk
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like this idea, but I want to make sure we have tested the scripts for all different options
helm, integration test, deployed cluster scoped, deployed namespace scoped
I have already tried with smoke and integration tests deployed with cluster scoped. I have triggered the namespace scoped, azure, and gcp workflows with this change to test them as well. Helm tests do not use this script, so that is unnecessary to test. I will remove the workflow file changes from this branch before merging after they are verified. |
* CSPL-3704 SmartStore ownerReferences removed * CSPL-3704 Integration tests enabled to check the PR * CSPL-3704 Fix failing tests * CSPL-3704 Remove branch from int tests * test * CSPL-3705 Ignoring an error if decommisioning already enabled * CSPL-3705 Removing branch from integ tests * CSPL-3705 Addressing a comment * clean-up deprecated dirs - .circleci & .devcontainer (#1499) Co-authored-by: Patryk Wasielewski <[email protected]> * set imagePullPolicy default in helm chart (#1513) * CSPL-3186: Upgrade Enterprise Security Version 8.0.2 (#1425) * Updates for integration tests and documentation with latest ES version * Update enterprise security versions and app files * remove branch from inte test workflow * CSPL-3688: Update Prerelease Workflow (#1502) * add more automation to pre-release workflow * update version and replaced version in bundle files * update dockerfile sok version, not enterprise version * fix typo * CSPL-3584: Split run-tests.sh into multiple files (#1507) * split run-tests.sh into multiple files * trigger integration tests on branch * use scriptdir to run sh files * remove trigger int test workflow * test azure, gcp, and namespace scoped workflows * cleanup workflows * feature: add support for pre-created PVs - admin-managed-pv annotation (#1509) * add support for admin-managed-pv annotation --------- Co-authored-by: Patryk Wasielewski <[email protected]> * Removing App Framework tests for C3 and M4 on ARM builds * CSPL-3702 Replacing () with {} in script variables * Ginkgo upgrade * CSPL-3768 Adding inputs to Graviton pipelines and tests * CSPL-3768 Not disclosing ECR secret value * CSPL-3678 Introducing pipeline for Graviton and some fixes * CSPL-3768 Fixes * CSPL-3768 Fixes * CSPL-3768 Addressing Copilot suggestions * CSPL-3768 Addressing comments * CSPL-3759 Addressing soon to be deprecated * CSPL-3784: Update base image to latest ubi8-minimal version (#1525) * update base image to latest ubi8-minimal version * trigger integration test cases for branch * trigger tests * cleanup * update Dockerfile comment * CSPL-3675 Update Operator-SDK to v1.39 (#1488) * v.1.39.0 migration --------- Co-authored-by: igor.grzankowski <@splunk.com> Co-authored-by: Vivek Reddy <[email protected]> Co-authored-by: rlieberman-splunk <[email protected]> Co-authored-by: kasiakoziol <[email protected]> Co-authored-by: Patryk Wasielewski <[email protected]> * CSPL_3759 Ginkgo types to v2 * Add DeepWiki badge (#1529) Co-authored-by: igor.grzankowski <@splunk.com> * Upgrade golang.org/x/net version to v0.38.0 (#1530) * upgrade golang.org/x/net version to v0.38.0 * trigger integration test cases on branch * remove integration test trigger * CSPL-3783: Update AppFramework docs with troubleshooting information (#1527) * Add more logs around invalid phase and downloadPending (#1528) * Remove kube rbac proxy from helm charts (#1531) Co-authored-by: igor.grzankowski <@splunk.com> * CSPL-3851 Adding info to docs about session stickiness for ingress * Remove in progress phase * Revert "Remove in progress phase" This reverts commit 3c919d6. * update SmartStore documentation for gcp and azure (#1541) * Backport main to develop for Splunk Operator Release 2.8.1 (#1542) (#1543) * release 2.8.1 chnages - backported --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Patryk Wasielewski <[email protected]> Co-authored-by: patrykw-splunk <[email protected]> * CSPL-3898 Fixing int-helm tests failing after SDK upgrade (#1544) * CSPL-3867: SHC and CM Error Message Visibility (#1548) * print error for update status failed * add deployerPhase initial value * set correct Kind in ClusterManager events * cleanup * use v4 in test * CSPL-3905: Security and dependency updates (#1551) * initial changes for aws-sdk-go-v2 * code compiles and unit tests pass * trigger smoke and integration tests, update context * set correct path for downloading from s3 bucket for tests * update ENTERPRISE_LICENSE_LOCATION * security updates, uncomment test suites * cleanup * [CSPL-3912] Allow Custom Probe Scripts (#1549) * Promote Develop to main for Splunk Operator Release 2.8.1 (#1542) * release preparation - release 2.8.1 --------- Co-authored-by: Patryk Wasielewski <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: patrykw-splunk <[email protected]> * check for existing configmap before creating a new one * update error handling * fix unit tests * cleanup and documentation updates --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Patryk Wasielewski <[email protected]> Co-authored-by: patrykw-splunk <[email protected]> * CSPL-3913: Pass dynamic environment variables in Splunk StatefulSet for Ansible-based config generation (#1555) * add new environment variables for config generation * unit test updates * add api version to env var * Promote Develop to main for Splunk Operator Release 2.8.1 (#1542) (#1553) * release 2.8.1 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Patryk Wasielewski <[email protected]> Co-authored-by: patrykw-splunk <[email protected]> * Document skipping apply cluster-bundle on cluster managers (#1556) * Update of shc upgrade process (#1547) Update of shc upgrade process --------- Co-authored-by: Patryk Wasielewski <[email protected]> * Update Helm.md (#1563) Co-authored-by: Patryk Wasielewski <[email protected]> * Merge Splunk10 feature changes into develop (#1559) * CSPL-3707: Update documentation around minimum number of indexer cluster peers (#1558) * document minimum number of CR replicas * update examples documentation * Default Build to multi-platform amd64 and arm64 (#1557) * add linux/arm64 as a platform to build the operator by default * set platforms in dockerfile * use tag instead of sha * update tag version * update base image to latest * Remove kube-rbac-proxy references and implement upgrade-sdk 1.38 changes (#1565) * remove kube-rbac-proxy references and implement upgrade-sdk 1.38 changes * fix kustomize references * fix container number for debug * cleanup * fix service for metrics --------- Co-authored-by: Patryk Wasielewski <[email protected]> * helm changes for 3.0.0 release (#1566) Co-authored-by: Patryk Wasielewski <[email protected]> * Fix kustomization templates after removing kube-rbac-proxy (#1570) * fix kustomization templates --------- Co-authored-by: Patryk Wasielewski <[email protected]> * Splunk Operator 3.0.0 release (#1572) * Update changelog --------- Co-authored-by: Igor-splunk <[email protected]> Co-authored-by: igor.grzankowski <@splunk.com> * remove old helm charts from splunk-enterprise/charts folder --------- Co-authored-by: kasiakoziol <[email protected]> Co-authored-by: patrykw-splunk <[email protected]> Co-authored-by: Patryk Wasielewski <[email protected]> Co-authored-by: rlieberman-splunk <[email protected]> Co-authored-by: Igor Grzankowski <[email protected]> Co-authored-by: Vivek Reddy <[email protected]> Co-authored-by: igor.grzankowski <@splunk.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: patrykw-splunk <[email protected]> Co-authored-by: Igor-splunk <[email protected]>
* Promote Develop to main for Splunk Operator Release 3.0.0 (#1573) * CSPL-3704 SmartStore ownerReferences removed * CSPL-3704 Integration tests enabled to check the PR * CSPL-3704 Fix failing tests * CSPL-3704 Remove branch from int tests * test * CSPL-3705 Ignoring an error if decommisioning already enabled * CSPL-3705 Removing branch from integ tests * CSPL-3705 Addressing a comment * clean-up deprecated dirs - .circleci & .devcontainer (#1499) Co-authored-by: Patryk Wasielewski <[email protected]> * set imagePullPolicy default in helm chart (#1513) * CSPL-3186: Upgrade Enterprise Security Version 8.0.2 (#1425) * Updates for integration tests and documentation with latest ES version * Update enterprise security versions and app files * remove branch from inte test workflow * CSPL-3688: Update Prerelease Workflow (#1502) * add more automation to pre-release workflow * update version and replaced version in bundle files * update dockerfile sok version, not enterprise version * fix typo * CSPL-3584: Split run-tests.sh into multiple files (#1507) * split run-tests.sh into multiple files * trigger integration tests on branch * use scriptdir to run sh files * remove trigger int test workflow * test azure, gcp, and namespace scoped workflows * cleanup workflows * feature: add support for pre-created PVs - admin-managed-pv annotation (#1509) * add support for admin-managed-pv annotation --------- Co-authored-by: Patryk Wasielewski <[email protected]> * Removing App Framework tests for C3 and M4 on ARM builds * CSPL-3702 Replacing () with {} in script variables * Ginkgo upgrade * CSPL-3768 Adding inputs to Graviton pipelines and tests * CSPL-3768 Not disclosing ECR secret value * CSPL-3678 Introducing pipeline for Graviton and some fixes * CSPL-3768 Fixes * CSPL-3768 Fixes * CSPL-3768 Addressing Copilot suggestions * CSPL-3768 Addressing comments * CSPL-3759 Addressing soon to be deprecated * CSPL-3784: Update base image to latest ubi8-minimal version (#1525) * update base image to latest ubi8-minimal version * trigger integration test cases for branch * trigger tests * cleanup * update Dockerfile comment * CSPL-3675 Update Operator-SDK to v1.39 (#1488) * v.1.39.0 migration --------- Co-authored-by: igor.grzankowski <@splunk.com> Co-authored-by: Vivek Reddy <[email protected]> Co-authored-by: rlieberman-splunk <[email protected]> Co-authored-by: kasiakoziol <[email protected]> Co-authored-by: Patryk Wasielewski <[email protected]> * CSPL_3759 Ginkgo types to v2 * Add DeepWiki badge (#1529) Co-authored-by: igor.grzankowski <@splunk.com> * Upgrade golang.org/x/net version to v0.38.0 (#1530) * upgrade golang.org/x/net version to v0.38.0 * trigger integration test cases on branch * remove integration test trigger * CSPL-3783: Update AppFramework docs with troubleshooting information (#1527) * Add more logs around invalid phase and downloadPending (#1528) * Remove kube rbac proxy from helm charts (#1531) Co-authored-by: igor.grzankowski <@splunk.com> * CSPL-3851 Adding info to docs about session stickiness for ingress * Remove in progress phase * Revert "Remove in progress phase" This reverts commit 3c919d6. * update SmartStore documentation for gcp and azure (#1541) * Backport main to develop for Splunk Operator Release 2.8.1 (#1542) (#1543) * release 2.8.1 chnages - backported --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Patryk Wasielewski <[email protected]> Co-authored-by: patrykw-splunk <[email protected]> * CSPL-3898 Fixing int-helm tests failing after SDK upgrade (#1544) * CSPL-3867: SHC and CM Error Message Visibility (#1548) * print error for update status failed * add deployerPhase initial value * set correct Kind in ClusterManager events * cleanup * use v4 in test * CSPL-3905: Security and dependency updates (#1551) * initial changes for aws-sdk-go-v2 * code compiles and unit tests pass * trigger smoke and integration tests, update context * set correct path for downloading from s3 bucket for tests * update ENTERPRISE_LICENSE_LOCATION * security updates, uncomment test suites * cleanup * [CSPL-3912] Allow Custom Probe Scripts (#1549) * Promote Develop to main for Splunk Operator Release 2.8.1 (#1542) * release preparation - release 2.8.1 --------- Co-authored-by: Patryk Wasielewski <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: patrykw-splunk <[email protected]> * check for existing configmap before creating a new one * update error handling * fix unit tests * cleanup and documentation updates --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Patryk Wasielewski <[email protected]> Co-authored-by: patrykw-splunk <[email protected]> * CSPL-3913: Pass dynamic environment variables in Splunk StatefulSet for Ansible-based config generation (#1555) * add new environment variables for config generation * unit test updates * add api version to env var * Promote Develop to main for Splunk Operator Release 2.8.1 (#1542) (#1553) * release 2.8.1 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Patryk Wasielewski <[email protected]> Co-authored-by: patrykw-splunk <[email protected]> * Document skipping apply cluster-bundle on cluster managers (#1556) * Update of shc upgrade process (#1547) Update of shc upgrade process --------- Co-authored-by: Patryk Wasielewski <[email protected]> * Update Helm.md (#1563) Co-authored-by: Patryk Wasielewski <[email protected]> * Merge Splunk10 feature changes into develop (#1559) * CSPL-3707: Update documentation around minimum number of indexer cluster peers (#1558) * document minimum number of CR replicas * update examples documentation * Default Build to multi-platform amd64 and arm64 (#1557) * add linux/arm64 as a platform to build the operator by default * set platforms in dockerfile * use tag instead of sha * update tag version * update base image to latest * Remove kube-rbac-proxy references and implement upgrade-sdk 1.38 changes (#1565) * remove kube-rbac-proxy references and implement upgrade-sdk 1.38 changes * fix kustomize references * fix container number for debug * cleanup * fix service for metrics --------- Co-authored-by: Patryk Wasielewski <[email protected]> * helm changes for 3.0.0 release (#1566) Co-authored-by: Patryk Wasielewski <[email protected]> * Fix kustomization templates after removing kube-rbac-proxy (#1570) * fix kustomization templates --------- Co-authored-by: Patryk Wasielewski <[email protected]> * Splunk Operator 3.0.0 release (#1572) * Update changelog --------- Co-authored-by: Igor-splunk <[email protected]> Co-authored-by: igor.grzankowski <@splunk.com> * remove old helm charts from splunk-enterprise/charts folder --------- Co-authored-by: kasiakoziol <[email protected]> Co-authored-by: patrykw-splunk <[email protected]> Co-authored-by: Patryk Wasielewski <[email protected]> Co-authored-by: rlieberman-splunk <[email protected]> Co-authored-by: Igor Grzankowski <[email protected]> Co-authored-by: Vivek Reddy <[email protected]> Co-authored-by: igor.grzankowski <@splunk.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: patrykw-splunk <[email protected]> Co-authored-by: Igor-splunk <[email protected]> * CSPL-4005: Remove CRDs from splunk/splunk-operator helm chart (#1579) * remove crds from helm chart, needs documentation * add documentation about adding CRDs before helm chart installation * repackage helm charts * cleanup helm test workflow * Helm package: helm package changes 3.0.0 (#1582) * helm test case * .env changes with spulnk 10.0.0 * updated splunk default version to 10 * removed helm test from branch --------- Co-authored-by: Vivek Reddy <[email protected]> * Vendor only Splunk Operator 3.0.0 in Splunk Enterprise chart to prevent fallback to older operator (#1583) * removing old splunk operator helm chart * generate helm packages --------- Co-authored-by: Vivek Reddy <[email protected]> * resolve merge conflict --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: kasiakoziol <[email protected]> Co-authored-by: patrykw-splunk <[email protected]> Co-authored-by: Patryk Wasielewski <[email protected]> Co-authored-by: Igor Grzankowski <[email protected]> Co-authored-by: Vivek Reddy <[email protected]> Co-authored-by: patrykw-splunk <[email protected]> Co-authored-by: Igor-splunk <[email protected]> Co-authored-by: vivekr-splunk <[email protected]>
Description
This pull request introduces changes to improve modularity and maintainability in the test scripts for deploying and testing the Splunk Operator. Key updates include the addition of new scripts for specific tasks, and refactoring of the main test script. The original run-test.sh file was split into multiple files, but the contents stay the same.
Key Changes
New Scripts for Deployment and Image Handling:
test/deploy-operator.shto handle Splunk Operator deployment, supporting both Helm and CRD-based installations.test/get-private-registry-enterprise.shandtest/get-private-registry-operator.shto manage private registry images for Splunk Enterprise and Operator, respectively. These scripts handle pulling, tagging, and pushing images. [1] [2]test/trigger-tests.shto encapsulate the logic for running tests, including setting up the environment and executing Ginkgo tests.Refactored Main Test Script:
test/run-tests.shby delegating deployment and image handling tasks to the new scripts. This improves readability and reduces duplication.Testing and Verification
Smoke and integration tests were run in the GitLab and GitHub pipelines.
Related Issues
https://splunk.atlassian.net/browse/CSPL-3584
PR Checklist