Skip to content

[CI] Create several schedules #13484

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mrodm merged 5 commits into from
Apr 10, 2025
Merged

[CI] Create several schedules #13484

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
c0423b6
Create several schedules with different steps via env vars
mrodm Apr 9, 2025
9f4c5b4
Test subject packages
mrodm Apr 9, 2025
8f03063
Changes for debugging
mrodm Apr 9, 2025
9f50d87
Update cronline settings
mrodm Apr 9, 2025
c493969
Remove changes for debugging
mrodm Apr 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions .buildkite/pipeline.schedule-daily.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ steps:
depends_on:
- step: "check"
allow_failure: false
if: |
build.env('TEST_PACKAGES_7_BRANCH') == "true"
Comment on lines +28 to +29
Copy link
Member

@jsoriano jsoriano Apr 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to have multiple pipelines instead of these conditions?

Copy link
Collaborator Author

@mrodm mrodm Apr 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought that it could be easier to just manage one buildkite pipeline with all the steps and select the steps required in each build using environment variables. That would avoid creating several new Buildkite jobs.

It could be tested like this, if there are any issues we could create new Buildkite pipelines later.


- label: "Check integrations local stacks - Stack Version v8.19"
trigger: "integrations"
Expand All @@ -37,6 +39,8 @@ steps:
depends_on:
- step: "check"
allow_failure: false
if: |
build.env('TEST_PACKAGES_8_BRANCH') == "true"

- label: "Check integrations local stacks - Stack Version v8.19 - LogsDB"
trigger: "integrations"
Expand All @@ -50,6 +54,37 @@ steps:
depends_on:
- step: "check"
allow_failure: false
if: |
build.env('TEST_PACKAGES_8_BRANCH') == "true"

- label: "Check integrations local stacks and basic subscription and LogsDB"
trigger: "integrations"
build:
env:
SERVERLESS: "false"
FORCE_CHECK_ALL: "true"
PUBLISH_COVERAGE_REPORTS: "false"
ELASTIC_SUBSCRIPTION: "basic"
STACK_LOGSDB_ENABLED: "true"
depends_on:
- step: "check"
allow_failure: false
if: |
build.env('TEST_PACKAGES_BASIC_SUBSCRIPTION') == "true"

- label: "Check integrations local stacks and basic subscription"
trigger: "integrations"
build:
env:
SERVERLESS: "false"
FORCE_CHECK_ALL: "true"
PUBLISH_COVERAGE_REPORTS: "false"
ELASTIC_SUBSCRIPTION: "basic"
depends_on:
- step: "check"
allow_failure: false
if: |
build.env('TEST_PACKAGES_BASIC_SUBSCRIPTION') == "true"

- label: "Check integrations local stacks - Stack Version v9.1"
trigger: "integrations"
Expand All @@ -62,6 +97,8 @@ steps:
depends_on:
- step: "check"
allow_failure: false
if: |
build.env('TEST_PACKAGES_9_BRANCH') == "true"

- label: "Check integrations in serverless - project: Observability"
key: "trigger-integrations-serverless-obs"
Expand All @@ -72,6 +109,8 @@ steps:
depends_on:
- step: "check"
allow_failure: false
if: |
build.env('TEST_PACKAGES_SERVERLESS') == "true"

- label: "Check integrations in serverless - project: Security"
key: "trigger-integrations-serverless-security"
Expand All @@ -82,10 +121,14 @@ steps:
depends_on:
- step: "check"
allow_failure: false
if: |
build.env('TEST_PACKAGES_SERVERLESS') == "true"

- label: ":package: Publish missing packages"
key: "trigger-integrations-publish"
trigger: "integrations-publish"
depends_on:
- step: "check"
allow_failure: false
if: |
build.env('REPUBLISH_PACKAGES') == "true"
25 changes: 24 additions & 1 deletion catalog-info.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -178,8 +178,31 @@ spec:
schedules:
main_daily:
branch: "main"
cronline: "30 1 * * *"
cronline: "30 0 * * *"
message: "Run the daily jobs"
env:
TEST_PACKAGES_7_BRANCH: "true"
TEST_PACKAGES_SERVERLESS: "true"
REPUBLISH_PACKAGES: "true"
Copy link
Member

@jsoriano jsoriano Apr 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, allowing to select steps this way also looks good.

main_daily_8_version:
branch: "main"
cronline: "30 1 * * *"
message: "Run the daily jobs for 8.x"
env:
TEST_PACKAGES_8_BRANCH: "true"
main_daily_9_version:
branch: "main"
cronline: "30 3 * * *"
message: "Run the daily jobs for 9.x"
env:
TEST_PACKAGES_9_BRANCH: "true"
main_daily_basic_subscription:
branch: "main"
cronline: "30 4 * * *"
message: "Run the daily jobs for basic subscription"
env:
TEST_PACKAGES_BASIC_SUBSCRIPTION: "true"

provider_settings:
trigger_mode: none # don't trigger jobs from github activity
build_pull_request_forks: false
Expand Down