CI: all-in-one workflow by snazy · Pull Request #3625 · apache/polaris

snazy · 2026-01-31T11:09:28Z

This change moves all CI jobs into a single workflow. A single workflow comes with a couple advantages:

all jobs are visible on one page
option to re-run all failed jobs at once

The refactoring also simplifies the .asf.yaml file by referencing a single required check that can only succeeds if dependent jobs were successful.

The actual CI jobs are in the ci.yml file, which is only triggered via a workflow_call event.
There are two workflows that call ci.yml:

ci-main.yml for main and release/* branches, with a concurrency group that does not cancel already running workflows
ci-pr.yml for PRs with a concurrency group that cancels previous CI runs

The names of these two calling workflows include information that enrich the workflow view, and the workflows for main, release branches and PRs are grouped separartely in the GH Actions page for the repository. In other words, the reference name (for main + release branches) or the PR number and title are shown in the workflow runs list on the GH Actions page.

This change includes #3626, #3627, #3628 as separate commits

snazy · 2026-02-03T10:18:49Z

As this PR updates and also requires the required_checks in .asf.yaml, #3646 needs to be merged before this one.

This change is only needed to update the required-checks to be able to eventually merge apache#3625.

This change moves all CI jobs into a single workflow. A single workflow comes with a couple advantages: * all jobs are visible on one page * option to re-run all failed jobs at once The refactoring also simplifies the `.asf.yaml` file by referencing a single required check that can only succeeds if dependent jobs were successful. The actual CI jobs are in the `ci.yml` file, which is only triggered via a `workflow_call` event. There are two workflows that call `ci.yml`: * `ci-main.yml` for `main` and `release/*` branches, with a concurrency group that does not cancel already running workflows * `ci-pr.yml` for PRs with a concurrency group that cancels previous CI runs The names of these two calling workflows include information that enrich the workflow view, and the workflows for main, release branches and PRs are grouped separartely in the GH Actions page for the repository. In other words, the reference name (for main + release branches) or the PR number and title are shown in the workflow runs list on the GH Actions page.

snazy · 2026-02-03T18:04:26Z

(force-pushed, conflict w/ #3652)

This change is only needed to update the required-checks to be able to eventually merge #3625.

After apache#3625 the `jq` select to look for required checks can be simplified to select only the `Required Checks` check/job.

After #3625 the `jq` select to look for required checks can be simplified to select only the `Required Checks` check/job.

This change is only needed to update the required-checks to be able to eventually merge apache#3625.

This change moves all CI jobs into a single workflow. A single workflow comes with a couple advantages: * all jobs are visible on one page * option to re-run all failed jobs at once The refactoring also simplifies the `.asf.yaml` file by referencing a single required check that can only succeeds if dependent jobs were successful. The actual CI jobs are in the `ci.yml` file, which is only triggered via a `workflow_call` event. There are two workflows that call `ci.yml`: * `ci-main.yml` for `main` and `release/*` branches, with a concurrency group that does not cancel already running workflows * `ci-pr.yml` for PRs with a concurrency group that cancels previous CI runs The names of these two calling workflows include information that enrich the workflow view, and the workflows for main, release branches and PRs are grouped separartely in the GH Actions page for the repository. In other words, the reference name (for main + release branches) or the PR number and title are shown in the workflow runs list on the GH Actions page.

After apache#3625 the `jq` select to look for required checks can be simplified to select only the `Required Checks` check/job.

* Releasey: adjust workflwo for Apache org level secrets (apache#3647) See reference [INFRA-27430](https://issues.apache.org/jira/browse/INFRA-27430), requiring us to use `DOCKERHUB_USER` + `DOCKERHUB_TOKEN` instead of `DOCKERHUB_USERNAME` + `DOCKERHUB_TOKEN`. * Guides: fix setup scripts to yield correct exit code (apache#3612) The statements in the shell scripts for the setup services are often concatenated using `;`, which means that a previous' command exit code is _not_ propagated and the service, although it failed, is determined to be successful. This change updates those scripts to use `&&` for the statement concatenation. "Final" setup services (aka "polaris-setup") now have a final `sleep 120`. This is due to the behavior of `docker compose up --detach --wait`, which considers _any_ service (without dependants) that exits with exit code 0 as a failure, leading to that docker-compose command yielding an error code. That would break the guides testing code (apache#3553). That `sleep 120` in "polaris-setup" services does **not** cause a delay of the compose starting up - it is purely a "hack around" Docker Compose not having a notion of "setup services". To avoid merge conflicts, this change also: * updates affected `curl` invocations (as apache#3610) * removes superfluous `restart: "no"` * Add PolarisEventType int codes and remove unused before/after commit view/table values. (apache#3608) This is a continuation of apache#3418 where we agreed we should remove associate enums values from Enum definition. The code also adds a constructor to help not having to rely on ordinals() to to have enum codes. I incremented enum code assignments by 100 based on categories. I am happy to take feedback here. Does not change logic and removes only enums that are no longer used so behavior does not change. * docs: Add quick guide for downstream builds (apache#3601) * docs: Add quick guide for downstream builds * IntegrationTestsHelper: fix extract & merge logic (apache#3650) Both methods were flawed: * `extractFromAnnotatedElements` wasn't properly delegating to the class if the method isn't annotated * `mergeFromAnnotatedElements` wasn't properly prioritizing method properties over class properties. * Community page: Yong Zheng - PPMC Member (apache#3653) * chore(deps): update actions/checkout digest to de0fac2 (apache#3652) * Use `quarkus.package.jar.type` (apache#3644) Switch to `quarkus.package.jar.type` instead of the old `quarkus.package.type` build property as suggested by Quarkus build warning: ``` 2026-02-03T00:26:05.672955189Z WorkerExecutor Queue WARN Configuration property 'quarkus.package.type' has been deprecated and replaced by: [quarkus.package.jar.enabled, quarkus.package.jar.type, quarkus.native.enabled, quarkus.native.sources-only] ``` * Site: Add blog post for Floe Polaris Integration (apache#3645) * "Stale" job: restrict executions and adjust issue permissions (apache#3636) * Add copyright on website (apache#3659) * Sanitize principal names in AWS STS role session names (apache#3525) Principal names containing invalid characters (spaces, parentheses, etc.) were causing AWS STS AssumeRole requests to fail with validation errors. AWS STS role session names must match the pattern [\w+=,.@-]*. This change: - Adds AwsRoleSessionNameSanitizer utility class to sanitize strings for use as AWS STS role session names - Replaces invalid characters with underscores and truncates to 64 characters (AWS maximum) - Updates AwsCredentialsStorageIntegration to sanitize principal names when INCLUDE_PRINCIPAL_NAME_IN_SUBSCOPED_CREDENTIAL is enabled - Adds tests to verify sanitization behavior and AWS pattern compliance Fixes issue where principal names like "Joe (local)" would produce invalid role session names like "polaris-Joe (local)" and cause AssumeRole to fail. Now sanitized to "polaris-Joe__local_". Co-authored-by: carc-prathyush-shankar <prathyush.shankar@carbonarc.co> * Site: Adds the copyright message to all site pages (apache#3661) * Site: Change Security link to local security reporting page (apache#3662) * fix(deps): update dependency org.mongodb:mongodb-driver-sync to v5.6.3 (apache#3654) * Releasey: use Apache org Nexus credentials (apache#3651) * Releasey: use the correct SVN credentials (apache#3648) * CI: Prerequisite PR for apache#3625 (apache#3646) This change is only needed to update the required-checks to be able to eventually merge apache#3625. * CI: all-in-one workflow (apache#3625) This change moves all CI jobs into a single workflow. A single workflow comes with a couple advantages: * all jobs are visible on one page * option to re-run all failed jobs at once The refactoring also simplifies the `.asf.yaml` file by referencing a single required check that can only succeeds if dependent jobs were successful. The actual CI jobs are in the `ci.yml` file, which is only triggered via a `workflow_call` event. There are two workflows that call `ci.yml`: * `ci-main.yml` for `main` and `release/*` branches, with a concurrency group that does not cancel already running workflows * `ci-pr.yml` for PRs with a concurrency group that cancels previous CI runs The names of these two calling workflows include information that enrich the workflow view, and the workflows for main, release branches and PRs are grouped separartely in the GH Actions page for the repository. In other words, the reference name (for main + release branches) or the PR number and title are shown in the workflow runs list on the GH Actions page. * Fix CI required checks (apache#3666) * Add Sung as committer (apache#3665) * Fix `CatalogFederationIntegrationTest.testFederatedCatalogWithCredentialVending()` for AWSSDK update (apache#3664) Recent AWSSDK versions introduce `software.amazon.awssdk.services.s3.model.AccessDeniedException`, hence the assertion on `S3Exception` fails. * CI/main: re-add commit message to `run_name` (apache#3668) The default `run_name` value is, in case of `push` events, the commit message. This change re-adds the commit message. * Add CI workflow to test against Iceberg unreleased versions (apache#3630) * Update dependency software.amazon.awssdk:bom to v2.41.21 (apache#3639) * Update actions/checkout digest to de0fac2 (apache#3671) * Nit: Fix wrong `Nullable` import (apache#3672) * Explicitly set build-time property `quarkus.datasource.db-kind` (apache#3674) The property is set for the Polaris admin tool, but not for Polaris server. This causes a startup error with Quarkus 3.31. Error message: ``` ERROR: Failed to start application java.lang.RuntimeException: Failed to start quarkus at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source) at io.quarkus.runtime.Application.start(Application.java:116) at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:119) at io.quarkus.runtime.Quarkus.run(Quarkus.java:79) at io.quarkus.runtime.Quarkus.run(Quarkus.java:50) at io.quarkus.runtime.Quarkus.run(Quarkus.java:143) at io.quarkus.runner.GeneratedMain.main(Unknown Source) at io.quarkus.bootstrap.runner.QuarkusEntryPoint.doRun(QuarkusEntryPoint.java:86) at io.quarkus.bootstrap.runner.QuarkusEntryPoint.main(QuarkusEntryPoint.java:37) Caused by: java.lang.IllegalStateException: Build time property cannot be changed at runtime: - quarkus.datasource.db-kind is set to 'postgresql' but it is build time fixed to 'null'. Did you change the property quarkus.datasource.db-kind after building the application? at io.quarkus.runtime.configuration.ConfigRecorder.handleConfigChange(ConfigRecorder.java:72) at io.quarkus.runner.recorded.ConfigGenerationBuildStep$checkForBuildTimeConfigChange1532146938.deploy_6(Unknown Source) at io.quarkus.runner.recorded.ConfigGenerationBuildStep$checkForBuildTimeConfigChange1532146938.deploy(Unknown Source) ... 9 more ``` * Last merged commit 2651e06 --------- Co-authored-by: Innocent Djiofack <djiofack007@gmail.com> Co-authored-by: Dmitri Bourlatchkov <dmitri.bourlatchkov@gmail.com> Co-authored-by: Alexandre Dutra <adutra@apache.org> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Neelesh Salian <nssalian@users.noreply.github.com> Co-authored-by: JB Onofré <jbonofre@apache.org> Co-authored-by: Prathyush Shankar <prathyush2018@gmail.com> Co-authored-by: carc-prathyush-shankar <prathyush.shankar@carbonarc.co> Co-authored-by: Russell Spitzer <russell.spitzer@GMAIL.COM>

* Simplify CatalogPrefixParser API (apache#3622) This component's methods do not need to have a `RealmContext` parameter: either the implementation is application-scoped, in which case the realm is irrelevant; or the implementation is request-scoped, in which case it can have the `RealmContext` injected. (FWIW, the default implementation is application-scoped.) This refactor will further simplify the remote request signing implementation. * Releasey: update check for required-checks (apache#3667) After apache#3625 the `jq` select to look for required checks can be simplified to select only the `Required Checks` check/job. * Wire external catalog properties into REST client config (apache#3480) * pass ExternalCatalog.properties through federation factories (Iceberg REST, Hive, Hadoop) * merge catalog properties with connection config, with connection config taking precedence * document proxy/timeout settings for Iceberg REST federation * add tests that exercise production merge logic * Correct build instruction, project properties require org.gradle.project prefix (apache#3680) * Correct build instruction, project properties require org.gradle.project prefix --------- Co-authored-by: Nandor Kollar <nkollar@cloudera.com> * Last merged commit 103c6ed --------- Co-authored-by: Alexandre Dutra <adutra@apache.org> Co-authored-by: Yong-Jin Lee <33987062+yj-lee0503@users.noreply.github.com> Co-authored-by: Nándor Kollár <nandorKollar@users.noreply.github.com> Co-authored-by: Nandor Kollar <nkollar@cloudera.com>

github-project-automation bot moved this to PRs In Progress in Basic Kanban Board Jan 31, 2026

github-project-automation bot added this to Basic Kanban Board Jan 31, 2026

snazy force-pushed the ci-all-in-one branch 28 times, most recently from eac0347 to aec345b Compare January 31, 2026 14:11

snazy mentioned this pull request Feb 3, 2026

CI: Prerequisite PR for #3625 #3646

Merged

snazy marked this pull request as ready for review February 3, 2026 10:17

adutra previously approved these changes Feb 3, 2026

View reviewed changes

github-project-automation bot moved this from PRs In Progress to Ready to merge in Basic Kanban Board Feb 3, 2026

snazy added 2 commits February 3, 2026 19:02

CI: Prerequisite PR for apache#3625

1f3c3b8

This change is only needed to update the required-checks to be able to eventually merge apache#3625.

snazy dismissed adutra’s stale review via e8ce7e5 February 3, 2026 18:04

snazy force-pushed the ci-all-in-one branch from b1278d2 to e8ce7e5 Compare February 3, 2026 18:04

dimas-b previously approved these changes Feb 3, 2026

View reviewed changes

adutra previously approved these changes Feb 3, 2026

View reviewed changes

adnanhemani approved these changes Feb 3, 2026

View reviewed changes

snazy added a commit that referenced this pull request Feb 4, 2026

CI: Prerequisite PR for #3625 (#3646)

3a6f292

This change is only needed to update the required-checks to be able to eventually merge #3625.

Merge branch 'main' into ci-all-in-one

f5fd7cb

snazy dismissed stale reviews from adutra and dimas-b via f5fd7cb February 4, 2026 07:15

jbonofre approved these changes Feb 4, 2026

View reviewed changes

snazy merged commit 6b6950c into apache:main Feb 4, 2026
13 checks passed

github-project-automation bot moved this from Ready to merge to Done in Basic Kanban Board Feb 4, 2026

snazy deleted the ci-all-in-one branch February 4, 2026 07:17

snazy added a commit to snazy/polaris that referenced this pull request Feb 4, 2026

Releasey: update check for required-checks

a601f5b

After apache#3625 the `jq` select to look for required checks can be simplified to select only the `Required Checks` check/job.

snazy added a commit to snazy/polaris that referenced this pull request Feb 4, 2026

Releasey: update check for required-checks

0814249

After apache#3625 the `jq` select to look for required checks can be simplified to select only the `Required Checks` check/job.

snazy mentioned this pull request Feb 4, 2026

Releasey: update check for required-checks #3667

Merged

snazy added a commit that referenced this pull request Feb 5, 2026

Releasey: update check for required-checks (#3667)

7270b07

After #3625 the `jq` select to look for required checks can be simplified to select only the `Required Checks` check/job.

sungwy pushed a commit to sungwy/polaris that referenced this pull request Feb 7, 2026

CI: Prerequisite PR for apache#3625 (apache#3646)

f9194ad

This change is only needed to update the required-checks to be able to eventually merge apache#3625.

sungwy pushed a commit to sungwy/polaris that referenced this pull request Feb 7, 2026

Releasey: update check for required-checks (apache#3667)

f75b5dc

After apache#3625 the `jq` select to look for required checks can be simplified to select only the `Required Checks` check/job.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI: all-in-one workflow#3625

CI: all-in-one workflow#3625
snazy merged 3 commits intoapache:mainfrom
snazy:ci-all-in-one

snazy commented Jan 31, 2026 •

edited

Loading

Uh oh!

snazy commented Feb 3, 2026

Uh oh!

snazy commented Feb 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

snazy commented Jan 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

snazy commented Feb 3, 2026

Uh oh!

snazy commented Feb 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

snazy commented Jan 31, 2026 •

edited

Loading