Skip to content

WIP: CI for guides#3553

Draft
snazy wants to merge 2 commits intoapache:mainfrom
snazy:guides-ci
Draft

WIP: CI for guides#3553
snazy wants to merge 2 commits intoapache:mainfrom
snazy:guides-ci

Conversation

@snazy
Copy link
Member

@snazy snazy commented Jan 26, 2026

No description provided.

@github-project-automation github-project-automation bot moved this to PRs In Progress in Basic Kanban Board Jan 26, 2026
@snazy snazy force-pushed the guides-ci branch 20 times, most recently from a89d496 to 21f7718 Compare January 27, 2026 22:14
@snazy snazy mentioned this pull request Jan 27, 2026
Merged
6 tasks
@snazy snazy force-pushed the guides-ci branch 7 times, most recently from d89a242 to fdc26b8 Compare January 28, 2026 13:34
snazy added a commit to snazy/polaris that referenced this pull request Jan 29, 2026
@snazy
Guides: fix setup scripts to yield correct exit code
4c23076 
The statements in the shell scripts for the setup services are often concatenated using `;`, which means that a previous' command exit code is _not_ propagated and the service, although it failed, is determined to be successful.

This change updates those scripts to use `&&` for the statement concatenation.

"Final" setup services (aka "polaris-setup") now have a final `sleep 120`. This is due to the behavior of `docker compose up --detach --wait`, which considers _any_ service (without dependants) that exits with exit code 0 as a failure, leading to that docker-compose command yielding an error code. That would break the guides testing code (apache#3553). That `sleep 120` in "polaris-setup" services does **not** cause a delay of the compose starting up - it is purely a "hack around" Docker Compose not having a notion of "setup services".

To avoid merge conflicts, this change also:
* updates affected `curl` invocations (as apache#3610)
* removes superfluous `restart: "no"`
@snazy snazy mentioned this pull request Jan 29, 2026
Merged
snazy added a commit to snazy/polaris that referenced this pull request Jan 29, 2026
@snazy
Guides: fix setup scripts to yield correct exit code
3645bae 
The statements in the shell scripts for the setup services are often concatenated using `;`, which means that a previous' command exit code is _not_ propagated and the service, although it failed, is determined to be successful.

This change updates those scripts to use `&&` for the statement concatenation.

"Final" setup services (aka "polaris-setup") now have a final `sleep 120`. This is due to the behavior of `docker compose up --detach --wait`, which considers _any_ service (without dependants) that exits with exit code 0 as a failure, leading to that docker-compose command yielding an error code. That would break the guides testing code (apache#3553). That `sleep 120` in "polaris-setup" services does **not** cause a delay of the compose starting up - it is purely a "hack around" Docker Compose not having a notion of "setup services".

To avoid merge conflicts, this change also:
* updates affected `curl` invocations (as apache#3610)
* removes superfluous `restart: "no"`
snazy added a commit to snazy/polaris that referenced this pull request Jan 29, 2026
@snazy
Guides: fix setup scripts to yield correct exit code
b44ad81 
The statements in the shell scripts for the setup services are often concatenated using `;`, which means that a previous' command exit code is _not_ propagated and the service, although it failed, is determined to be successful.

This change updates those scripts to use `&&` for the statement concatenation.

"Final" setup services (aka "polaris-setup") now have a final `sleep 120`. This is due to the behavior of `docker compose up --detach --wait`, which considers _any_ service (without dependants) that exits with exit code 0 as a failure, leading to that docker-compose command yielding an error code. That would break the guides testing code (apache#3553). That `sleep 120` in "polaris-setup" services does **not** cause a delay of the compose starting up - it is purely a "hack around" Docker Compose not having a notion of "setup services".

To avoid merge conflicts, this change also:
* updates affected `curl` invocations (as apache#3610)
* removes superfluous `restart: "no"`
snazy added a commit to snazy/polaris that referenced this pull request Jan 29, 2026
@snazy
Guides: fix setup scripts to yield correct exit code
2368303 
The statements in the shell scripts for the setup services are often concatenated using `;`, which means that a previous' command exit code is _not_ propagated and the service, although it failed, is determined to be successful.

This change updates those scripts to use `&&` for the statement concatenation.

"Final" setup services (aka "polaris-setup") now have a final `sleep 120`. This is due to the behavior of `docker compose up --detach --wait`, which considers _any_ service (without dependants) that exits with exit code 0 as a failure, leading to that docker-compose command yielding an error code. That would break the guides testing code (apache#3553). That `sleep 120` in "polaris-setup" services does **not** cause a delay of the compose starting up - it is purely a "hack around" Docker Compose not having a notion of "setup services".

To avoid merge conflicts, this change also:
* updates affected `curl` invocations (as apache#3610)
* removes superfluous `restart: "no"`
snazy added a commit to snazy/polaris that referenced this pull request Jan 30, 2026
@snazy
Guides: fix setup scripts to yield correct exit code
2040cb5 
The statements in the shell scripts for the setup services are often concatenated using `;`, which means that a previous' command exit code is _not_ propagated and the service, although it failed, is determined to be successful.

This change updates those scripts to use `&&` for the statement concatenation.

"Final" setup services (aka "polaris-setup") now have a final `sleep 120`. This is due to the behavior of `docker compose up --detach --wait`, which considers _any_ service (without dependants) that exits with exit code 0 as a failure, leading to that docker-compose command yielding an error code. That would break the guides testing code (apache#3553). That `sleep 120` in "polaris-setup" services does **not** cause a delay of the compose starting up - it is purely a "hack around" Docker Compose not having a notion of "setup services".

To avoid merge conflicts, this change also:
* updates affected `curl` invocations (as apache#3610)
* removes superfluous `restart: "no"`
snazy added a commit to snazy/polaris that referenced this pull request Jan 31, 2026
@snazy
Guides: fix setup scripts to yield correct exit code
48e4bf0 
The statements in the shell scripts for the setup services are often concatenated using `;`, which means that a previous' command exit code is _not_ propagated and the service, although it failed, is determined to be successful.

This change updates those scripts to use `&&` for the statement concatenation.

"Final" setup services (aka "polaris-setup") now have a final `sleep 120`. This is due to the behavior of `docker compose up --detach --wait`, which considers _any_ service (without dependants) that exits with exit code 0 as a failure, leading to that docker-compose command yielding an error code. That would break the guides testing code (apache#3553). That `sleep 120` in "polaris-setup" services does **not** cause a delay of the compose starting up - it is purely a "hack around" Docker Compose not having a notion of "setup services".

To avoid merge conflicts, this change also:
* updates affected `curl` invocations (as apache#3610)
* removes superfluous `restart: "no"`
snazy added a commit to snazy/polaris that referenced this pull request Jan 31, 2026
@snazy
Guides: fix setup scripts to yield correct exit code
6603e1a 
The statements in the shell scripts for the setup services are often concatenated using `;`, which means that a previous' command exit code is _not_ propagated and the service, although it failed, is determined to be successful.

This change updates those scripts to use `&&` for the statement concatenation.

"Final" setup services (aka "polaris-setup") now have a final `sleep 120`. This is due to the behavior of `docker compose up --detach --wait`, which considers _any_ service (without dependants) that exits with exit code 0 as a failure, leading to that docker-compose command yielding an error code. That would break the guides testing code (apache#3553). That `sleep 120` in "polaris-setup" services does **not** cause a delay of the compose starting up - it is purely a "hack around" Docker Compose not having a notion of "setup services".

To avoid merge conflicts, this change also:
* updates affected `curl` invocations (as apache#3610)
* removes superfluous `restart: "no"`
@snazy snazy force-pushed the guides-ci branch 3 times, most recently from 8e9d08b to 2c4b065 Compare February 2, 2026 12:15
snazy added a commit to snazy/polaris that referenced this pull request Feb 2, 2026
@snazy
Guides: fix setup scripts to yield correct exit code
17084a5 
The statements in the shell scripts for the setup services are often concatenated using `;`, which means that a previous' command exit code is _not_ propagated and the service, although it failed, is determined to be successful.

This change updates those scripts to use `&&` for the statement concatenation.

"Final" setup services (aka "polaris-setup") now have a final `sleep 120`. This is due to the behavior of `docker compose up --detach --wait`, which considers _any_ service (without dependants) that exits with exit code 0 as a failure, leading to that docker-compose command yielding an error code. That would break the guides testing code (apache#3553). That `sleep 120` in "polaris-setup" services does **not** cause a delay of the compose starting up - it is purely a "hack around" Docker Compose not having a notion of "setup services".

To avoid merge conflicts, this change also:
* updates affected `curl` invocations (as apache#3610)
* removes superfluous `restart: "no"`
snazy added a commit to snazy/polaris that referenced this pull request Feb 2, 2026
@snazy
Guides: fix setup scripts to yield correct exit code
f507134 
The statements in the shell scripts for the setup services are often concatenated using `;`, which means that a previous' command exit code is _not_ propagated and the service, although it failed, is determined to be successful.

This change updates those scripts to use `&&` for the statement concatenation.

"Final" setup services (aka "polaris-setup") now have a final `sleep 120`. This is due to the behavior of `docker compose up --detach --wait`, which considers _any_ service (without dependants) that exits with exit code 0 as a failure, leading to that docker-compose command yielding an error code. That would break the guides testing code (apache#3553). That `sleep 120` in "polaris-setup" services does **not** cause a delay of the compose starting up - it is purely a "hack around" Docker Compose not having a notion of "setup services".

To avoid merge conflicts, this change also:
* updates affected `curl` invocations (as apache#3610)
* removes superfluous `restart: "no"`
@snazy snazy added the documentation Improvements or additions to documentation, especially web site content label Feb 2, 2026
snazy added a commit that referenced this pull request Feb 3, 2026
@snazy
Guides: fix setup scripts to yield correct exit code (#3612)
890b33a 
The statements in the shell scripts for the setup services are often concatenated using `;`, which means that a previous' command exit code is _not_ propagated and the service, although it failed, is determined to be successful.

This change updates those scripts to use `&&` for the statement concatenation.

"Final" setup services (aka "polaris-setup") now have a final `sleep 120`. This is due to the behavior of `docker compose up --detach --wait`, which considers _any_ service (without dependants) that exits with exit code 0 as a failure, leading to that docker-compose command yielding an error code. That would break the guides testing code (#3553). That `sleep 120` in "polaris-setup" services does **not** cause a delay of the compose starting up - it is purely a "hack around" Docker Compose not having a notion of "setup services".

To avoid merge conflicts, this change also:
* updates affected `curl` invocations (as #3610)
* removes superfluous `restart: "no"`
@snazy snazy force-pushed the guides-ci branch 3 times, most recently from 6716d92 to fcb57c2 Compare February 4, 2026 09:14
sungwy pushed a commit to sungwy/polaris that referenced this pull request Feb 7, 2026
@snazy @sungwy
Guides: fix setup scripts to yield correct exit code (apache#3612)
77e21d7 
The statements in the shell scripts for the setup services are often concatenated using `;`, which means that a previous' command exit code is _not_ propagated and the service, although it failed, is determined to be successful.

This change updates those scripts to use `&&` for the statement concatenation.

"Final" setup services (aka "polaris-setup") now have a final `sleep 120`. This is due to the behavior of `docker compose up --detach --wait`, which considers _any_ service (without dependants) that exits with exit code 0 as a failure, leading to that docker-compose command yielding an error code. That would break the guides testing code (apache#3553). That `sleep 120` in "polaris-setup" services does **not** cause a delay of the compose starting up - it is purely a "hack around" Docker Compose not having a notion of "setup services".

To avoid merge conflicts, this change also:
* updates affected `curl` invocations (as apache#3610)
* removes superfluous `restart: "no"`
@snazy snazy mentioned this pull request Feb 9, 2026
Open
snazy added a commit to snazy/polaris that referenced this pull request Feb 11, 2026
@snazy @evindj
@dimas-b @adutra @renovate-bot @nssalian @jbonofre @yushesp @carc-prathyush-shankar @RussellSpitzer
Dremio merge 2026 02 05 11 51 (apache#191)
bd34cc8 
* Releasey: adjust workflwo for Apache org level secrets (apache#3647)

See reference [INFRA-27430](https://issues.apache.org/jira/browse/INFRA-27430), requiring us to use `DOCKERHUB_USER` + `DOCKERHUB_TOKEN` instead of `DOCKERHUB_USERNAME` + `DOCKERHUB_TOKEN`.

* Guides: fix setup scripts to yield correct exit code (apache#3612)

The statements in the shell scripts for the setup services are often concatenated using `;`, which means that a previous' command exit code is _not_ propagated and the service, although it failed, is determined to be successful.

This change updates those scripts to use `&&` for the statement concatenation.

"Final" setup services (aka "polaris-setup") now have a final `sleep 120`. This is due to the behavior of `docker compose up --detach --wait`, which considers _any_ service (without dependants) that exits with exit code 0 as a failure, leading to that docker-compose command yielding an error code. That would break the guides testing code (apache#3553). That `sleep 120` in "polaris-setup" services does **not** cause a delay of the compose starting up - it is purely a "hack around" Docker Compose not having a notion of "setup services".

To avoid merge conflicts, this change also:
* updates affected `curl` invocations (as apache#3610)
* removes superfluous `restart: "no"`

* Add PolarisEventType int codes and remove unused before/after commit view/table values. (apache#3608)

This is a continuation of apache#3418 where we agreed we should remove associate enums values from Enum definition.

The code also adds a constructor to help not having to rely on ordinals() to to have enum codes.

I incremented enum code assignments by 100 based on categories. I am happy to take feedback here.

Does not change logic and removes only enums that are no longer used so behavior does not change.

* docs: Add quick guide for downstream builds (apache#3601)

* docs: Add quick guide for downstream builds

* IntegrationTestsHelper: fix extract & merge logic (apache#3650)

Both methods were flawed:

* `extractFromAnnotatedElements` wasn't properly delegating to the class if the method isn't annotated
* `mergeFromAnnotatedElements` wasn't properly prioritizing method properties over class properties.

* Community page: Yong Zheng - PPMC Member (apache#3653)

* chore(deps): update actions/checkout digest to de0fac2 (apache#3652)

* Use `quarkus.package.jar.type` (apache#3644)

Switch to `quarkus.package.jar.type` instead of the old `quarkus.package.type` build property as suggested by Quarkus build warning:

```
2026-02-03T00:26:05.672955189Z WorkerExecutor Queue WARN Configuration property 'quarkus.package.type' has been deprecated and replaced by: [quarkus.package.jar.enabled, quarkus.package.jar.type, quarkus.native.enabled, quarkus.native.sources-only]
```

* Site: Add blog post for Floe Polaris Integration (apache#3645)

* "Stale" job: restrict executions and adjust issue permissions (apache#3636)

* Add copyright on website (apache#3659)

* Sanitize principal names in AWS STS role session names (apache#3525)

Principal names containing invalid characters (spaces, parentheses,
etc.) were causing AWS STS AssumeRole requests to fail with validation
errors. AWS STS role session names must match the pattern [\w+=,.@-]*.

This change:
- Adds AwsRoleSessionNameSanitizer utility class to sanitize strings
  for use as AWS STS role session names
- Replaces invalid characters with underscores and truncates to 64
  characters (AWS maximum)
- Updates AwsCredentialsStorageIntegration to sanitize principal names
  when INCLUDE_PRINCIPAL_NAME_IN_SUBSCOPED_CREDENTIAL is enabled
- Adds tests to verify sanitization behavior and AWS pattern compliance

Fixes issue where principal names like "Joe (local)" would produce
invalid role session names like "polaris-Joe (local)" and cause
AssumeRole to fail. Now sanitized to "polaris-Joe__local_".

Co-authored-by: carc-prathyush-shankar <prathyush.shankar@carbonarc.co>

* Site: Adds the copyright message to all site pages (apache#3661)

* Site: Change Security link to local security reporting page (apache#3662)

* fix(deps): update dependency org.mongodb:mongodb-driver-sync to v5.6.3 (apache#3654)

* Releasey: use Apache org Nexus credentials (apache#3651)

* Releasey: use the correct SVN credentials (apache#3648)

* CI: Prerequisite PR for apache#3625 (apache#3646)

This change is only needed to update the required-checks to be able to eventually merge apache#3625.

* CI: all-in-one workflow (apache#3625)

This change moves all CI jobs into a single workflow. A single workflow comes with a couple advantages:
* all jobs are visible on one page
* option to re-run all failed jobs at once

The refactoring also simplifies the `.asf.yaml` file by referencing a single required check that can only succeeds if dependent jobs were successful.

The actual CI jobs are in the `ci.yml` file, which is only triggered via a `workflow_call` event.
There are two workflows that call `ci.yml`:
* `ci-main.yml` for `main` and `release/*` branches, with a concurrency group that does not cancel already running workflows
* `ci-pr.yml` for PRs with a concurrency group that cancels previous CI runs

The names of these two calling workflows include information that enrich the workflow view, and the workflows for main, release branches and PRs are grouped separartely in the GH Actions page for the repository. In other words, the reference name (for main + release branches) or the PR number and title are shown in the workflow runs list on the GH Actions page.

* Fix CI required checks (apache#3666)

* Add Sung as committer (apache#3665)

* Fix `CatalogFederationIntegrationTest.testFederatedCatalogWithCredentialVending()` for AWSSDK update (apache#3664)

Recent AWSSDK versions introduce `software.amazon.awssdk.services.s3.model.AccessDeniedException`, hence the assertion on `S3Exception` fails.

* CI/main: re-add commit message to `run_name` (apache#3668)

The default `run_name` value is, in case of `push` events, the commit message. This change re-adds the commit message.

* Add CI workflow to test against Iceberg unreleased versions (apache#3630)

* Update dependency software.amazon.awssdk:bom to v2.41.21 (apache#3639)

* Update actions/checkout digest to de0fac2 (apache#3671)

* Nit: Fix wrong `Nullable` import (apache#3672)

* Explicitly set build-time property `quarkus.datasource.db-kind` (apache#3674)

The property is set for the Polaris admin tool, but not for Polaris server. This causes a startup error with Quarkus 3.31.

Error message:
```
ERROR: Failed to start application
java.lang.RuntimeException: Failed to start quarkus
	at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
	at io.quarkus.runtime.Application.start(Application.java:116)
	at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:119)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:79)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:50)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:143)
	at io.quarkus.runner.GeneratedMain.main(Unknown Source)
	at io.quarkus.bootstrap.runner.QuarkusEntryPoint.doRun(QuarkusEntryPoint.java:86)
	at io.quarkus.bootstrap.runner.QuarkusEntryPoint.main(QuarkusEntryPoint.java:37)
Caused by: java.lang.IllegalStateException: Build time property cannot be changed at runtime:
 - quarkus.datasource.db-kind is set to 'postgresql' but it is build time fixed to 'null'. Did you change the property quarkus.datasource.db-kind after building the application?
	at io.quarkus.runtime.configuration.ConfigRecorder.handleConfigChange(ConfigRecorder.java:72)
	at io.quarkus.runner.recorded.ConfigGenerationBuildStep$checkForBuildTimeConfigChange1532146938.deploy_6(Unknown Source)
	at io.quarkus.runner.recorded.ConfigGenerationBuildStep$checkForBuildTimeConfigChange1532146938.deploy(Unknown Source)
	... 9 more
```

* Last merged commit 2651e06

---------

Co-authored-by: Innocent Djiofack <djiofack007@gmail.com>
Co-authored-by: Dmitri Bourlatchkov <dmitri.bourlatchkov@gmail.com>
Co-authored-by: Alexandre Dutra <adutra@apache.org>
Co-authored-by: Mend Renovate <bot@renovateapp.com>
Co-authored-by: Neelesh Salian <nssalian@users.noreply.github.com>
Co-authored-by: JB Onofré <jbonofre@apache.org>
Co-authored-by: Prathyush Shankar <prathyush2018@gmail.com>
Co-authored-by: carc-prathyush-shankar <prathyush.shankar@carbonarc.co>
Co-authored-by: Russell Spitzer <russell.spitzer@GMAIL.COM>
snazy added 2 commits February 12, 2026 10:21
@snazy
[Site] Add getting-started guides as "Guides" to the web site
92ade02 
This change follows up on the [dev-mailing list discussion](https://lists.apache.org/thread/gwxn34nzvhdqdjw2bst6kwlqt8jhb91d) and adds the guides under `getting-started/` to the web site as "Guides".

* The guides had to me moved from the `getting-started/` folder into the `site/content/` folder structure, because symlinks from the `site/content/` _folder_ do not work (the contents are not pulled in).
* Each guide's `README.md` file has been renamed to `index.md`, added a front-matter, which instructs Hugo to that also lets Hugo include the guide in the top-level menu.
* Assets for each guide are listed at the top of each guide, so users do not have to go via GitHub.
* Links in the guides have been updated.
* A new layout named `guides` has been added for this use case.

Two changes on the Ceph guide:
* updated the misplaced `yaml` type on a code snippet
* renamed `.env.example` to `dot-env.example`, because files starting with a `.` are not included
@snazy
WIP/EXPERIMENT: CI for guides
57f2dbb
@snazy snazy changed the title WIP/EXPERIMENT: CI for guides WIP: CI for guides Feb 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

documentation Improvements or additions to documentation, especially web site content

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@snazy