Merge branch 'master' into trino-plugin-pulsar-sql

trinodb · Oct 2, 2024 · 7379bad · 7379bad
2 parents 3091405 + 7ba7299
commit 7379bad
Show file tree

Hide file tree

Showing 1,347 changed files with 28,381 additions and 37,584 deletions.
diff --git a/.github/DEVELOPMENT.md b/.github/DEVELOPMENT.md
@@ -6,47 +6,11 @@ which covers thing like development philosophy and contribution process.
 More information about the writing and building the documentation can
 be found in the [docs module](../docs).
 
-* [Commits and pull requests](#commits-and-pull-requests)
 * [Code style](#code-style)
 * [Additional IDE configuration](#additional-ide-configuration)
 * [Building the Web UI](#building-the-web-ui)
 * [CI pipeline](#ci-pipeline)
 
-## Commits and pull requests
-
-### Format Git commit messages
-
-When writing a Git commit message, follow these [guidelines](https://chris.beams.io/posts/git-commit/).
-
-### Git merge strategy
-
-Pull requests are usually merged into `master` using the  [`rebase and merge`](https://docs.github.com/en/github/collaborating-with-pull-requests/incorporating-changes-from-a-pull-request/about-pull-request-merges#rebase-and-merge-your-pull-request-commits) strategy.
-
-A typical pull request should strive to contain a single logical change (but not
-necessarily a single commit). Unrelated changes should generally be extracted
-into their own PRs.
-
-If a pull request contains a stack of more than one commit, then
-popping any number of commits from the top of the stack, should not
-break the PR, ie. every commit should build and pass all tests.
-
-Commit messages and history are important as well, because they are
-used by other developers to keep track of the motivation behind
-changes. Keep logical diffs grouped together in separate commits and
-order commits in a way that explains by itself the evolution of the
-change. Rewriting and reordering commits is a natural part of the
-review process. Mechanical changes like refactoring, renaming, removing
-duplication, extracting helper methods, static imports should be kept
-separated from logical and functional changes like adding a new feature
-or modifying code behaviour. This makes reviewing the code much easier
-and reduces the chance of introducing unintended changes in behavior.
-
-Whenever in doubt on splitting a change into a separate commit, ask
-yourself the following question: if all other work in the PR needs to
-be reverted after merging to master for some objective reason (eg. a
-bug has been discovered), is it worth keeping that commit still in
-master.
-
 ## Code Style
 
 We recommend you use IntelliJ as your IDE. The code style template for the
@@ -262,16 +226,22 @@ Trino aims for frequent releases, generally once per week. This is a goal but
 not a guarantee, as critical bugs may lead to a release being pushed back or
 require an extra emergency release to patch the issue.
 
-At the start of each release cycle, a GitHub issue is filed and pinned to track
-all necessary release notes. For example, see [the issue for Trino 395](https://github.com/trinodb/trino/issues/13913).
-In addition, a release notes pull request is updated and maintained throughout
-the week, tracking all merged commits to ensure every change is properly
-documented and noted. This uses the [release note template](../docs/release-template.md),
-with changes in each section arranged to have new features first, performance
-improvements second, and bugfixes third. See [the release notes for 395](https://github.com/trinodb/trino/pull/13975)
-as an example.
-
-Once it is time to release, the release process is kicked off. A code freeze is
-announced on the Trino Slack in the #releases channel, and then a maintainer
-utilizes the [release scripts](https://github.com/trinodb/release-scripts) to
-update Trino to the next version.
+At the start of each release cycle, a release notes pull request (PR) is started
+and maintained throughout the week, tracking all merged PRs to ensure every
+change is properly documented and noted.
+
+The PR uses the [release note template](../docs/release-template.md) and follows
+the [release notes
+guidelines](https://trino.io/development/process#release-note) to use and
+improve the proposed release note entries from the merged PRs. When necessary,
+documentation and clarification for the release notes entries is requested from
+the merging maintainer and the contributor.
+
+See [the release notes for
+455](https://github.com/trinodb/trino/pull/23096) as an example.
+
+Once it is time to release, the release notes PR is merged and the process is
+kicked off. A code freeze is announced on the Trino Slack in the #releases
+channel, and then a maintainer utilizes the [release
+scripts](https://github.com/trinodb/release-scripts) to update Trino to the next
+version.
diff --git a/.github/actions/setup/action.yml b/.github/actions/setup/action.yml
@@ -42,6 +42,35 @@ runs:
         with:
           distribution: 'temurin' # use same JDK distro as in Trino docker images
           java-version: ${{ inputs.java-version }}
+      - name: Get NodeJS version from parent pom
+        shell: bash
+        run: |
+          node_version=$(grep -m 1 '<dep.frontend-node.version>' "pom.xml" | sed -n 's/.*<dep.frontend-node.version>\(.*\)<\/dep.frontend-node.version>.*/\1/p')
+          if [ -n "$node_version" ]; then
+            echo "node version is $node_version"
+            echo "NODE_VERSION=$node_version" >>${GITHUB_ENV}
+            exit 0
+          fi
+      - name: Cache and Restore local NodeJS
+        id: cache-nodejs
+        if: ${{ format('{0}', inputs.cache) == 'true' && env.NODE_VERSION != '' }}
+        uses: actions/cache@v4
+        with:
+          # Note: must be same set of paths as for cache:restore mode
+          path: .node
+          key: ${{ runner.os }}-nodejs-${{ env.NODE_VERSION }}
+          restore-keys: |
+            ${{ runner.os }}-nodejs-
+      - name: Restore local NodeJS
+        id: cache_restore-nodejs
+        if: ${{ format('{0}', inputs.cache) == 'restore' && env.NODE_VERSION != '' }}
+        uses: actions/cache/restore@v4
+        with:
+          # Note: must be same set of paths as for cache:true mode
+          path: .node
+          key: ${{ runner.os }}-nodejs-${{ env.NODE_VERSION }}
+          restore-keys: |
+            ${{ runner.os }}-nodejs-
       - name: Cache and Restore local Maven repo
         id: cache
         if: ${{ format('{0}', inputs.cache) == 'true' }}

diff --git a/.github/bin/free-disk-space.sh b/.github/bin/free-disk-space.sh
@@ -3,42 +3,31 @@ set -euo pipefail
 
 function list_installed_packages()
 {
-    apt list --installed "$1" 2>/dev/null | awk -F'/' 'NR>1{print $1}' | tr '\n' ' '
+    apt list --installed "$1" 2>/dev/null | awk -F'/' 'NR>1{print $1}'
 }
 
 function free_up_disk_space_ubuntu()
 {
     local packages=(
         'azure-cli'
         'aspnetcore-*'
-        'dotnet-*'
         'firefox*'
         'google-chrome-*'
         'google-cloud-*'
         'libmono-*'
         'llvm-*'
-        'imagemagick'
-        'postgresql-*'
-        'rubu-*'
-        'spinxsearch'
-        'unixodbc-dev'
-        'mercurial'
-        'esl-erlang'
-        'microsoft-edge-stable'
-        'mono-*'
-        'msbuild'
         'mysql-server-core-*'
-        'php-*'
-        'php7*'
         'powershell*'
-        'mongo*'
-        'microsoft-edge*'
-        'subversion')
+        'microsoft-edge*')
 
     for package in "${packages[@]}"; do
-        installed_packages=$(list_installed_packages "${package}")
-        echo "Removing packages by pattern ${package}: ${installed_packages}"
-        sudo apt-get --auto-remove -y purge ${installed_packages}
+        mapfile -t installed_packages < <(list_installed_packages "${package}")
+        if [ ${#installed_packages[@]} -eq 0 ]; then
+            echo "No packages matched by pattern ${package}"
+        else
+            echo "Removing packages by pattern ${package}: ${installed_packages[*]}"
+            sudo apt-get --auto-remove -y purge "${installed_packages[@]}"
+        fi
     done
 
     echo "Autoremoving packages"
@@ -60,11 +49,14 @@ function free_up_disk_space_ubuntu()
       sudo docker system prune --all -f
 }
 
+
 echo "Disk space usage before cleaning:"
 df -k .
 
-echo "Clearing up disk usage:"
+echo "::group::Clearing up disk usage"
 free_up_disk_space_ubuntu
+echo "::endgroup::"
 
 echo "Disk space usage after cleaning:"
 df -k .
+
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
@@ -1,22 +1,26 @@
-<!-- Thank you for submitting a pull request! Find more information in our development guide at https://github.com/trinodb/trino/blob/master/.github/DEVELOPMENT.md and contact us on #dev in Slack. -->
-<!-- Provide an overview of the PR for maintainers and reviewers. -->
+<!-- Thank you for submitting a pull request! Find more information 
+at https://trino.io/development/process.html, 
+at https://github.com/trinodb/trino/blob/master/.github/DEVELOPMENT.md 
+and contact us on #core-dev in Slack. -->
+<!-- Provide an overview for maintainers and reviewers. -->
 ## Description
 
 
 
-<!-- Provide details that would help an engineer who is unfamiliar with this part of the code. -->
+<!-- Provide details that help an engineer who is unfamiliar with this part of the code. -->
 ## Additional context and related issues
 
 
 
-<!-- Mark the appropriate option with an (x). Propose a release note if you can. -->
+<!-- Mark the appropriate option with an (x). Propose a release note if you can.
+More info at https://trino.io/development/process#release-note -->
 ## Release notes
 
-( ) This is not user-visible or is docs only, and no release notes are required.
-( ) Release notes are required. Please propose a release note for me.
-( ) Release notes are required, with the following suggested text:
+- [ ] This is not user-visible or is docs only, and no release notes are required.
+- [ ] Release notes are required. Please propose a release note for me.
+- [ ] Release notes are required, with the following suggested text:
 
 ```markdown
-# Section
+## Section
 * Fix some things. ({issue}`issuenumber`)
 ```
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -57,7 +57,7 @@ jobs:
       fail-fast: false
       matrix:
         include:
-          - { java-version: 23, cache: 'restore', cleanup-node: true }
+          - { java-version: 23, cache: 'true', cleanup-node: true }
           - { java-version: 24-ea, cache: 'restore', cleanup-node: true }
     timeout-minutes: 45
     steps:
@@ -370,7 +370,6 @@ jobs:
             !:trino-phoenix5,
             !:trino-pinot,
             !:trino-postgresql,
-            !:trino-raptor-legacy,
             !:trino-redis,
             !:trino-redshift,
             !:trino-resource-group-managers,
@@ -476,7 +475,6 @@ jobs:
             - { modules: plugin/trino-phoenix5 }
             - { modules: plugin/trino-pinot }
             - { modules: plugin/trino-postgresql }
-            - { modules: plugin/trino-raptor-legacy }
             - { modules: plugin/trino-redis }
             - { modules: plugin/trino-redshift }
             - { modules: plugin/trino-redshift, profile: cloud-tests }
@@ -881,6 +879,8 @@ jobs:
             - suite-delta-lake-databricks113
             - suite-delta-lake-databricks122
             - suite-delta-lake-databricks133
+            - suite-delta-lake-databricks143
+            - suite-delta-lake-databricks154
             - suite-databricks-unity-http-hms
             - suite-gcs
             - suite-clients
@@ -929,6 +929,12 @@ jobs:
             - suite: suite-delta-lake-databricks133
               ignore exclusion if: >-
                 ${{ env.CI_SKIP_SECRETS_PRESENCE_CHECKS != '' || secrets.DATABRICKS_TOKEN != '' }}
+            - suite: suite-delta-lake-databricks143
+              ignore exclusion if: >-
+                ${{ env.CI_SKIP_SECRETS_PRESENCE_CHECKS != '' || secrets.DATABRICKS_TOKEN != '' }}
+            - suite: suite-delta-lake-databricks154
+              ignore exclusion if: >-
+                ${{ env.CI_SKIP_SECRETS_PRESENCE_CHECKS != '' || secrets.DATABRICKS_TOKEN != '' }}
             - suite: suite-databricks-unity-http-hms
               config: hdp3
             - suite: suite-databricks-unity-http-hms
@@ -987,6 +993,8 @@ jobs:
           DATABRICKS_113_JDBC_URL: ""
           DATABRICKS_122_JDBC_URL: ""
           DATABRICKS_133_JDBC_URL: ""
+          DATABRICKS_143_JDBC_URL: ""
+          DATABRICKS_154_JDBC_URL: ""
           DATABRICKS_UNITY_JDBC_URL: ""
           DATABRICKS_UNITY_CATALOG_NAME: ""
           DATABRICKS_UNITY_EXTERNAL_LOCATION: ""
@@ -1063,6 +1071,8 @@ jobs:
           DATABRICKS_113_JDBC_URL: ${{ vars.DATABRICKS_113_JDBC_URL }}
           DATABRICKS_122_JDBC_URL: ${{ vars.DATABRICKS_122_JDBC_URL }}
           DATABRICKS_133_JDBC_URL: ${{ vars.DATABRICKS_133_JDBC_URL }}
+          DATABRICKS_143_JDBC_URL: ${{ vars.DATABRICKS_143_JDBC_URL }}
+          DATABRICKS_154_JDBC_URL: ${{ vars.DATABRICKS_154_JDBC_URL }}
           DATABRICKS_UNITY_JDBC_URL: ${{ vars.DATABRICKS_UNITY_JDBC_URL }}
           DATABRICKS_UNITY_CATALOG_NAME: ${{ vars.DATABRICKS_UNITY_CATALOG_NAME }}
           DATABRICKS_UNITY_EXTERNAL_LOCATION: ${{ vars.DATABRICKS_UNITY_EXTERNAL_LOCATION }}

diff --git a/.gitignore b/.gitignore
@@ -33,3 +33,4 @@ product-test-reports
 .github/bin/redshift/.cluster-identifier
 **/dependency-reduced-pom.xml
 core/trino-web-ui/src/main/resources/webapp/dist/
+.node
diff --git a/.mvn/maven.config b/.mvn/maven.config
@@ -10,3 +10,4 @@
 -b
 smart
 -Dair.main.basedir=${session.rootDirectory}
+-Dnode.tmpdir=${session.rootDirectory}/.node/
diff --git a/client/trino-cli/pom.xml b/client/trino-cli/pom.xml
@@ -5,7 +5,7 @@
     <parent>
         <groupId>io.trino</groupId>
         <artifactId>trino-root</artifactId>
-        <version>459-SNAPSHOT</version>
+        <version>460-SNAPSHOT</version>
         <relativePath>../../pom.xml</relativePath>
     </parent>
 

diff --git a/client/trino-cli/src/test/java/io/trino/cli/TestQueryRunner.java b/client/trino-cli/src/test/java/io/trino/cli/TestQueryRunner.java
@@ -115,7 +115,7 @@ static ClientSession createClientSession(MockWebServer server)
     {
         return ClientSession.builder()
                 .server(server.url("/").uri())
-                .principal(Optional.of("user"))
+                .user(Optional.of("user"))
                 .source("source")
                 .clientInfo("clientInfo")
                 .catalog("catalog")

diff --git a/client/trino-client/pom.xml b/client/trino-client/pom.xml
@@ -5,7 +5,7 @@
     <parent>
         <groupId>io.trino</groupId>
         <artifactId>trino-root</artifactId>
-        <version>459-SNAPSHOT</version>
+        <version>460-SNAPSHOT</version>
         <relativePath>../../pom.xml</relativePath>
     </parent>