Skip to content

Fix version-specific Spark profiles#25940

Merged
ghelmling merged 1 commit intoprestodb:masterfrom
ghelmling:fix_spark3_profile
Sep 4, 2025
Merged

Fix version-specific Spark profiles#25940
ghelmling merged 1 commit intoprestodb:masterfrom
ghelmling:fix_spark3_profile

Conversation

@ghelmling
Copy link
Copy Markdown
Contributor

Description

This reworks the presto-spark-classloader-spark2 and presto-spark-classloader-spark3 modules used for binding to version-specific Spark releases. This changes the way the modules are handled during the build by controlling their inclusion via Maven profiles:

  • spark2: (on by default, or when spark-version=2) includes the presto-spark-classloader-spark2 module, and sets the spark-core artifact to v2.0.2
  • spark3: (active when spark-version=3) includes the presto-spark-classloader-spark3 module, and sets the spark-core artifact to 3.4.1

Motivation and Context

A prior version of the spark3 profile was added in PR #25552. However, this controlled the module included by manipulating the module name, appending a version number at the end. Not having the full module name listed confused other plugins which analyze the project POM, like the versions plugin. As a result, running mvn versions:set was not able to update the version for the presto-spark-classloader-spark2 and presto-spark-classloader-spark3 modules. With this change mvn versions:set is able to update both module versions.

Impact

Test Plan

Built with both the spark2 and spark3 modules and verified that the correct presto-spark-classloader-sparkN dependency is included.

Contributor checklist

  • Please make sure your submission complies with our contributing guide, in particular code style and commit standards.
  • PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
  • Documented new properties (with its default value), SQL syntax, functions, or other functionality.
  • If release notes are required, they follow the release notes guidelines.
  • Adequate tests were added if applicable.
  • CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== NO RELEASE NOTE ==

@ghelmling ghelmling merged commit 9c5004f into prestodb:master Sep 4, 2025
68 of 69 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

from:Meta PR from Meta

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants