Skip to content

Commit a5fc89c

Browse files
pan3793dongjoon-hyun
authored andcommitted
[SPARK-54127][BUILD] Fix sbt inconsistent shading package
### What changes were proposed in this pull request? Fix sbt inconsistent shading package for `spark-connect` module ### Why are the changes needed? Take 4.1.0-preview3 as the example, which is built by Maven. ``` $ jar tf jars/spark-connect_2.13-4.1.0-preview3.jar | grep 'connect/grpc' | head org/sparkproject/connect/grpc/ org/sparkproject/connect/grpc/inprocess/ org/sparkproject/connect/grpc/inprocess/InProcessTransport$InProcessStream$InProcessServerStream.class org/sparkproject/connect/grpc/inprocess/InProcessTransport$2.class org/sparkproject/connect/grpc/inprocess/InProcessTransport$InProcessStream$InProcessClientStream.class org/sparkproject/connect/grpc/inprocess/InProcessTransport$SingleMessageProducer.class org/sparkproject/connect/grpc/inprocess/InProcessTransport$6.class org/sparkproject/connect/grpc/inprocess/InProcessTransport$1.class org/sparkproject/connect/grpc/inprocess/InProcessSocketAddress.class org/sparkproject/connect/grpc/inprocess/InProcessTransport$InProcessStream.class ``` the produced jar built by SBT has a different relocated package for grpc classes ``` $ jar tf jars/spark-connect_2.13-4.1.0-SNAPSHOT.jar | grep 'connect/grpc' | head org/sparkproject/connect/grpc/ org/sparkproject/connect/grpc/io/ org/sparkproject/connect/grpc/io/grpc/ org/sparkproject/connect/grpc/io/grpc/Attributes$1.class org/sparkproject/connect/grpc/io/grpc/Attributes$Builder.class org/sparkproject/connect/grpc/io/grpc/Attributes$Key.class org/sparkproject/connect/grpc/io/grpc/Attributes.class org/sparkproject/connect/grpc/io/grpc/BinaryLog.class org/sparkproject/connect/grpc/io/grpc/BindableService.class org/sparkproject/connect/grpc/io/grpc/CallCredentials$MetadataApplier.class ``` ### Does this PR introduce _any_ user-facing change? No, so far I don't see such inconsistent causes real issue. ### How was this patch tested? Verified the output of `dev/make-distribution.sh --sbt-enabled` ``` jar tf jars/spark-connect_2.13-4.1.0-SNAPSHOT.jar | grep 'connect/grpc' | head org/sparkproject/connect/grpc/ org/sparkproject/connect/grpc/Attributes$1.class org/sparkproject/connect/grpc/Attributes$Builder.class org/sparkproject/connect/grpc/Attributes$Key.class org/sparkproject/connect/grpc/Attributes.class org/sparkproject/connect/grpc/BinaryLog.class org/sparkproject/connect/grpc/BindableService.class org/sparkproject/connect/grpc/CallCredentials$MetadataApplier.class org/sparkproject/connect/grpc/CallCredentials$RequestInfo.class org/sparkproject/connect/grpc/CallCredentials.class ``` ### Was this patch authored or co-authored using generative AI tooling? No. Closes #52825 from pan3793/SPARK-54127. Authored-by: Cheng Pan <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 3d92bdf) Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent d160af3 commit a5fc89c

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

project/SparkBuild.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -808,7 +808,7 @@ object SparkConnect {
808808
},
809809

810810
(assembly / assemblyShadeRules) := Seq(
811-
ShadeRule.rename("io.grpc.**" -> "org.sparkproject.connect.grpc.@0").inAll,
811+
ShadeRule.rename("io.grpc.**" -> "org.sparkproject.connect.grpc.@1").inAll,
812812
ShadeRule.rename("com.google.common.**" -> "org.sparkproject.connect.guava.@1").inAll,
813813
ShadeRule.rename("com.google.thirdparty.**" -> "org.sparkproject.connect.guava.@1").inAll,
814814
ShadeRule.rename("com.google.protobuf.**" -> "org.sparkproject.connect.protobuf.@1").inAll,

0 commit comments

Comments
 (0)