Add notes in spark-3.5 advisories

spark-3.5-scala-2.12.advisories.yaml

@@ -21,6 +21,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/hadoop-client-runtime-3.3.6.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: This relates to nimbus-jose-jwt v9.8.1 included by the shaded JAR hadoop-client-runtime-3.3.6.jar. Spark is planning an upgrade to Hadoop 3.4.0 for Spark 4.0.0, but as of today, the shaded JAR for Hadoop 3.4.0 still includes this vulnerability.
 
   - id: CGA-2whx-g953-gpmc
     aliases:
@@ -39,6 +43,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/hadoop-client-runtime-3.3.6.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: commons-io v2.8.0 is a transitive dependency that is brought in under hadoop-client-runtime-3.3.6.jar. This requires a hadoop-client-runtime update from upstream maintainers
 
   - id: CGA-2x96-jhr3-824h
     aliases:
@@ -57,6 +65,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/hadoop-client-runtime-3.3.6.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: This requires other packages to be bumped and might break the build, waiting for upstream to update the dependencies.
 
   - id: CGA-3h6q-7rxp-58mp
     aliases:
@@ -75,6 +87,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/hadoop-shaded-guava-1.1.1.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: fix-not-planned
+        data:
+          note: This relates to guava v30.1.1-jre, which is included by the shaded JARs hadoop-shaded-guava-1.1.1.jar and hadoop-client-runtime-3.3.6.jar.
 
   - id: CGA-75v9-fc2q-898r
     aliases:
@@ -93,6 +109,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/mesos-1.4.3-shaded-protobuf.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: This relates to protobuf-java v3.3.0 included by the shaded JARs mesos-1.4.3-shaded-protobuf.jar and hadoop-client-runtime-3.3.6.jar. There are no newer versions of these shaded JARs available to fix the vulnerability.
 
   - id: CGA-8x25-m2vp-q84p
     aliases:
@@ -129,6 +149,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/mesos-1.4.3-shaded-protobuf.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: This relates to protobuf-java v3.3.0 included by the shaded JARs mesos-1.4.3-shaded-protobuf.jar and hadoop-client-runtime-3.3.6.jar. There are no newer versions of these shaded JARs available to fix the vulnerability.
 
   - id: CGA-c5jh-9f56-9q3j
     aliases:
@@ -147,6 +171,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/hadoop-client-runtime-3.3.6.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: Updating jetty to a non-vulnerable version would require 3 major version bumps, which would be a very significant upgrade with multiple breaking changes, and should only be undertaken by the upstream maintainers.
 
   - id: CGA-c83x-4wc2-v54h
     aliases:
@@ -183,6 +211,11 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/jackson-mapper-asl-1.9.13.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: false-positive-determination
+        data:
+          type: vulnerable-code-not-in-execution-path
+          note: This relates to jackson-mapper-asl, which is no longer maintained. Upstream have confirmed the libraries this CVE impacts are not used by Apache Spark. https://issues.apache.org/jira/browse/CASSANDRA-16056
 
   - id: CGA-cqpj-2pg7-9f9v
     aliases:
@@ -201,6 +234,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/hadoop-client-runtime-3.3.6.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: This relates to commons-compress 1.21 included by the shaded JARs hadoop-client-runtime-3.3.6.jar. There are no newer versions of the shaded JARs available to fix the vulnerability.
 
   - id: CGA-cr98-6286-9j39
     aliases:
@@ -219,6 +256,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/mesos-1.4.3-shaded-protobuf.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: This relates to protobuf-java v3.3.0 included by the shaded JARs mesos-1.4.3-shaded-protobuf.jar and hadoop-client-runtime-3.3.6.jar. There are no newer versions of these shaded JARs available to fix the vulnerability.
 
   - id: CGA-cwcj-754w-xm64
     aliases:
@@ -237,6 +278,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/libthrift-0.12.0.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: Spark v3.5.0 is incompatible with higher versions of libthrift. https://github.com/apache/spark/pull/34878
 
   - id: CGA-ffxr-hrxc-hfpm
     aliases:
@@ -255,6 +300,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/hadoop-client-runtime-3.3.6.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: This relates to commons-configuration2 2.8.0 included by the shaded JARs hadoop-client-runtime-3.3.6.jar. Spark is planning an upgrade to Hadoop 3.4.0 for Spark 4.0.0, but as of today, the shaded JAR for Hadoop 3.4.0 still includes this vulnerability.
 
   - id: CGA-g7h9-jx7c-7w3c
     aliases:
@@ -273,6 +322,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/libthrift-0.12.0.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: Spark v3.5.0 is incompatible with higher versions of libthrift. https://github.com/apache/spark/pull/34878
 
   - id: CGA-g972-4w58-jj5c
     aliases:
@@ -291,6 +344,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/hadoop-client-runtime-3.3.6.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: This relates to json-smart v1.3.2 included by the shaded JAR hadoop-client-runtime-3.3.6.jar. There are no newer versions of this shaded JAR available to fix the vulnerability.
 
   - id: CGA-gvxp-wjw6-3q9g
     aliases:
@@ -309,6 +366,11 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/netty-common-4.1.108.Final.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: false-positive-determination
+        data:
+          type: vulnerable-code-cannot-be-controlled-by-adversary
+          note: Vulnerability affects only Windows systems.
 
   - id: CGA-hfgh-8x66-8pq3
     aliases:
@@ -327,6 +389,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/hadoop-shaded-guava-1.1.1.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: This relates to guava v30.1.1-jre, which is included by the shaded JARs hadoop-shaded-guava-1.1.1.jar and hadoop-client-runtime-3.3.6.jar.
 
   - id: CGA-jgpv-2j8j-5mwv
     aliases:
@@ -345,6 +411,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/mesos-1.4.3-shaded-protobuf.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: This relates to protobuf-java v3.3.0 included by the shaded JARs mesos-1.4.3-shaded-protobuf.jar and hadoop-client-runtime-3.3.6.jar. There are no newer versions of these shaded JARs available to fix the vulnerability.
 
   - id: CGA-jvxv-jw4c-qmcg
     aliases:
@@ -363,6 +433,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/mesos-1.4.3-shaded-protobuf.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: This relates to mesos-1.4.3-shaded-protobuf, which is a shaded jar with no new upstream release.
 
   - id: CGA-jwf5-xmv5-8v4w
     aliases:
@@ -381,6 +455,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/hadoop-client-runtime-3.3.6.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: The commons-io dependency that exists in the spark-3.5 package and related subpackages is brought in as transitive from hadoop-client-runtime-3.3.6.jar. This dependency is not able to be upgraded to a higher version and requires upstream maintainers to implement.
 
   - id: CGA-mqf4-8v8m-5gcr
     aliases:
@@ -399,6 +477,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/hadoop-client-runtime-3.3.6.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: This relates to json-smart v1.3.2 included by the shaded JAR hadoop-client-runtime-3.3.6.jar. There are no newer versions of this shaded JAR available to fix the vulnerability.
 
   - id: CGA-pcrp-37wm-7pp6
     aliases:
@@ -417,6 +499,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/mesos-1.4.3-shaded-protobuf.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: This relates to protobuf-java v3.3.0 included by the shaded JARs mesos-1.4.3-shaded-protobuf.jar and hadoop-client-runtime-3.3.6.jar. There are no newer versions of these shaded JARs available to fix the vulnerability.
 
   - id: CGA-pj5x-465x-3ch4
     aliases:
@@ -435,6 +521,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/mesos-1.4.3-shaded-protobuf.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: This relates to protobuf-java v3.3.0 included by the shaded JARs mesos-1.4.3-shaded-protobuf.jar and hadoop-client-runtime-3.3.6.jar. There are no newer versions of these shaded JARs available to fix the vulnerability.
 
   - id: CGA-pqmx-9gfc-r76g
     aliases:
@@ -453,6 +543,11 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/derby-10.14.2.0.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: fix-not-planned
+        data:
+          note: |
+            This relates to 'derby',Spark-3.5 currently uses version 10.14.2.0, while the closest fixed version available in the Maven Central repository is 10.17.1.0. However, this version requires a minimum of Java 17 to build, whereas Spark-3.5 is built with Java 8 and 11 as well. Upgrading to 10.17.1.0 would cause a build break due to Java bytecode version incompatibility. At this time, we are not planning to upgrade the version of Derby in Spark-3.5. The upstream project has updated to version 10.16.1.1, which does not resolve the vulnerability. The upstream is currently waiting for a backport to Derby version 10.16.2.x which they have planed to fix in version spark-4 or later. For reference, see: https://github.com/apache/spark/pull/44174
 
   - id: CGA-r5px-mvhg-cw5m
     aliases:
@@ -471,6 +566,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/hadoop-client-runtime-3.3.6.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: This relates to commons-configuration2 2.8.0 included by the shaded JARs hadoop-client-runtime-3.3.6.jar. Spark is planning an upgrade to Hadoop 3.4.0 for Spark 4.0.0, but as of today, the shaded JAR for Hadoop 3.4.0 still includes this vulnerability.
 
   - id: CGA-r84w-h5xq-qhr6
     aliases:
@@ -489,6 +588,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/hadoop-client-runtime-3.3.6.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: This relates to commons-compress 1.21 included by the shaded JARs hadoop-client-runtime-3.3.6.jar. There are no newer versions of the shaded JARs available to fix the vulnerability.
 
   - id: CGA-rj77-p9x4-qgmq
     aliases:
@@ -525,6 +628,10 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/hadoop-client-api-3.3.6.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: pending-upstream-fix
+        data:
+          note: 'The changes required to implement an upgrade from hadoop 3.3.6 to hadoop 3.4.0 require core code changes which are set to be released as a part of the spark 4.0.0 release that is in preview now. PR can be found here: https://github.com/apache/spark/commit/49b4c3bc9c09325de941dfaf41e4fd3a4a4c345f '
 
   - id: CGA-xmgm-rjh2-22q4
     aliases:
@@ -543,3 +650,7 @@ advisories:
             componentType: java-archive
             componentLocation: /usr/lib/spark/jars/jackson-mapper-asl-1.9.13.jar
             scanner: grype
+      - timestamp: 2024-12-23T14:41:33Z
+        type: fix-not-planned
+        data:
+          note: 'This issue concerns codehaus jackson-mapper-asl, which is no longer maintained. Spark has a transitive dependency on this library due to Hive 2.3, which requires it to initialize the FunctionRegistry. Hive 3.x, planned for Spark 4.x, should remove the dependency on codehaus-jackson. However, even if the vulnerability is fixed in Spark 4.x, it won''t be possible to backport the fix to Spark 3.5.x due to its dependency on Hive 2.3. For more details: https://issues.apache.org/jira/browse/SPARK-44114, https://github.com/apache/spark/pull/40893, https://issues.apache.org/jira/browse/SPARK-30466'