Skip to content

Commit 1e737da

Browse files
razvanNickLarsenNZ
andauthored
fix: spark 3 add bouncy castle as compile time dependency (#1212)
* fix: spark 3 add bouncy castle as compile time dependency * put the dep in the right place * update patches for spark 3.5.5 * update changelog * Update CHANGELOG.md Co-authored-by: Nick <10092581+NickLarsenNZ@users.noreply.github.com> --------- Co-authored-by: Nick <10092581+NickLarsenNZ@users.noreply.github.com>
1 parent 7d630e5 commit 1e737da

File tree

3 files changed

+170
-2
lines changed

3 files changed

+170
-2
lines changed

CHANGELOG.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,7 @@ All notable changes to this project will be documented in this file.
6060
- zookeeper: bump netty version for CVE-2025-24970 in 3.9.3 ([#1180])
6161
- hadoop: backport HADOOP-19352, HADOOP-19335, HADOOP-19465, HADOOP-19456 and HADOOP-19225 to fix vulnerabilities in Hadoop `3.4.1` ([#1184])
6262
- hadoop: Backport HADOOP-18583 to make OpenSSL 3.x work with the native hadoop libraries ([#1209]).
63+
- spark: backport [SPARK-51311] Promote bcprov-jdk18on to compile scope ([#1212]).
6364

6465
### Changed
6566

@@ -223,14 +224,15 @@ All notable changes to this project will be documented in this file.
223224
[#1189]: https://github.com/stackabletech/docker-images/pull/1189
224225
[#1197]: https://github.com/stackabletech/docker-images/pull/1197
225226
[#1209]: https://github.com/stackabletech/docker-images/pull/1209
227+
[#1212]: https://github.com/stackabletech/docker-images/pull/1212
226228

227229
## [25.3.0] - 2025-03-21
228230

229231
### Added
230232

231233
- omid: Added 1.1.3-SNAPSHOT to allow for easier scanning pre-release
232234
- airflow: Add OPA support to Airflow ([#978]).
233-
- nifi: Activate `include-hadoop` profile for NiFi version 2.* ([#958]).
235+
- nifi: Activate `include-hadoop` profile for NiFi version 2 ([#958]).
234236
- nifi: Add NiFi hadoop Azure and GCP libraries ([#943]).
235237
- superset: Add role mapping from OPA ([#979]).
236238
- base: Add containerdebug tool ([#928], [#959]).
@@ -763,7 +765,7 @@ All notable changes to this project will be documented in this file.
763765
- BREAKING: Use RPM instead of tar.gz for Vector. Because of that, the
764766
location of the Vector executable changed, and the operator-rs version
765767
0.45.0 or newer is required ([#429]).
766-
- spark-k8s: Rework spark images to build on top of java-base image. This fixes the missing tzdata-java package in 0.0.0-dev versions ([#434]).
768+
- spark-k8s: Rework spark images to build on top of java-base image. This fixes the missing tzdata-java package in 0.0.0-dev versions ([#434]).
767769

768770
- airflow: Updated git-sync to 3.6.8 ([#431]).
769771
- airflow: Updated statsd-exporter to 0.24, this was accidentally moved to a very old version previously (0.3.0) ([#431]).
Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
From c4a23f0060f34a2e1c3b826b9698ad56a5ce7176 Mon Sep 17 00:00:00 2001
2+
From: Razvan-Daniel Mihai <84674+razvan@users.noreply.github.com>
3+
Date: Tue, 22 Jul 2025 18:51:28 +0200
4+
Subject: Backport [SPARK-51311][BUILD] Promote bcprov-jdk18on to compile scope
5+
6+
---
7+
LICENSE-binary | 1 +
8+
assembly/pom.xml | 14 ++++++++++++++
9+
dev/deps/spark-deps-hadoop-3-hive-2.3 | 1 +
10+
licenses-binary/LICENSE-bouncycastle.txt | 13 +++++++++++++
11+
4 files changed, 29 insertions(+)
12+
create mode 100644 licenses-binary/LICENSE-bouncycastle.txt
13+
14+
diff --git a/LICENSE-binary b/LICENSE-binary
15+
index 05645977a0..9834cf333f 100644
16+
--- a/LICENSE-binary
17+
+++ b/LICENSE-binary
18+
@@ -480,6 +480,7 @@ org.typelevel:algebra_2.12:jar
19+
org.typelevel:cats-kernel_2.12
20+
org.typelevel:machinist_2.12
21+
net.razorvine:pickle
22+
+org.bouncycastle:bcprov-jdk18on
23+
org.slf4j:jcl-over-slf4j
24+
org.slf4j:jul-to-slf4j
25+
org.slf4j:slf4j-api
26+
diff --git a/assembly/pom.xml b/assembly/pom.xml
27+
index 2066bbeb7e..05edd80958 100644
28+
--- a/assembly/pom.xml
29+
+++ b/assembly/pom.xml
30+
@@ -85,8 +85,22 @@
31+
<artifactId>guava</artifactId>
32+
<scope>${hadoop.deps.scope}</scope>
33+
</dependency>
34+
+
35+
+ <!--
36+
+ Adapted from https://github.com/apache/spark/pull/50077
37+
+
38+
+ SPARK-51311: HDFS-15098 (3.4.0) adds hard dependency on bcprov-jdk18on, Spark fails to submit
39+
+ to Kerberized cluster without this dependency, until HADOOP-19152 (3.5.0, unreleased)
40+
+ -->
41+
+ <dependency>
42+
+ <groupId>org.bouncycastle</groupId>
43+
+ <artifactId>bcprov-jdk18on</artifactId>
44+
+ <scope>${hadoop.deps.scope}</scope>
45+
+ </dependency>
46+
+
47+
</dependencies>
48+
49+
+
50+
<build>
51+
<plugins>
52+
<plugin>
53+
diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3 b/dev/deps/spark-deps-hadoop-3-hive-2.3
54+
index 4feea62dfe..df85dcb6f5 100644
55+
--- a/dev/deps/spark-deps-hadoop-3-hive-2.3
56+
+++ b/dev/deps/spark-deps-hadoop-3-hive-2.3
57+
@@ -28,6 +28,7 @@ aws-java-sdk-bundle/1.12.262//aws-java-sdk-bundle-1.12.262.jar
58+
azure-data-lake-store-sdk/2.3.9//azure-data-lake-store-sdk-2.3.9.jar
59+
azure-keyvault-core/1.0.0//azure-keyvault-core-1.0.0.jar
60+
azure-storage/7.0.1//azure-storage-7.0.1.jar
61+
+bcprov-jdk18on/1.77//bcprov-jdk18on-1.77.jar
62+
blas/3.0.3//blas-3.0.3.jar
63+
bonecp/0.8.0.RELEASE//bonecp-0.8.0.RELEASE.jar
64+
breeze-macros_2.12/2.1.0//breeze-macros_2.12-2.1.0.jar
65+
diff --git a/licenses-binary/LICENSE-bouncycastle.txt b/licenses-binary/LICENSE-bouncycastle.txt
66+
new file mode 100644
67+
index 0000000000..277dcd1ebb
68+
--- /dev/null
69+
+++ b/licenses-binary/LICENSE-bouncycastle.txt
70+
@@ -0,0 +1,13 @@
71+
+Copyright (c) 2000-2024 The Legion of the Bouncy Castle Inc. (https://www.bouncycastle.org).
72+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and
73+
+associated documentation files (the "Software"), to deal in the Software without restriction,
74+
+including without limitation the rights to use, copy, modify, merge, publish, distribute,
75+
+sub license, and/or sell copies of the Software, and to permit persons to whom the Software is
76+
+furnished to do so, subject to the following conditions: The above copyright notice and this
77+
+permission notice shall be included in all copies or substantial portions of the Software.
78+
+
79+
+**THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT
80+
+NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
81+
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
82+
+DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT
83+
+OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.**
Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
From 37d866706d952702effd640babf891fef349da7d Mon Sep 17 00:00:00 2001
2+
From: Razvan-Daniel Mihai <84674+razvan@users.noreply.github.com>
3+
Date: Tue, 22 Jul 2025 17:34:03 +0200
4+
Subject: Backport [SPARK-51311][BUILD] Promote bcprov-jdk18on to compile scope
5+
6+
---
7+
LICENSE-binary | 1 +
8+
assembly/pom.xml | 14 ++++++++++++++
9+
dev/deps/spark-deps-hadoop-3-hive-2.3 | 1 +
10+
licenses-binary/LICENSE-bouncycastle.txt | 13 +++++++++++++
11+
4 files changed, 29 insertions(+)
12+
create mode 100644 licenses-binary/LICENSE-bouncycastle.txt
13+
14+
diff --git a/LICENSE-binary b/LICENSE-binary
15+
index 05645977a0..9834cf333f 100644
16+
--- a/LICENSE-binary
17+
+++ b/LICENSE-binary
18+
@@ -480,6 +480,7 @@ org.typelevel:algebra_2.12:jar
19+
org.typelevel:cats-kernel_2.12
20+
org.typelevel:machinist_2.12
21+
net.razorvine:pickle
22+
+org.bouncycastle:bcprov-jdk18on
23+
org.slf4j:jcl-over-slf4j
24+
org.slf4j:jul-to-slf4j
25+
org.slf4j:slf4j-api
26+
diff --git a/assembly/pom.xml b/assembly/pom.xml
27+
index dcc46b0b82..def40ad52e 100644
28+
--- a/assembly/pom.xml
29+
+++ b/assembly/pom.xml
30+
@@ -85,8 +85,22 @@
31+
<artifactId>guava</artifactId>
32+
<scope>${hadoop.deps.scope}</scope>
33+
</dependency>
34+
+
35+
+ <!--
36+
+ Adapted from https://github.com/apache/spark/pull/50077
37+
+
38+
+ SPARK-51311: HDFS-15098 (3.4.0) adds hard dependency on bcprov-jdk18on, Spark fails to submit
39+
+ to Kerberized cluster without this dependency, until HADOOP-19152 (3.5.0, unreleased)
40+
+ -->
41+
+ <dependency>
42+
+ <groupId>org.bouncycastle</groupId>
43+
+ <artifactId>bcprov-jdk18on</artifactId>
44+
+ <scope>${hadoop.deps.scope}</scope>
45+
+ </dependency>
46+
+
47+
</dependencies>
48+
49+
+
50+
<build>
51+
<plugins>
52+
<plugin>
53+
diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3 b/dev/deps/spark-deps-hadoop-3-hive-2.3
54+
index dbf0cb34c5..689f50612b 100644
55+
--- a/dev/deps/spark-deps-hadoop-3-hive-2.3
56+
+++ b/dev/deps/spark-deps-hadoop-3-hive-2.3
57+
@@ -28,6 +28,7 @@ aws-java-sdk-bundle/1.12.262//aws-java-sdk-bundle-1.12.262.jar
58+
azure-data-lake-store-sdk/2.3.9//azure-data-lake-store-sdk-2.3.9.jar
59+
azure-keyvault-core/1.0.0//azure-keyvault-core-1.0.0.jar
60+
azure-storage/7.0.1//azure-storage-7.0.1.jar
61+
+bcprov-jdk18on/1.77//bcprov-jdk18on-1.77.jar
62+
blas/3.0.3//blas-3.0.3.jar
63+
bonecp/0.8.0.RELEASE//bonecp-0.8.0.RELEASE.jar
64+
breeze-macros_2.12/2.1.0//breeze-macros_2.12-2.1.0.jar
65+
diff --git a/licenses-binary/LICENSE-bouncycastle.txt b/licenses-binary/LICENSE-bouncycastle.txt
66+
new file mode 100644
67+
index 0000000000..277dcd1ebb
68+
--- /dev/null
69+
+++ b/licenses-binary/LICENSE-bouncycastle.txt
70+
@@ -0,0 +1,13 @@
71+
+Copyright (c) 2000-2024 The Legion of the Bouncy Castle Inc. (https://www.bouncycastle.org).
72+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and
73+
+associated documentation files (the "Software"), to deal in the Software without restriction,
74+
+including without limitation the rights to use, copy, modify, merge, publish, distribute,
75+
+sub license, and/or sell copies of the Software, and to permit persons to whom the Software is
76+
+furnished to do so, subject to the following conditions: The above copyright notice and this
77+
+permission notice shall be included in all copies or substantial portions of the Software.
78+
+
79+
+**THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT
80+
+NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
81+
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
82+
+DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT
83+
+OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.**

0 commit comments

Comments
 (0)