Skip to content

Commit e8ad0fb

Browse files
feat(hbase): Add dependencies for ABFS and ADLS (#853)
* feat(hbase): Add dependencies for ABFS and ADLS * fix: Hadolint warnings * fix: Hadolint warnings
1 parent 4d0d9ca commit e8ad0fb

File tree

3 files changed

+30
-0
lines changed

3 files changed

+30
-0
lines changed

.hadolint.yaml

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,26 @@ ignored:
3030
# Reason: I've yet to see this being useful, where this happens we usually have no way to actually provide the file
3131
- SC1091
3232

33+
# Expressions don't expand in single quotes, use double quotes for that.
34+
# https://www.shellcheck.net/wiki/SC2016
35+
# Reason: Ignoring because envsubst requires environment variable names
36+
# as parameter. These environment variables must not be expanded, e.g.:
37+
# envsubst '${PRODUCT}:${HBASE_OPERATOR_TOOLS}' < /stackable/bin/hbck2.env
38+
- SC2016
39+
3340
# Use cd ... || exit in case cd fails.
3441
# https://github.com/koalaman/shellcheck/wiki/SC2164
3542
# Reason: Ignoring because we inherit SHELL from the base image which contains "-e" for bash
3643
- SC2164
44+
45+
# In POSIX sh, [[ ]] is undefined.
46+
# https://www.shellcheck.net/wiki/SC3010
47+
# Reason: Ignoring because we inherit SHELL from the base image which
48+
# sets the default shell to Bash where [[ ]] is defined.
49+
- SC3010
50+
51+
# In POSIX sh, string replacement is undefined.
52+
# https://www.shellcheck.net/wiki/SC3060
53+
# Reason: Ignoring because we inherit SHELL from the base image which
54+
# sets the default shell to Bash where string replacement is supported.
55+
- SC3060

CHANGELOG.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ All notable changes to this project will be documented in this file.
1616
- trino-cli: Add version `455` ([#822]).
1717
- spark: Add version `3.5.2` ([#848]).
1818
- hadoop: Add patch "HADOOP-18516: Support Fixed SAS Token for ABFS Authentication" ([#852]).
19+
- hbase: Add hadoop-azure.jar to the lib directory to support the Azure Blob Filesystem and
20+
the Azure Data Lake Storage ([#853]).
1921

2022
### Changed
2123

@@ -54,6 +56,7 @@ All notable changes to this project will be documented in this file.
5456
[#848]: https://github.com/stackabletech/docker-images/pull/848
5557
[#851]: https://github.com/stackabletech/docker-images/pull/851
5658
[#852]: https://github.com/stackabletech/docker-images/pull/852
59+
[#853]: https://github.com/stackabletech/docker-images/pull/853
5760

5861
## [24.7.0] - 2024-07-24
5962

hbase/Dockerfile

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -317,6 +317,14 @@ COPY --chown=stackable:stackable --from=phoenix-builder /stackable/phoenix /stac
317317
COPY --chown=stackable:stackable --from=hadoop-s3-builder /stackable/bin/export-snapshot-to-s3 /stackable/bin/export-snapshot-to-s3
318318
COPY --chown=stackable:stackable --from=hadoop-s3-builder /stackable/hadoop/share/hadoop/tools/lib/ /stackable/hadoop/share/hadoop/tools/lib/
319319

320+
# Copy the dependencies from Hadoop which are required for the Azure Data Lake
321+
# Storage (ADLS) to /stackable/hbase-${PRODUCT}/lib which is on the classpath.
322+
# hadoop-azure-${HADOOP}.jar contains the AzureBlobFileSystem which is required
323+
# by hadoop-common-${HADOOP}.jar if the scheme of a file system is "abfs://".
324+
COPY --chown=stackable:stackable --from=hadoop-builder \
325+
/stackable/hadoop/share/hadoop/tools/lib/hadoop-azure-${HADOOP}.jar \
326+
/stackable/hbase-${PRODUCT}/lib/
327+
320328
COPY --chown=stackable:stackable --from=opa-authorizer-builder /stackable/hbase-opa-authorizer/target/hbase-opa-authorizer*.jar /stackable/hbase-${PRODUCT}/lib
321329

322330
RUN <<EOF

0 commit comments

Comments
 (0)