Skip to content

Various update: log4shell removal, testing-tools uid/gid #1192

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
+81 −215
Open

Various update: log4shell removal, testing-tools uid/gid #1192

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
fc81eb4
Various update: log4shell removal, testing-tools uid/gid
lfrancke Jul 7, 2025
d5ef172
address review feedback
lfrancke Jul 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion druid/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ COPY --chown=${STACKABLE_USER_UID}:0 druid/stackable/patches/${PRODUCT} /stackab

COPY --from=hadoop-builder --chown=${STACKABLE_USER_UID}:0 /stackable/patched-libs /stackable/patched-libs
# Cache mounts are owned by root by default
# We need to explicitly give the uid to use which is hardcoded to "1000" in stackable-base
# We need to explicitly give the uid to use.
# The cache id has to include the product version that we are building because otherwise
# docker encounters race conditions when building multiple versions in parallel, as all
# builder containers will share the same cache and the `rm -rf` commands will fail
Expand Down
68 changes: 34 additions & 34 deletions java-devel/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,40 +23,40 @@ gpgcheck=1
gpgkey=https://packages.adoptium.net/artifactory/api/gpg/key/public
EOF

RUN microdnf update && \
microdnf install -y \
cmake \
cyrus-sasl-devel \
# diff is required by maven during the build of hbase \
# Cannot run program "diff" (in directory "/stackable/hbase-2.4.12-src/hbase-shaded/hbase-shaded-check-invariants"
diffutils \
fuse-devel \
gcc \
gcc-c++ \
# The GNU gettext utilities contain the envsubst program which
# substitutes the values of environment variables.
gettext \
# For the apply_patches.sh script
git \
# Needed by the maven ant run plugin for the "set-hostname-property" step in zookeeper
hostname \
# Needed for compiling Java projects
"temurin-${PRODUCT}-jdk" \
krb5-devel \
libcurl-devel \
make \
maven \
openssl-devel \
# Required to unpack Omid tarball
tar \
wget \
which \
xz \
zlib-devel \
# Required for log4shell.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The diff looks like it is now missing a &&

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I'll take a look!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

d5ef172 (#1192)

unzip zip && \
microdnf clean all && \
rm -rf /var/cache/yum
RUN <<EOF
microdnf update
microdnf install \
cmake \
cyrus-sasl-devel \
`# diff is required by maven during the build of hbase` \
`# Cannot run program "diff" (in directory "/stackable/hbase-2.4.12-src/hbase-shaded/hbase-shaded-check-invariants"` \
diffutils \
fuse-devel \
gcc \
gcc-c++ \
`# The GNU gettext utilities contain the envsubst program which` \
`# substitutes the values of environment variables.` \
gettext \
`# For the apply_patches.sh script`\
git \
`# Needed by the maven ant run plugin for the "set-hostname-property" step in zookeeper` \
hostname \
`# Needed for compiling Java projects` \
"temurin-${PRODUCT}-jdk" \
krb5-devel \
libcurl-devel \
make \
maven \
openssl-devel \
`# Required to unpack Omid tarball` \
tar \
wget \
which \
xz \
zlib-devel
microdnf clean all
rm -rf /var/cache/yum
EOF

ENV JAVA_HOME="/usr/lib/jvm/temurin-${PRODUCT}-jdk"

Expand Down
18 changes: 0 additions & 18 deletions omid/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,24 +49,6 @@ if [ "${DELETE_CACHES}" = "true" ] ; then
fi
EOF

# ===
# For earlier versions this script removes the .class file that contains the
# vulnerable code.
# TODO: This can be restricted to target only versions which do not honor the environment
# varible that has been set above but this has not currently been implemented
COPY shared/log4shell.sh /bin
RUN /bin/log4shell.sh /stackable/omid-tso-server-${PRODUCT}-stackable${RELEASE}

# Ensure no vulnerable files are left over
# This will currently report vulnerable files being present, as it also alerts on
# SocketNode.class, which we do not remove with our scripts.
# Further investigation will be needed whether this should also be removed.
COPY shared/log4shell_1.6.1-log4shell_Linux_x86_64 /bin/log4shell_scanner_x86_64
COPY shared/log4shell_1.6.1-log4shell_Linux_aarch64 /bin/log4shell_scanner_aarch64
COPY shared/log4shell_scanner /bin/log4shell_scanner
RUN /bin/log4shell_scanner s /stackable/omid-tso-server-${PRODUCT}-stackable${RELEASE}
# ===

FROM stackable/image/java-base

ARG PRODUCT
Expand Down
3 changes: 0 additions & 3 deletions shared/log4j.txt
View file
Open in desktop

This file was deleted.

119 changes: 0 additions & 119 deletions shared/log4shell.sh
View file
Open in desktop

This file was deleted.

Binary file removed shared/log4shell_1.6.1-log4shell_Linux_aarch64
View file
Open in desktop
Binary file not shown.
Binary file removed shared/log4shell_1.6.1-log4shell_Linux_x86_64
View file
Open in desktop
Binary file not shown.
3 changes: 0 additions & 3 deletions shared/log4shell_scanner
View file
Open in desktop

This file was deleted.

83 changes: 46 additions & 37 deletions testing-tools/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@ FROM python:3.12-slim-bullseye@sha256:229b2897e6b5c630d83a721e42dd1c96e3ec996323
ARG PRODUCT
ARG RELEASE
ARG KEYCLOAK_VERSION
ARG STACKABLE_USER_UID
ARG STACKABLE_USER_GID
ARG STACKABLE_USER_NAME

LABEL name="Stackable Testing Tools" \
maintainer="info@stackable.tech" \
Expand All @@ -25,49 +28,55 @@ SHELL ["/bin/bash", "-euo", "pipefail", "-c"]
# This is needed so that krb5-user installs without prompting for a realm.
ENV DEBIAN_FRONTEND=noninteractive

# krb5-user/libkrb5-dev are needed for Kerberos support.
RUN apt-get update && \
apt-get install -y --no-install-recommends \
build-essential \
ca-certificates \
curl \
gzip \
jq \
krb5-user \
kubernetes-client \
libkrb5-dev \
libssl-dev \
libxml2-dev \
libxslt1-dev \
pkg-config \
python3-certifi \
python3-idna \
python3-semver \
python3-thrift \
python3-toml \
python3-urllib3 \
tar \
zip \
unzip \
# Java 11 seems like the best middle-ground for all tools
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I still dislike us commenting like 20 lines above and prefer keeping the comments at the line where they apply.
Not blocking this PR

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not possible with the EOF/heredoc style unfortunately.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use a trick in other places, such as

docker-images/trino/trino/Dockerfile

Lines 46 to 55 in e93d8f3

./mvnw \
--batch-mode \
--no-transfer-progress \
install \
`# -Dmaven.test.skip # Unable to skip test compilation without an unused dependency error for software.amazon.awssdk:identity-spi` \
-DskipTests `# Skip test execution` \
-Dcheckstyle.skip `# Skip checkstyle checks. We dont care if the code is properly formatted, it just wastes time` \
-Dmaven.javadoc.skip=true `# Dont generate javadoc` \
-Ddep.presto-jdbc-under-test=${NEW_VERSION} \
--projects="$SKIP_PROJECTS"

The following diff works:

diff --git a/testing-tools/Dockerfile b/testing-tools/Dockerfile
index 06e10f0..898404f 100644
--- a/testing-tools/Dockerfile
+++ b/testing-tools/Dockerfile
@@ -33,7 +33,6 @@ COPY testing-tools/python /stackable/python
 COPY testing-tools/licenses /licenses
 
 # krb5-user/libkrb5-dev are needed for Kerberos support.
-# Java 11 seems like the best middle-ground for all tools
 RUN <<EOF
 apt-get update
 apt-get install -y --no-install-recommends \
@@ -42,8 +41,10 @@ apt-get install -y --no-install-recommends \
   curl \
   gzip \
   jq \
+  `# Needed for Kerberos support` \
   krb5-user \
   kubernetes-client \
+  `# Needed for Kerberos support` \
   libkrb5-dev \
   libssl-dev \
   libxml2-dev \
@@ -58,6 +59,7 @@ apt-get install -y --no-install-recommends \
   tar \
   zip \
   unzip \
+  `# Java 11 seems like the best middle-ground for all tools` \
   openjdk-11-jdk-headless
 
 apt-get clean

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not know that syntax, thanks.,

d5ef172 (#1192)

openjdk-11-jdk-headless && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

COPY testing-tools/python /stackable/python
COPY testing-tools/licenses /licenses

ENV PATH=/stackable/keycloak/bin:$PATH
RUN curl --fail -L https://repo.stackable.tech/repository/packages/keycloak/keycloak-${KEYCLOAK_VERSION}.tar.gz | tar -xzC /stackable && \
ln -s /stackable/keycloak-${KEYCLOAK_VERSION} /stackable/keycloak

RUN pip install --no-cache-dir --upgrade pip && \
pip install --no-cache-dir -r /stackable/python/requirements.txt && \
groupadd -r stackable --gid=1000 && \
useradd -r -g stackable --uid=1000 stackable && \
chown -R stackable:stackable /stackable
RUN <<EOF
apt-get update
apt-get install -y --no-install-recommends \
build-essential \
ca-certificates \
curl \
gzip \
jq \
`# krb5-user/libkrb5-dev are needed for Kerberos support. ` \
krb5-user \
libkrb5-dev \
kubernetes-client \
libssl-dev \
libxml2-dev \
libxslt1-dev \
pkg-config \
python3-certifi \
python3-idna \
python3-semver \
python3-thrift \
python3-toml \
python3-urllib3 \
tar \
zip \
unzip \
`# Java 11 seems like the best middle-ground for all tools` \
openjdk-11-jdk-headless

apt-get clean
rm -rf /var/lib/apt/lists/*

curl --fail -L https://repo.stackable.tech/repository/packages/keycloak/keycloak-${KEYCLOAK_VERSION}.tar.gz | tar -xzC /stackable
ln -s /stackable/keycloak-${KEYCLOAK_VERSION} /stackable/keycloak

pip install --no-cache-dir --upgrade pip
pip install --no-cache-dir -r /stackable/python/requirements.txt
groupadd -r ${STACKABLE_USER_NAME} --gid=${STACKABLE_USER_GID}
useradd -r -g ${STACKABLE_USER_NAME} --uid=${STACKABLE_USER_UID} ${STACKABLE_USER_NAME}
chown -R ${STACKABLE_USER_UID}:0 /stackable
EOF

ENV PATH=/stackable/keycloak/bin:$PATH

USER stackable
USER ${STACKABLE_USER_UID}

ENV STACKABLE_PRODUCT_VERSION=${PRODUCT}

Expand Down