chown/chmod massively balloons the size of our docker images

[nightkr]

This was a regression introduced in #849 (and its followups, tracked in stackabletech/issues#645).

When you chmod/chown a file owned by a prior layer, its contents are copied into the new layer, even though they are identical. Since clients download the whole layer stack, they get to download both versions.

As an experiment, removing the chmod/chowns from the nifi image (added in #890):

diff --git a/trino/Dockerfile b/trino/Dockerfile
index 68383bdb..9f6f2f42 100644
--- a/trino/Dockerfile
+++ b/trino/Dockerfile
@@ -154,8 +154,8 @@ ln -s /stackable/jmx/jmx_prometheus_javaagent-${JMX_EXPORTER}.jar /stackable/jmx
 
 # All files and folders owned by root group to support running as arbitrary users.
 # This is best practice as all container users will belong to the root group (0).
-chown -R ${STACKABLE_USER_UID}:0 /stackable
-chmod -R g=u /stackable
+# chown -R ${STACKABLE_USER_UID}:0 /stackable
+# chmod -R g=u /stackable
 EOF
 
 # ----------------------------------------

took the total image size from 5.4GiB to 1.6GiB (according to dive).

That aside, chowning all of /stackable is a tad overly aggressive. The products shouldn't need access to overwrite their own source code, for example.

[maltesander]

I did some investigation and testing / fixing PRs. Most important ones are NiFi and Trino.

• Use check permissions script and consolidation #1054
• Fix: druid reduce image size #1039
• Fix: hbase reduce image size  #1028
• Fix: hadoop reduce image size #1029
• Fix: hive reduce image size #1040
• Fix: kafka reduce image size #1041
• Fix: nifi reduce image size #1027
• Fix: spark-k8s reduce image size #1042
• Use check permissions script and consolidation #1055
• Use check permissions script and consolidation #1053
• Fix: trino reduce image size #1025
• Fix: zookeeper reduce image size #1043
• Fix: OPA reduce image size  #1038