Using Trino with HiveStandAlone Metastore and MinoS3 on kubernetes

[alexcpn]

Hi,
I am getting the S3 key error when accessing S3 throuh trino-cli

$ kubectl exec -it trino-cli /bin/bash 

[trino@trino-cli /]$ /bin/trino --server 10.244.1.42:8080 --catalog hive --schema default
trino:default> CREATE SCHEMA test2 WITH (location ='s3a://test/');
Query 20220615_173640_00002_egaec failed: Got exception: java.nio.file.AccessDeniedException test: doesBucketExistV2 on test: com.amazonaws.services.s3.model.AmazonS3Exception: The Access Key Id you provided does not exist in our records. (Service: Amazon S3; Status Code: 403; Error Code: InvalidAccessKeyId; Request ID: 16F8DBDC0D5EDC46; S3 Extended Request ID: 4f93084c-1ea0-4ee6-8736-ad144d253656; Proxy: null), S3 Extended Request ID: 4f93084c-1ea0-4ee6-8736-ad144d253656:InvalidAccessKeyId

Documented more details here https://github.com/alexcpn/presto_in_kubernetes

Have given the S3 Access and Secret properly (https://github.com/alexcpn/presto_in_kubernetes/blob/40449e7657381b4404a50059e3b0c59749283d83/hive/hive-meta-store-standalone.yaml#L32)
and checked also in metastore pod that the environment variable has the proper values

$ kubectl exec -it metastore-75d5857d54-kd7g5  bash
bash-5.1# echo $AWS_SECRET_ACCESS_KEY
’minio123’
bash-5.1# echo $AWS_ACCESS_KEY_ID
’minio’

Note that I can access Minio/S3 console UI with this same access and secret key and can see the bucket named 'test' that I have created

No extra logs in any other pods

e$ kubectl logs -f metastore-75d5857d54-kd7g5
2022-06-15 17:16:07: Starting Metastore Server
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/apache-hive-metastore-3.1.3-bin/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop-3.3.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2022-06-15 17:16:09,099 main INFO Log4j appears to be running in a Servlet environment, but there's no log4j-web module available. If you want better web container support, please add the log4j-web JAR to your web archive or server lib directory.

[alexcpn]

I put the following properties in metastore-cfg.yaml and it works now till the schema creation part

https://github.com/alexcpn/presto_in_kubernetes/blob/main/hive/metastore-cfg.yaml#L17-L24

       <property>
                <name>fs.s3a.access.key</name>
                <value>minio</value>
            </property>
            <property>
                <name>fs.s3a.secret.key</name>
                <value>minio123</value>
            </property>