Improve GraalVM Reachability Metadata and corresponding nativeTest related unit tests

[linghengqian]

Feature Request

For English only, other languages will not be accepted.

Please pay attention on issues you submitted, because we maybe need more details.
If no response anymore and we cannot make decision by current information, we will close it.

Please answer these questions before submitting your issue. Thanks!

Is your feature request related to a problem?

• Make ShardingSphere Proxy in GraalVM Native Image form available #21347.

Describe the feature you would like.

• In the current master branch, for the sharding, readwrite-splitting, mask, shadow and encrypt features in standalone mode and H2Database, there are already corresponding GraalVM Reachability Metadata and corresponding nativeTest-related unit tests located under GraalVM Native Image.

• The previous PR did not cover further situations. We need to add additional unit tests and corresponding GraalVM Reachability Metadata for the following functions. Some unit tests will require the GraalVM Reachability Metadata of testcontainers.

  • Modes except Local
    • apache/zookeeper
    • gcr.io/etcd-development/etcd:v3.5.10
  • Distributed Transactions
    • atomikos/transactions-essentials
    • jbosstm/narayana
    • apache/seata-server:2.3.0
  • Databases except h2database
    • mysql:9.1.0-oraclelinux9
    • postgres:17.2-bookworm
    • opengauss/opengauss-server:7.0.0-RC1
    • mcr.microsoft.com/mssql/server:2022-CU16-ubuntu-22.04
    • gvenzl/oracle-free or gvenzl/oracle-xe
    • apache/doris? Where is the docker image of runtime?
    • prestodb/presto:0.292
    • firebirdsql/firebird:5.0.1
    • apache/hive:4.0.1
    • clickhouse/clickhouse-server:25.4.5.24
  • Other features
    • DistSQL
    • Logging - qos-ch/logback

• In the current master branch, ShardingSphere temporarily hosts the GraalVM Reachability Metadata of some third-party libraries. These irrelevant json entries should be submitted to https://github.com/oracle/graalvm-reachability-metadata and then deleted from the ShardingSphere side. This sub-work is initially planned to be completed before the release of GraalVM CE for JDK 29.

  • ch.qos.logback:logback-core:1.2.12
  • org.apache.hive:hive-jdbc:4.0.1
  • com.alibaba:druid:1.2.20
  • com.atomikos:transactions:6.0.0
  • com.clickhouse:clickhouse-jdbc:0.6.3
  • com.github.ben-manes.caffeine:caffeine:2.9.3
    • Partially located at Add support for com.github.ben-manes.caffeine:caffeine:2.9.3 oracle/graalvm-reachability-metadata#388 .
  • com.github.docker-java:docker-java:3.4.0
  • com.google.protobuf:protobuf-java:3.21.12
    • Partially located at Add support for com.google.protobuf:protobuf-java-util:3.21.12 oracle/graalvm-reachability-metadata#167 .
  • com.h2database:h2:2.2.224
  • com.mysql:mysql-connector-j:8.3.0
  • com.typesafe:config:1.2.1
  • com.zaxxer:HikariCP:4.0.3
  • io.grpc:grpc-netty:1.65.1
  • io.netty:netty-transport-classes-epoll:4.1.112.Final
  • io.vertx:vertx-core:4.5.1
  • org.apache.calcite:calcite-core:1.38.0
  • org.apache.groovy:groovy:4.0.22
  • org.apache.hadoop:hadoop-common:3.3.6
  • org.apache.seata:seata-all:2.3.0
  • org.apache.zookeeper:zookeeper:3.9.0
  • org.apiguardian:apiguardian-api:1.1.2
  • org.jboss.logging:jboss-logging:3.2.1.Final
  • org.jboss.narayana.jta:jta:5.12.7.Final
  • org.junit.jupiter:junit-jupiter:5.11.1
  • org.slf4j:slf4j-api:1.7.36
  • org.springframework:spring-jcl:5.3.27
  • org.testcontainers:junit-jupiter:1.20.1
  • org.firebirdsql.jdbc:jaybird:5.0.6.java8
  • org.hamcrest:hamcrest:3.0
  • com.facebook.presto:presto-jdbc:0.292
  • org.apache.shardingsphere.elasticjob:elasticjob-lite-core:3.0.4
    • Removed via Enable Safe Composition of Metadata for ShardingSphere Proxy Native #33179.
  • org.glassfish.jaxb:jaxb-runtime:2.3.1
    • Removed directly in Removes use of JAXB API via jackson-dataformat-xml #29384, no need to commit to GraalVM Reachability Metadata Repository side.

• The current issue involves several user-side experience issues.

  • Remove the requirement for --report-unsupported-elements-at-runtime of buildArgs in GraalVM Native Build Tools. This requires further breaking ShardingSphere's compatibility with previous versions under GraalVM Native Image.
  • Increased GraalVM CE version for handling NativeTest and building ShardingSphere Proxy Native from GraalVM CE 23.0.2 For JDK 17.0.9 to GraalVM CE 24.0.2 For JDK 22.0.2.

[linghengqian]

• I'm working on this issue.

[linghengqian]

user-side experience:

• GraalVM CE 23.1.1 still fails to solve the problem that occurred at Javac-related exceptions are thrown when building Native Images with GraalVM 23.1.1 For JDK 21.0.1 oracle/graal#7500 in GraalVM CE 23.1.0 . I have refactored a small unit test at https://github.com/linghengqian/shardingsphere-graalvm-v2311-native-test .
  • Under investigation at Update the infra-expr-espresso module related GraalVM CE version to 23.1.1 #28609 . This involves breaking changes to the GraalVM Truffle framework.
  • update in 2023.12.16: Maybe [GR-50854] Fix clinit simulation for arraycopy with null values. oracle/graal#8039 .
  • update in 2024.01.18: Fixed by Updates the minimum GraalVM CE version for compiling ShardingSphere's Native Image products to JDK 21.0.2 #29764 .
  • update in 2024.06.07: Fixed by Bump the minimum GraalVM CE version required to compile ShardingSphere's GraalVM Native Image artifacts to JDK22 #31630 .
• Due to the impact of [GR-48612] Enable --strict-image-heap by default. oracle/graal#7474 in GraalVM CE 24.0.0 for JDK 22, I believe that to remove --report-unsupported-elements-at-runtime, then we must set a batch of --initialize-at-run-time=xxx.xxx.xxx in buildArgs like Spring Framework.
  • update in 2024.07.18: Maybe [GR-43837] Enable --report-unsupported-elements-at-runtime by default and deprecate the option. oracle/graal#8815 .