Skip to content

[KYUUBI #6691] A new Spark SQL command to merge small files #21801

[KYUUBI #6691] A new Spark SQL command to merge small files

[KYUUBI #6691] A new Spark SQL command to merge small files #21801

Triggered via pull request September 13, 2024 14:39
Status Failure
Total duration 2h 49m 47s
Artifacts 7

master.yml

on: pull_request
Kyuubi Server On Kubernetes Integration Test
1h 1m
Kyuubi Server On Kubernetes Integration Test
Spark Engine On Kubernetes Integration Test
46m 58s
Spark Engine On Kubernetes Integration Test
Matrix: Kyuubi and Spark Test
Matrix: Hive Test
Matrix: JDBC Trino TPC Tests
Matrix: Scala Test
Matrix: Spark Connector Cross Version Test
Matrix: Zookeeper Integration Test
Fit to window
Zoom out
Zoom in

Annotations

45 errors and 35 warnings
Kyuubi and Spark Test (17, 3.5, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.5, normal)
Timeout to lock on path [/engine_test2_1.10.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (17, 3.5, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.5, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.5, normal)
Timeout to lock on path [/engine_test2_1.10.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (17, 3.5, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.5, normal)
Process completed with exit code 1.
Kyuubi and Spark Test (17, 3.3, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.3, normal)
Timeout to lock on path [/engine_test2_1.10.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (17, 3.3, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.3, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.3, normal)
Timeout to lock on path [/engine_test2_1.10.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (17, 3.3, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (8, 3.4, normal)
Timeout to lock on path [/engine_test2_1.10.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (8, 3.4, normal)
Timeout to lock on path [/engine_test2_1.10.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (17, 3.4, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.4, normal)
Timeout to lock on path [/engine_test2_1.10.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (17, 3.4, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.4, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.4, normal)
Timeout to lock on path [/engine_test2_1.10.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (17, 3.4, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (8, 3.5, normal)
Timeout to lock on path [/engine_test2_1.10.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (8, 3.5, normal)
Timeout to lock on path [/engine_test2_1.10.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (8, 3.5, normal)
Process completed with exit code 1.
Kyuubi and Spark Test (8, 3.3, normal)
Timeout to lock on path [/engine_test2_1.10.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (8, 3.3, normal)
Timeout to lock on path [/engine_test2_1.10.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Scala Test (2.13, 8, 3.5)
Timeout to lock on path [/engine_test2_1.10.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Scala Test (2.13, 8, 3.5)
Timeout to lock on path [/engine_test2_1.10.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Scala Test (2.13, 8, 3.5)
Process completed with exit code 1.
Kyuubi and Spark Test (21, 4.0, -Pscala-2.13, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (21, 4.0, -Pscala-2.13, normal)
Timeout to lock on path [/engine_test2_1.10.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (21, 4.0, -Pscala-2.13, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (21, 4.0, -Pscala-2.13, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (21, 4.0, -Pscala-2.13, normal)
Timeout to lock on path [/engine_test2_1.10.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (21, 4.0, -Pscala-2.13, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.5, -Pscala-2.13 -Dspark.archive.mirror=https://archive.apache.org/di...
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.5, -Pscala-2.13 -Dspark.archive.mirror=https://archive.apache.org/di...
Timeout to lock on path [/engine_test2_1.10.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (17, 3.5, -Pscala-2.13 -Dspark.archive.mirror=https://archive.apache.org/di...
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.5, -Pscala-2.13 -Dspark.archive.mirror=https://archive.apache.org/di...
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.5, -Pscala-2.13 -Dspark.archive.mirror=https://archive.apache.org/di...
Timeout to lock on path [/engine_test2_1.10.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (17, 3.5, -Pscala-2.13 -Dspark.archive.mirror=https://archive.apache.org/di...
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Zookeeper Integration Test (8, 3.4, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Spark Connector Cross Version Test (17, 2.12, 3.5, 3.4, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Zookeeper Integration Test (8, 3.5, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Zookeeper Integration Test (8, 3.6, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Spark Connector Cross Version Test (17, 2.12, 3.5, 3.3, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Zookeeper Integration Test (8, 3.7, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Hive Test (8, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Spark Connector Cross Version Test (17, 2.13, 3.5, 3.4, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Spark Engine On Kubernetes Integration Test
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Spark Connector Cross Version Test (17, 2.13, 3.5, 3.3, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Flink Test (8, 1.17, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Spark Connector Cross Version Test (17, 2.13, 3.5, 4.0, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Kyuubi Server On Kubernetes Integration Test
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
JDBC Trino TPC Tests (8, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Hive Test (8, -Dhive.archive.mirror=https://archive.apache.org/dist/hive/hive-2.3.10 -Dhive.archi...
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Kyuubi and Spark Test (17, 3.5, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3, actions/upload-artifact@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Kyuubi and Spark Test (17, 3.3, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Kyuubi and Spark Test (8, 3.4, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Flink Test (8, 1.18, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Kyuubi and Spark Test (17, 3.4, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Kyuubi and Spark Test (8, 3.5, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3, actions/upload-artifact@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Kyuubi and Spark Test (8, 3.3, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Scala Test (2.13, 8, 3.5)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3, actions/upload-artifact@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Hive Test (8, -Dhive.archive.mirror=https://github.com/pan3793/cdh-hive/releases/download/cdh6.3....
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Flink Test (8, 1.19, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Kyuubi and Spark Test (21, 4.0, -Pscala-2.13, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Kyuubi and Spark Test (8, 3.5, -Dspark.archive.mirror=https://archive.apache.org/dist/spark/spark...
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3, actions/upload-artifact@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Kyuubi and Spark Test (8, 3.5, -Dspark.archive.mirror=https://archive.apache.org/dist/spark/spark...
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3, actions/upload-artifact@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Kyuubi and Spark Test (17, 3.5, -Pscala-2.13 -Dspark.archive.mirror=https://archive.apache.org/di...
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3, actions/upload-artifact@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Flink Test (8, 1.20, normal)
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Kyuubi and Spark Test (8, 3.5, -Dspark.archive.mirror=https://archive.apache.org/dist/spark/spark...
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3, actions/upload-artifact@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Flink Test (8, 1.17, -Dflink.archive.mirror=https://archive.apache.org/dist/flink/flink-1.18.1 -D...
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Flink Test (8, 1.17, -Dflink.archive.mirror=https://archive.apache.org/dist/flink/flink-1.19.1 -D...
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Flink Test (8, 1.17, -Dflink.archive.mirror=https://archive.apache.org/dist/flink/flink-1.20.0 -D...
The following actions use a deprecated Node.js version and will be forced to run on node20: actions/cache@v3. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Deprecation notice: v1, v2, and v3 of the artifact actions
The following artifacts were uploaded using a version of actions/upload-artifact that is scheduled for deprecation: "unit-tests-log-java-17-spark-3.5-normal", "unit-tests-log-java-17-spark-3.5-verify-on-spark-4.0-binary", "unit-tests-log-java-8-spark-3.5-normal", "unit-tests-log-java-8-spark-3.5-verify-on-spark-3.2-binary", "unit-tests-log-java-8-spark-3.5-verify-on-spark-3.3-binary", "unit-tests-log-java-8-spark-3.5-verify-on-spark-3.4-binary", "unit-tests-log-scala-2.13-java-8-spark-3.5". Please update your workflow to use v4 of the artifact actions. Learn more: https://github.blog/changelog/2024-04-16-deprecation-notice-v3-of-the-artifact-actions/

Artifacts

Produced during runtime
Name Size Digest
unit-tests-log-java-17-spark-3.5-normal Expired
34.6 MB
unit-tests-log-java-17-spark-3.5-verify-on-spark-4.0-binary Expired
71.5 MB
unit-tests-log-java-8-spark-3.5-normal Expired
104 MB
unit-tests-log-java-8-spark-3.5-verify-on-spark-3.2-binary Expired
112 MB
unit-tests-log-java-8-spark-3.5-verify-on-spark-3.3-binary Expired
105 MB
unit-tests-log-java-8-spark-3.5-verify-on-spark-3.4-binary Expired
108 MB
unit-tests-log-scala-2.13-java-8-spark-3.5 Expired
73.7 MB