-
Notifications
You must be signed in to change notification settings - Fork 257
Labels
Spark 4.0+Spark 4.0+ issuesSpark 4.0+ issuesfeature requestNew feature or requestNew feature or request
Description
Is your feature request related to a problem? Please describe.
Multiply expression falls back to CPU when ANSI mode is enabeld. We need to accelerate this expression on spark-rapids plugin.
Simple repro case on spark-shell:
val df = spark.range(5).select(col("id"))
val result = df.withColumn("multiplied", col("id") * 100)
result.explain
25/07/03 11:12:41 WARN GpuOverrides:
!Exec <ProjectExec> cannot run on GPU because not all expressions can be replaced
@Expression <AttributeReference> id#10L could run on GPU
@Expression <Alias> (id#10L * 100) AS multiplied#15L could run on GPU
**!Expression <Multiply> (id#10L * 100) cannot run on GPU because GPU Multiplication does not support ANSI mode**
@Expression <AttributeReference> id#10L could run on GPU
@Expression <Literal> 100 could run on GPU
== Physical Plan ==
*(1) Project [id#10L, (id#10L * 100) AS multiplied#15L]
+- GpuColumnarToRow false
+- GpuRange (0, 5, step=1, splits=64)
Discussion on supporting this on other issue for reference:
#3472
#3472 (comment)
Metadata
Metadata
Assignees
Labels
Spark 4.0+Spark 4.0+ issuesSpark 4.0+ issuesfeature requestNew feature or requestNew feature or request