You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[SPARK-52331][PS][TESTS] Adjust test for promotion from float32 to float64 during division
### What changes were proposed in this pull request?
Adjust test for promotion bug from float32 to float64 during division
### Why are the changes needed?
Pass nightly build with ANSI off.
Part of https://issues.apache.org/jira/browse/SPARK-52169.
The promotion bug is shown as below:
```
>>> ps.set_option("compute.fail_on_ansi_mode", False)
>>> spark.conf.set("spark.sql.ansi.enabled", False)
>>>
>>> import pandas as pd
>>> import numpy as np
>>> pdf = pd.DataFrame(
... {
... "a": [1.0, -1.0, 0.0, np.nan],
... "b": [0.0, 0.0, 0.0, 0.0],
... },
... dtype=np.float32,
... )
>>>
>>> psdf = ps.from_pandas(pdf)
>>>
>>> psdf["a"] / psdf["b"]
0 inf
1 -inf
2 NaN
3 NaN
dtype: float64
>>>
>>> pdf["a"] / pdf["b"]
0 inf
1 -inf
2 NaN
3 NaN
dtype: float32
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Test changes only.
```
% SPARK_ANSI_SQL_MODE=false ./python/run-tests --python-executables=python3.10 --testnames "pyspark.pandas.tests.computation.test_binary_ops FrameBinaryOpsTests.test_divide_by_zero_behavior"
Running PySpark tests. Output is in /Users/xinrong.meng/spark/python/unit-tests.log
Will test against the following Python executables: ['python3.10']
Will test the following Python tests: ['pyspark.pandas.tests.computation.test_binary_ops FrameBinaryOpsTests.test_divide_by_zero_behavior']
python3.10 python_implementation is CPython
python3.10 version is: Python 3.10.16
Starting test(python3.10): pyspark.pandas.tests.computation.test_binary_ops FrameBinaryOpsTests.test_divide_by_zero_behavior (temp output: /Users/xinrong.meng/spark/python/target/f26cd9b9-f6c3-48ec-86f1-d1d7f6158361/python3.10__pyspark.pandas.tests.computation.test_binary_ops_FrameBinaryOpsTests.test_divide_by_zero_behavior__wrk8yuzn.log)
Finished test(python3.10): pyspark.pandas.tests.computation.test_binary_ops FrameBinaryOpsTests.test_divide_by_zero_behavior (5s)
Tests passed in 5 seconds
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closesapache#51035 from xinrong-meng/etest_promo.
Authored-by: Xinrong Meng <xinrong@apache.org>
Signed-off-by: Xinrong Meng <xinrong@apache.org>
0 commit comments