Skip to content

Commit ff93cbb

Browse files
MaxGekkMaxGekk
committed
[SPARK-52634][SQL][DOCS] Update the ANSI compliance page regarding the TIME type
### What changes were proposed in this pull request? In the PR, I propose to update the doc page https://spark.apache.org/docs/latest/sql-ref-ansi-compliance.html#cast regarding the TIME data type. After the update, the page looks like: <img width="947" alt="Screenshot 2025-07-04 at 10 32 51" src="https://github.com/user-attachments/assets/f3f55e95-a883-4ae5-bfe5-5f7b72d88350" /> Store assignment rules: <img width="953" alt="Screenshot 2025-07-04 at 10 33 13" src="https://github.com/user-attachments/assets/6a3f4da2-a9b2-4709-9eaf-d516e000a8b8" /> ### Why are the changes needed? To provide actual information about the TIME data type to Spark SQL users. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Manually by building and checking the page: ``` $ SKIP_API=1 bundle exec jekyll serve --watch ``` ### Was this patch authored or co-authored using generative AI tooling? No. Closes #51333 from MaxGekk/time-cast-doc. Authored-by: Max Gekk <max.gekk@gmail.com> Signed-off-by: Max Gekk <max.gekk@gmail.com>
1 parent 08e43d3 commit ff93cbb

File tree

1 file changed

+29
-26
lines changed

1 file changed

+29
-26
lines changed

‎docs/sql-ref-ansi-compliance.md

Lines changed: 29 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -110,19 +110,20 @@ Besides, the ANSI SQL mode disallows the following type conversions which are al
110110
The valid combinations of source and target data type in a `CAST` expression are given by the following table.
111111
“Y” indicates that the combination is syntactically valid without restriction and “N” indicates that the combination is not valid.
112112

113-
| Source\Target | Numeric | String | Date | Timestamp | Timestamp_NTZ | Interval | Boolean | Binary | Array | Map | Struct |
114-
|---------------|--------------------------------------|--------------------------------------|--------------------------------------|--------------------------------------|--------------------------------------|--------------------------------------|--------------------------------------|--------|--------------------------------------|--------------------------------------|--------------------------------------|
115-
| Numeric | <span style="color:red">**Y**</span> | <span style="color:red">**Y**</span> | N | <span style="color:red">**Y**</span> | N | <span style="color:red">**Y**</span> | Y | N | N | N | N |
116-
| String | <span style="color:red">**Y**</span> | Y | <span style="color:red">**Y**</span> | <span style="color:red">**Y**</span> | <span style="color:red">**Y**</span> | <span style="color:red">**Y**</span> | <span style="color:red">**Y**</span> | Y | N | N | N |
117-
| Date | N | Y | Y | Y | Y | N | N | N | N | N | N |
118-
| Timestamp | <span style="color:red">**Y**</span> | Y | Y | Y | Y | N | N | N | N | N | N |
119-
| Timestamp_NTZ | N | Y | Y | Y | Y | N | N | N | N | N | N |
120-
| Interval | <span style="color:red">**Y**</span> | Y | N | N | N | Y | N | N | N | N | N |
121-
| Boolean | Y | Y | N | N | N | N | Y | N | N | N | N |
122-
| Binary | N | Y | N | N | N | N | N | Y | N | N | N |
123-
| Array | N | Y | N | N | N | N | N | N | <span style="color:red">**Y**</span> | N | N |
124-
| Map | N | Y | N | N | N | N | N | N | N | <span style="color:red">**Y**</span> | N |
125-
| Struct | N | Y | N | N | N | N | N | N | N | N | <span style="color:red">**Y**</span> |
113+
| Source\Target | Numeric | String | Date | Time | Timestamp | Timestamp_NTZ | Interval | Boolean | Binary | Array | Map | Struct |
114+
|---------------|--------------------------------------|--------------------------------------|--------------------------------------|------|--------------------------------------|--------------------------------------|--------------------------------------|--------------------------------------|--------|--------------------------------------|--------------------------------------|--------------------------------------|
115+
| Numeric | <span style="color:red">**Y**</span> | <span style="color:red">**Y**</span> | N | N |<span style="color:red">**Y**</span> | N | <span style="color:red">**Y**</span> | Y | N | N | N | N |
116+
| String | <span style="color:red">**Y**</span> | Y | <span style="color:red">**Y**</span> | Y |<span style="color:red">**Y**</span> | <span style="color:red">**Y**</span> | <span style="color:red">**Y**</span> | <span style="color:red">**Y**</span> | Y | N | N | N |
117+
| Date | N | Y | Y | N | Y | Y | N | N | N | N | N | N |
118+
| Time | N | Y | N | Y | N | N | N | N | N | N | N | N |
119+
| Timestamp | <span style="color:red">**Y**</span> | Y | Y | N | Y | Y | N | N | N | N | N | N |
120+
| Timestamp_NTZ | N | Y | Y | N | Y | Y | N | N | N | N | N | N |
121+
| Interval | <span style="color:red">**Y**</span> | Y | N | N | N | N | Y | N | N | N | N | N |
122+
| Boolean | Y | Y | N | N | N | N | N | Y | N | N | N | N |
123+
| Binary | N | Y | N | N | N | N | N | N | Y | N | N | N |
124+
| Array | N | Y | N | N | N | N | N | N | N | <span style="color:red">**Y**</span> | N | N |
125+
| Map | N | Y | N | N | N | N | N | N | N | N | <span style="color:red">**Y**</span> | N |
126+
| Struct | N | Y | N | N | N | N | N | N | N | N | N | <span style="color:red">**Y**</span> |
126127

127128
In the table above, all the `CAST`s with new syntax are marked as red <span style="color:red">**Y**</span>:
128129
* CAST(Numeric AS Numeric): raise an overflow exception if the value is out of the target data type's range.
@@ -198,19 +199,20 @@ While casting of a decimal with a fraction to an interval type with SECOND as th
198199
### Store assignment
199200
As mentioned at the beginning, when `spark.sql.storeAssignmentPolicy` is set to `ANSI`(which is the default value), Spark SQL complies with the ANSI store assignment rules on table insertions. The valid combinations of source and target data type in table insertions are given by the following table.
200201

201-
| Source\Target | Numeric | String | Date | Timestamp | Timestamp_NTZ | Interval | Boolean | Binary | Array | Map | Struct |
202-
|:-------------:|:-------:|:------:|:----:|:---------:|---------------|:--------:|:-------:|:------:|:-----:|:---:|:------:|
203-
| Numeric | Y | Y | N | N | N | N | N | N | N | N | N |
204-
| String | N | Y | N | N | N | N | N | N | N | N | N |
205-
| Date | N | Y | Y | Y | Y | N | N | N | N | N | N |
206-
| Timestamp | N | Y | Y | Y | Y | N | N | N | N | N | N |
207-
| Timestamp_NTZ | N | Y | Y | Y | Y | N | N | N | N | N | N |
208-
| Interval | N | Y | N | N | N | N* | N | N | N | N | N |
209-
| Boolean | N | Y | N | N | N | N | Y | N | N | N | N |
210-
| Binary | N | Y | N | N | N | N | N | Y | N | N | N |
211-
| Array | N | N | N | N | N | N | N | N | Y** | N | N |
212-
| Map | N | N | N | N | N | N | N | N | N | Y** | N |
213-
| Struct | N | N | N | N | N | N | N | N | N | N | Y** |
202+
| Source\Target | Numeric | String | Date | Time | Timestamp | Timestamp_NTZ | Interval | Boolean | Binary | Array | Map | Struct |
203+
|:-------------:|:-------:|:------:|:----:|:----:|:---------:|---------------|:--------:|:-------:|:------:|:-----:|:---:|:------:|
204+
| Numeric | Y | Y | N | N | N | N | N | N | N | N | N | N |
205+
| String | N | Y | N | Y | N | N | N | N | N | N | N | N |
206+
| Date | N | Y | Y | N | Y | Y | N | N | N | N | N | N |
207+
| Time | N | Y | N | Y | N | N | N | N | N | N | N | N |
208+
| Timestamp | N | Y | Y | N | Y | Y | N | N | N | N | N | N |
209+
| Timestamp_NTZ | N | Y | Y | N | Y | Y | N | N | N | N | N | N |
210+
| Interval | N | Y | N | N | N | N | N* | N | N | N | N | N |
211+
| Boolean | N | Y | N | N | N | N | N | Y | N | N | N | N |
212+
| Binary | N | Y | N | N | N | N | N | N | Y | N | N | N |
213+
| Array | N | N | N | N | N | N | N | N | N | Y** | N | N |
214+
| Map | N | N | N | N | N | N | N | N | N | N | Y** | N |
215+
| Struct | N | N | N | N | N | N | N | N | N | N | N | Y** |
214216

215217
\* Spark doesn't support interval type table column.
216218

@@ -239,6 +241,7 @@ At the heart of this conflict resolution is the Type Precedence List which defin
239241
| Float | Float -> Double |
240242
| Double | Double |
241243
| Date | Date -> Timestamp_NTZ -> Timestamp |
244+
| Time | Time |
242245
| Timestamp | Timestamp |
243246
| String | String, Long -> Double, Date -> Timestamp_NTZ -> Timestamp , Boolean, Binary ** |
244247
| Binary | Binary |

0 commit comments

Comments
 (0)