Skip to content

Commit ecb181e

Browse files
belieferyhuang-db
authored andcommitted
[SPARK-52384][CONNECT] Fix bug Connect should insensitive for JDBC options
### What changes were proposed in this pull request? This PR aims to fix bug Connect should insensitive for JDBC options. Please refer to the comments. apache#50059 (comment) In fact, the built-in Scala API ensures these parameters are lowercase. https://github.com/apache/spark/blob/b18b956f967038db4b751a3845154f2b1d4f5f79/sql/connect/common/src/main/scala/org/apache/spark/sql/connect/DataFrameReader.scala#L126 ### Why are the changes needed? Fix bug Connect should insensitive for JDBC options. ### Does this PR introduce _any_ user-facing change? 'Yes'. Restore the original behavior. ### How was this patch tested? GA. ### Was this patch authored or co-authored using generative AI tooling? 'No'. Closes apache#51068 from beliefer/SPARK-52384. Authored-by: beliefer <beliefer@163.com> Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
1 parent 48ed73b commit ecb181e

File tree

1 file changed

+5
-5
lines changed

1 file changed

+5
-5
lines changed

sql/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1516,20 +1516,20 @@ class SparkConnectPlanner(
15161516

15171517
case proto.Read.ReadTypeCase.DATA_SOURCE if !rel.getIsStreaming =>
15181518
val reader = session.read
1519+
val localMap = CaseInsensitiveMap[String](rel.getDataSource.getOptionsMap.asScala.toMap)
15191520
if (rel.getDataSource.getFormat == "jdbc" && rel.getDataSource.getPredicatesCount > 0) {
1520-
if (!rel.getDataSource.getOptionsMap.containsKey(JDBCOptions.JDBC_URL) ||
1521-
!rel.getDataSource.getOptionsMap.containsKey(JDBCOptions.JDBC_TABLE_NAME)) {
1521+
if (!localMap.contains(JDBCOptions.JDBC_URL) ||
1522+
!localMap.contains(JDBCOptions.JDBC_TABLE_NAME)) {
15221523
throw InvalidInputErrors.invalidJdbcParams()
15231524
}
15241525

1525-
val url = rel.getDataSource.getOptionsMap.get(JDBCOptions.JDBC_URL)
1526-
val table = rel.getDataSource.getOptionsMap.get(JDBCOptions.JDBC_TABLE_NAME)
1526+
val url = localMap.get(JDBCOptions.JDBC_URL).get
1527+
val table = localMap.get(JDBCOptions.JDBC_TABLE_NAME).get
15271528
val predicates = rel.getDataSource.getPredicatesList.asScala.toArray
15281529
val properties = new Properties()
15291530
properties.putAll(rel.getDataSource.getOptionsMap)
15301531
reader.jdbc(url, table, predicates, properties).queryExecution.analyzed
15311532
} else if (rel.getDataSource.getPredicatesCount == 0) {
1532-
val localMap = CaseInsensitiveMap[String](rel.getDataSource.getOptionsMap.asScala.toMap)
15331533
if (rel.getDataSource.hasFormat) {
15341534
reader.format(rel.getDataSource.getFormat)
15351535
}

0 commit comments

Comments
 (0)