Delta Sharing 0.6.10

We’d like to announce the release of Delta Sharing 0.6.10, which introduces the following improvements.
Improvements:

• Support refresh token to refresh the pre-signed urls correctly for queries on the latest snapshot. (#390)
• Improve error message on unrecognized file from server, and add queryTableVersionIntervalSeconds (#401)

Credits: Charlene Lyu, Lin Zhou

Delta Sharing 1.0.0

We are super excited to announce the release of Delta Sharing 1.0.0, which includes several exciting new features and artifacts.

Delta Format Sharing

In order to support Delta advanced features such as DeletionVectors and ColumnMapping in the Delta Sharing OSS Spark connector, “Delta format sharing” is introduced. The core Delta Sharing functionalities are grouped into the delta-sharing-client library and will be released separately. The response of Delta format sharing will not be processed by delta-sharing-client; instead,advanced Delta Spark library will be leveraged to read the data. We also upgraded the Delta Sharing server to support this. (#334, #335, #337, #338, #370, #374, #375, #376, #377, #378, #388, #389, #392, #393, #395)

JsonPredicate Enhancements (JsonPredicatesV2)

We enhanced JsonPredicates to support predicate-based filtering on all columns and not just partition columns. We also added support for the following schema types: Float, Double, Timestamp. (#349, #352, #368, #394)
We added JsonPredicate support in the pandas connector (#396)

Pagination on Delta Sharing Queries

Pagination on Delta Sharing data access APIs enables control over the number of files returned per Query Table request. This improves the scalability of the Delta Sharing service when working with tables with large file volumes, preventing timeouts and resource limit exceedances. (#352, #353, #354, #356, #360, #362, #366)

Other improvements and fixes

Support the release of Maven artifacts for both Scala 2.12 and Scala 2.13. #333
Support refresh token to refresh the pre-signed urls correctly for queries on the latest snapshot. (#385, #386, #387)
Fix bug where to_pandas doesn't properly handle the case when column names between Delta and parquet metadata are different in casing (#384)
A couple build and test fixes. (#330, #364, #369)
A couple fixes on the PROTOCOL description. #340, #365
Fix an issue in DeltaSharingSource: to double check sortedFetchedFiles contain the correct files and set previousOffset correctly. #328, #344

Credits

Abhijit Chakankar, Charlene Lyu, Kr eis, Lin Zhou, William Chau, andyl-db

Delta Sharing 0.7.5

We’d like to announce the release of Delta Sharing 0.7.5, which introduces the following improvements.
Improvements:

• Fix build errors: use a static Cast init function in all places under spark/ (#371)

Credits: Lin Zhou

Delta Sharing 0.7.4

We’d like to announce the release of Delta Sharing 0.7.4, which introduces the following improvements.
Improvements:

• Fix two build errors: use a static Cast init function and isinstance for type comparison in python (#361)

Credits: Lin Zhou

Delta Sharing 0.7.3

We’d like to announce the release of Delta Sharing 0.7.3, which introduces the following improvements.

Improvements:

• Use JsonRawValue in fasterxml.jackson instead of codehaus.jackson (#358)

Credits: Lin Zhou

Delta Sharing 0.7.2

We’d like to announce the release of Delta Sharing 0.7.2, which introduces the following improvements.

Improvements:

• Upgrade to scala 2.13 and spark 3.3.2 in branch-0.7(#355)

Credits: Lin Zhou

Delta Sharing 0.7.1

We’d like to announce the release of Delta Sharing 0.6.9, which introduces the following buf fixes.

Bug fixes:

• [Branch-0.7] Double check sortedFetchedFiles and set previousOffset (#343)

Credits: Lin Zhou

Delta Sharing 0.6.9

We’d like to announce the release of Delta Sharing 0.6.9, which introduces the following buf fixes.

Bug fixes:

• [Branch-0.6] Double check sortedFetchedFiles and set previousOffset (#342)

Credits: Lin Zhou

Delta Sharing 0.6.8

We’d like to announce the release of Delta Sharing 0.6.8, which introduces the following improvements.

Improvements:

• Support endingVersion in queryTable RPC in delta-sharing 0.6 (#327)

Credits: Lin Zhou

Delta Sharing 0.7.0

We’d like to announce the release of Delta Sharing 0.6.5, which introduces the following improvement and bug fixes.
Improvements:

• Fix for issue #320 (#326)
• Issue ref #269: OAuth 2.0 Credential Format for Delta Sharing Python … (#309)
• Add maxRetryDuration in the retry logic in spark client; consolidate configs; add unit tests (#316)
• Add documentation for integrating R2 (#317)
• feat: copied OpenAPI document from rivebank repo (#300)
• Add an optional expirationTimestamp field for pre signed urls (#312)
• Support get_table_version/get_table_protocol/get_table_metadata in python (#302)
• Support endingVersion in queryTable RPC (#304)

Credits: Abhijit Chakankar, Amsteel, Dima Alberg, Lin Zhou, Tom Zhu, pavan-kumar-chalamcharla