Will Lance support incremental read like Iceberg in the future?

[king4face]

Hi, I've noticed that Lance only supports reading all the data of a specific version. Will it support incremental read like Iceberg (https://iceberg.apache.org/docs/nightly/spark-queries/#incremental-read) in the future?

[yanghua]

Hi @king4face, I am implementing the diff API #3991 that can realize incremental reads between two versions. It contains:

• diff_meta: list transactions between two versions, it is finished feat(rust): support diff_meta api #4404 ;
• diff: diff data between two versions, it is under WIP Support diff api for dataset #3991 , needs some other preparatory work to be done.

Will be back to this feature soon.