July 20, 2025: This Week in OpenDAL

[github-actions[bot]]

This weekly update is generated by LLMs. You're welcome to join our Discord for in-depth discussions.

Overall Activity Summary

The Apache OpenDAL repository has seen robust activity over the past week, including significant contributions to refactor services, enhance bindings, and improve CI workflows. Notable progress was made in the upcoming v0.54.0 release, with discussions and voting on its approval. Several PRs addressed critical bug fixes, feature implementations, and dependency updates across multiple bindings and services.

Key Ongoing Projects

• Release of v0.54.0:

  • The release candidate for v0.54.0 was finalized, and voting concluded successfully. Contributors like @Xuanwo, @geruh, and @yihong0618 played key roles in preparing and verifying the release (Tracking issues of OpenDAL 0.54 Release #6381).
  • The release introduces options-based APIs, removes deprecated services, and enhances bindings for Java, Python, and Node.js.

• Service Refactoring:

  • Refactor efforts continue to align services with the backend.rs and core.rs convention. For example:
    • refactor: Migrate cloudflare_kv from adapter::kv to Access instead #6340 by @kingsword09 refactors the cloudflare_kv service to implement Access directly.
    • refactor: Migrate redis from adapter::kv to Access instead #6291 by @Xuanwo migrates the redis service from adapter::kv to Access.

• Enhancements to Bindings:

  • Node.js bindings received updates for StatOptions, ReadOptions, and WriteOptions (feat(bindings/nodejs): Add WriteOptions support for new options API #6322, feat(bindings/nodejs): Add DeleteOptions support for new options API #6349).
  • Python bindings were enhanced for better error handling and metadata support (fix(bindings/python): Fix Writer doesn't throw correct error code #6315, feat(bindings/python): Enhance Stat, Lister, Metadata & Entry  #6232).

• CI and Dependency Updates:

  • CI workflows were streamlined, including updates to GitHub Actions (chore(github-actions): update 1password #6405 by @erickguan).
  • Dependabot managed multiple dependency updates, such as bumping tokio (chore(deps): bump tokio from 1.45.0 to 1.46.1 in /core in the async-runtime group #6390) and rand (chore(deps): bump the http-serialization-utils group in /core with 7 updates #6389).

Priority Items

• Refactor Services:

  • Services like HDFS and HDFS Native need refactoring to adopt the backend.rs and core.rs convention (Tracking issue: Refactor services to use the backend.rs and core.rs convention #5702). Contributors are encouraged to pick these tasks.

• Bug Fixes:

  • Address the panic issue in listing objects with Huawei Cloud (Panic on listing objects with huaweicloud #6419). This requires immediate attention to ensure service reliability.

• Dependency Management:

  • Exclude crates like rand and getrandom from Dependabot updates to avoid breaking changes (chore(dependabot): exclude rand and getrandom crates #6397 by @erickguan).

Notable Discussions

• Open Vector Access Layer (Open Vector Access Layer #6426): @Xuanwo proposed the idea of creating an Open Vector Access Layer (OpenVAL) to align with the industry's move toward vector-first storage.
• GDrive List Performance Issues (GDrive list weirdness #6371): @funlennysub raised concerns about slow listing performance and shared folder visibility in the GDrive service.
• HTTP Operator with Custom Certificate (HTTP operator with custom certificate in python bindings #6418): @bennyz discussed challenges in using custom certificates with the HTTP operator in Python bindings.

Emerging Trends

• Options-Based APIs: The introduction of structured options for operations like read, write, and list is becoming a standard across bindings and services.
• Refactoring for Maintainability: The push to refactor services into backend.rs and core.rs is improving code readability and maintainability.
• Focus on Bindings: Significant efforts are being made to enhance bindings for Java, Python, and Node.js, making OpenDAL more accessible across languages.

Good First Issues

• Implement Write Returns Metadata for All Services (Implement Write Returns Metadata for all services #5693): This involves extracting metadata from service responses. It’s a straightforward task with clear guidelines.
• Enable Feature Selection in C++ Bindings (new feature: cpp binding enable feature by user #6250): Contributors can help implement feature toggles for C++ bindings, similar to work done for other languages.
• Publish Source and Lock Files for Binaries (new feature: publish the source and lock files for binaries (oli, oay...) #6198): This task involves publishing source code and lock files for binaries like oli and oay, making them easier to package for distributions like NixOS.
• Refactor Services to Use backend.rs and core.rs Convention (Tracking issue: Refactor services to use the backend.rs and core.rs convention #5702): Pick a service like HDFS or HDFS Native and refactor it to align with the new convention.

---

Join the community and contribute to these exciting initiatives!