July 27, 2025: This Week in OpenDAL

[github-actions[bot]]

This weekly update is generated by LLMs. You're welcome to join our Discord for in-depth discussions.

---

Overall Activity Summary

The Apache OpenDAL repository has seen vibrant activity over the past week, with contributions spanning bug fixes, feature enhancements, CI improvements, and discussions about future directions. Notable areas of focus include backend refactoring, bindings updates, and new feature proposals. Contributors like @tisonkun, @asukaminato0721, @flaneur2020, and others have been instrumental in driving progress.

---

Key Ongoing Projects

• Backend Enhancements and Fixes:

  • fix: patch futures buffer concurrent #6442 by @tisonkun addresses a critical patch for futures buffer_by_ordered to ensure stability in the absence of upstream releases.
  • feat: allow using object_store as opendal's backend #6283 by @flaneur2020 introduces a new object_store backend, enabling advanced operations like parallel fetching and caching.

• Language Bindings Development:

  • ci(bindings/cpp): cpp test framework #6296 by @asukaminato0721 introduces a C++ test framework for bindings, improving testing capabilities.
  • feat(bindings/ocaml): add more blocking api for ocaml #6238 by @asukaminato0721 adds more blocking APIs for OCaml, expanding functionality for OCaml users.

• Concurrent Read Improvements:

  • fix: throttle concurrent loading #6441 by @satlank fixes a critical issue with back-pressure on concurrent reads, preventing resource exhaustion during large file operations.

• Documentation and CI Updates:

  • docs(release): fix CI workflow links for bindings #6437 by @kingsword09 improves release documentation by fixing CI workflow links for bindings.

---

Priority Items

• Critical Bug Fixes:

  • bug: no back-pressure on concurrent reads with chunks #6440 by @satlank: Addresses the lack of back-pressure on concurrent reads, which can lead to resource exhaustion. This issue is linked to fix: throttle concurrent loading #6441 and requires immediate attention.

• Backend Refactoring:

  • Refactor services like HDFS and HDFS Native to align with the backend.rs and core.rs convention (Tracking issue: Refactor services to use the backend.rs and core.rs convention #5702). This is crucial for maintainability and consistency.

• CI Workflow Migration:

  • Migrate opendal release workflow to crates.io's Trusted Publishing #6439 by @Xuanwo: Migrate the release workflow to crates.io's Trusted Publishing for enhanced security and automation.

---

Notable Discussions

• July 20, 2025: This Week in OpenDAL #6438: July 20, 2025: This Week in OpenDAL:
  A community-driven summary of the week's activities, highlighting progress on v0.54.0, service refactoring, and bindings enhancements.

• Open Vector Access Layer #6426: Open Vector Access Layer (OpenVAL):
  @Xuanwo proposed creating an Open Vector Access Layer to align with the industry's shift toward vector-first storage.

• HTTP operator with custom certificate in python bindings #6418: HTTP Operator with Custom Certificates:
  @bennyz raised challenges in using custom certificates with the HTTP operator in Python bindings, sparking discussions on improving flexibility.

---

Emerging Trends

• Refactoring for Maintainability:
  A strong focus on refactoring services to adopt the backend.rs and core.rs convention is improving code organization and readability.

• Advanced Backend Features:
  The addition of backends like object_store (feat: allow using object_store as opendal's backend #6283) reflects a trend toward enabling advanced operations such as parallel fetching and caching.

• Focus on Bindings:
  Significant efforts are being made to enhance bindings for C++, OCaml, and other languages, broadening OpenDAL's accessibility.

---

Good First Issues

• Implement Write Returns Metadata for all services #5693: Implement Write Returns Metadata for All Services:
  Extend the feature to extract metadata from service responses. This is a straightforward task with clear guidelines, suitable for Rust developers.

• new feature: cpp binding enable feature by user #6250: Enable Feature Selection in C++ Bindings:
  Implement feature toggles for C++ bindings, similar to work done for other languages. Ideal for contributors familiar with C++.

• new feature: publish the source and lock files for binaries (oli, oay...) #6198: Publish Source and Lock Files for Binaries:
  Publish source code and lock files for binaries like oli and oay to facilitate packaging for distributions like NixOS.

• Tracking issue: Refactor services to use the backend.rs and core.rs convention #5702: Refactor Services to Use backend.rs and core.rs Convention:
  Pick a service like HDFS or HDFS Native and refactor it to align with the new convention. A great way to learn about OpenDAL's architecture.

• Tracking issues of RFC-5479: Context #5480: Tracking Issues of RFC-5479: Context:
  Help implement context propagation across services. This is a beginner-friendly task with clear steps outlined in the RFC.

---

Join the community and contribute to these exciting initiatives!