How to design opendal c binding's async api

[asukaminato0721]

c users have some choices. The main problem is c doesn't have async/await/yield keyword.

• use callback with libuv, libevent
• Manual State Machines
• Macro-based "Coroutines" / Protothreads (something like https://dunkels.com/adam/pt/ or modern version https://github.com/naasking/async.h)
• Coroutine Libraries (https://github.com/semistrict/libcoro or https://github.com/hnes/libaco)

all seems complicated...

the async io part seems also complicated...

---

Refer to #6072 for more context.

[asukaminato0721]

2 ideas:

1. Callback-based API (very common, but call back hell)
2. Event Loop Integration (integrate into libuv, libevent or other framework)

[yuchanns]

Cool. I'm looking forward to it. Go binding can be beneficial from this.

[asukaminato0721]

I made a "callback" version in the PR with test and example, feed back welcome.

[Xuanwo]

Some other things to consider:

• How should we handle the existing tokio code?
• Should we consider splitting the library into a blocking version and an async version?

I can imagine that users who want to use the async API don't want to have tokio included.

---

Or maybe the biggest question is: Where are our C users, and what do they want?

[kpole]

Hi, xuanwo, oceanbase currently uses the blocking version of cbinding and wants to implement async version to reduce the overhead introduced by threading.
In response to your question, I have the following thoughts：

1. As far as I know, tokio has very good asynchronous IO capabilities, and I prefer tokio over writing a C asynchronous framework from scratch
2. Does "splitting the library into a blocking version and an async version" refer to creating two independent libraries, or does it mean two separate sets of APIs?
   • The former might lead to redundant inclusion, especially since OceanBase is introduced in the form of RPM packages, which include the Rust standard library implementation.
   • I believe the latter is feasible, as async IO and sync IO have many differences in interface parameters and significant implementation variations.

[Xuanwo]

For 1, so you prefer to use tokio directly instead of exposing callbacks and relying on the async runtime from the C side?

For 2, yes, having two sets of APIs makes sense to me.

[kpole]

yes, this will cut down the expenses of using the cbinding async framework, at least for what we're doing.

[Xuanwo]

Here lies the issue: the existing blocking C API already uses tokio under the hood. Could you share some insights into your design for the async C API?

[kpole]

That's a good question, the existing blocking C API use tokio, but block_on doesn't use tokio's runtime, instand, it operates on the current thread. In contrast, async IO execute at runtime and communicates with external through callbacks or event loops.

I initially thought sync io would be built on top of async io. However, for now, sync io seems more stable, and I prefer to provide a separate api to implement async io for produciton use.