Getting worklets into vision-camera

[hannojg]

Hey everyone 👋

Now that worklets is becoming a standalone package we are reevaluating the usage of worklets over our worklets-core package to be used in VisionCamera (and other libraries).

The one thing we see missing to make this happen is an Api that allows us to pass our own owned thread.
In VisionCamera we need to execute on certain DispatchQueues (Swift) / Coroutines (Kotlin).

However, we see that worklets is creating the thread on its own here:

react-native-reanimated/packages/react-native-worklets/Common/cpp/worklets/Tools/AsyncQueue.cpp

Line 9 in 2af7a20

auto thread = std::thread([name, state = state_] {

---

One approach we have in mind to make this happen is to allow for a new argument to be passed to createWorkletRuntime, which could be HostObject that implements worklets::AsyncQueue.
We make worklets::AsyncQueue abstract/virtual, and you can provide a default implementation.
We then can take care on our side of implementing / bridging this to our DispatchQueues / Coroutines.

[hannojg]

@tomekzaw @mrousavy

[tjzel]

Those were our plans too - however, the (C++) code that handles runtime creation is quite volatile now as we're still tweaking the pipeline. Is JS API sufficient for you now, or do you need C++ as well?

[hannojg]

@mrousavy correct me if I am wrong, but being able to provide a HostObject from JS would be sufficient for now (long term a nice C++ API sounds good as well, but for now we should be fine)

[mrousavy]

Agreed yes - we just need a stable AsyncQueue virtual base class (a jsi::HostObject) that we can implement with our DispatchQueue/Coroutine stuff. This is something we pass to JS, and ultimately to your JS Worklet creation API.

[tjzel]

I open a draft, tell me if it works for you: #7851