Naive and initial thoughts on WebAssembly modding

[afonsolage]

I'm toying with modding on WebAssembly and I'm having a hard time figuring out all pieces together. I'm not experienced with WebAssembly, besides having a good understand on how it works.

So I decided to write all those thoughts somewhere to help me organize my mind and maybe this helps someone if the future (me included).

Runtimes

So far working with wasmtime is very easy, the hardest part is getting WebAssembly modding working on it's own home: the Browser.

When working with wasmtime (and also wasmer) it's easy to understand the workflow, you setup a wasm runtime, load modules, inject some imports, compile the module, instantiate it and call the methods. It's fair easy to get this working if you have any previous experience working with embedded runtime.

BUT when targeting wasm32-unknown-unknown the game IS also a wasm module. So here is where things starts to get tricky. What we are trying to do is something like:

1. JavaScript init the game wasm module;
2. Game wasm module asks for JavaScript to download a mod wasm module;
3. JavaScript send the bytes to the Game wasm module;
4. Game wasm module asks JavaScript to compile/instantiate the mod wasm module;
5. Game wasm module asks JavaScript to inject imports;
6. Game wasm module asks JavaScript to instantiate the mod wasm module;
7. Game wasm module asks JavaScript to call the method;

Luckily for us, steps 1~3 are handled by bevy_assets, so it's easy to get it done.
For step 4 there is a OK-ish example on how to do that.
Step 6 and 7 are easy to do.

So where is the struggle? It's on step 5 - inject imports dependencies.
Any modding API needs to provide, well, an API for modules to use, so how to inject those methods calls, from Game wasm module, on another WebAssembly module? Still don't know, but I bet probably this needs to be some JavaScript bindings, so in order to inject those binds, one would need to bind from Rust to JavaScript and from JavaScript to WebAssembly module, seems easy, right?

well...

wasm32-unknown-unknown vs wasm32-wasi

The problem is with regards to WASI. My first attempt was trying to build a mod targeting wasm32-wasi, since any modding would benefit from having access to IO related features, like stdin, stdout, stderr and maybe stdio. Using stdin/stdout to communicate with mods seems a good approach, much like Lapce does

In order for that work, I need to provide the WASI bindings to the wasm module. Again, using wasmtime that was a piece of cake, but when targeting wasm32-unknown-unknown things didn't go that well.

wasm_bindgen doesn't work with wasi

Even tho cargo-wasi supports wasm_bindgen, which detects if the crate uses wasm-bindgen and auto call it for us, the opposite doesn't apply: When trying to generate bindings with wasm_bindgen targeting wasm32-wasi, it panics:

thread 'main' panicked at 'unknown instruction LocalTee(LocalTee { local: Id { idx: 38797 } })', crates\wasm-interpreter\src\lib.rs:363:18
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

And it won't be fixed until Component Model is out.

wasmtime support doesn't wasm_bindgen anymore

Another problem is that wasmtime dropped the support for wasm_bindgen until Component Model is also out.

This makes very hard to develop an API that targets both web and native, since for the former wasm_bindgen is require, while for the latter, a custom API bindings needs to be used.

Final naive and initial thoughts

Since modding for bevy only makes sense if wasm32-unknown-unknown is also targeted, WASI won't help us here. Also, the very immature state of WebAssembly outside the browser, makes it very hard to build an API that works for both targets.

Next steps

The obvious conclusion here would be to wait for Component Model gets done and wait for wasm_bindgen supports WASI and wasmtime supports wasm_bindgen, this would make things A LOT easier, but I will do more researches and check if there is some workaround for the problems I listed.

I'm hopping also someone hardly disagrees with me and shows me that it can be done in other ways.

Anyway, thanks for reading and any comments, suggestions and thoughts are appreciated!

[bjorn3]

wasm-bindgen has never been supported outside of the browser or node. It's purpose is to bind to javascript api's. For other use cases there is wit-bindgen. It targets both the component model and MVP wasm using the canonical abi. It can generate both the host and guest side for rust and the host side for javascript.

[afonsolage]

I agree, but in order to use wit-bindgen with bevy, which uses wasm-bindgen, I would have to find a way to "inject" wit-bindgen functions inside the game mod (which, isn't the host on the web).

info: func(message: string)

When exporting this wit file, the JS host expects the game wasm module to have also an export named __wbindgen_realloc, so it can do proper String allocation, but wasm-bindgen doesn't generate this export, since it's a wit-bindgen thing. For that to work, cargo wasi would have to be able to "inject" both wit-bindgen and wasm-bindgen export bindings on the same file, which doesn't.

So that's the main problem with wit-bindgen today, I can't find a way to make a module have both bindings, since else, there is no way to use wit files, AFAIK.

What I'm trying to do now is to have two APIs, one for wasm-bindgen only and another for wit-bindgen only, but this will be hard to achieve, since the final API for any modding module, must match, so I would need to have a WebProxyAPI which would forward calls to wasm-bindgen bindings and a NativeProxyAPI which would forward calls to wit-bindgen bindings, and I think that would add a overhead, but without this proxy, I can't see how a guest modding module would, for instance, send/receive strings from/to host, be it the JS or wasmtime.

[afonsolage]

A follow-up on those thoughts. It was meant to be just a quick summary, but I ended writing a lot 😅

TL;DR Prototype

I was able to find a way to communicate between host and guest across native and web. I'm working on a experimental project here: https://github.com/afonsolage/wabi

It's very unstable and shouldn't be used, yet.

Client-server like

Everything was easier to understand and to wrap my mind around when I finally understood that host and guest communication is just like a client/server approach (Thanks @erlend-sh for pointing me to Veloren modding ), where they know nothing about each other, except what is defined in a protocol. The difference is that on host/guest communication the latency is in microseconds¹, while on a networking, it is in milliseconds¹, so the challenge is to build an API which use few contexts switching as possible and allow a seamless integration with Bevy.

The way they share data is writing/reading to/from a WebAssembly Memory, but unlike client/server, we can pass arguments and receive return values.

Sync communication

Wasm mods are intended to run in a single-thread, since there is no use case to run a multi-threaded wasm mod² (correct me if I'm wrong here), so we can use this to our advantage and make a simple sync communication workflow between host/guest:

1. Host loads guest;
2. Host calls an allocation function on guest;
3. Guest allocates a buffer and return the addr to host;
4. Host write some state on the guest memory, at the given addr, and send the written length to Guest;
5. Guest reads the state on buffer, using the length, and do some stuff with it;
6. Guest writes some state on buffer and send an action code and the state len to host;
7. Host reads the state on guest memory, at given addr, using the received length and do some work;
8. If there is a response to be sent, repeat 4.

The communication flow is much like a normal function call, except there is an extra step, which is sending arguments and returning result using the wasm memory.

So the communication between guest/host is understood. Now it's the API time!

Neutral API

The hardest part has been developing an API which isn't Rust biased, since one of the main selling points of wasm is to enable multi-language targeting, modders without knowledge of Rust, must be able to use and develop games with Bevy.

That's another point where previous experience with networking protocols is handy, since usually you can make a client in one language and server in another, it's just a matter of protocol neutrality.

Many crates complexity

Another downside of developing a mod for modding in Bevy, using wasm, is because we need many crates, due to high flexibility of this scenario:

1. wabi - The main mod crate, this is the one Bevy users will add to their game, to allow modding using wasm.
2. wabi_runtime_api - Contains the API to be implemented by runtimes. Atm there are two runtimes: wasmtime and wasm_bindgen, but since wasm is still in early stages, I decided to separate the runtime API, so if we would like to add support for wasmer, wasm3 or any other future runtimes, just implement this api.
3. wabi_wasmtime - The runtime API implementation for wasmtime, which is used in native builds (Android and iOS isn't supported);
4. wabi_wasm_bindgen - The runtime API implementation for wasm_bindgen, which is used in web builds;
5. wabi_mod_api - This is the neutral mod API (protocol) which will be used by any wasm mod which will interact with Bevy. It is separated so it'll be easier to have a documentation for non-Rust implementations, like MessagePack or even JSON description of messages, actions and events.
6. wabi_mod_impl - This is defacto the rust wasm implementation of wabi_mod_api. For now it's just a test crate, but in the future, it'll be a wrapper arround wabi_mod_api to make the UX as close as possible to vanilla Bevy.

This is what I've come up with atm, crates may be merged on new one added and names may change before the first release on crates.io (even tho I loved the name already 😬 )

Next steps

Now it's time to try to refine the API and add Bevy features like Components, Commands, Res, ResMut, Query and so on. My first goal is to recreate the breakout example in a wasm mod, oc bootstraping the App is out of scope of this wasm mod, but all systems, including startup_systems are included.

---

¹ This was a very rough estimation that I made when benchmarking the context switch latency between host/guest using wasmtime and JS on Chrome. A simple call, which write to buffer, call function on wasm and read memory back, took nearly ~100μs in an optimized build (host and mod).
² Even tho there are use cases for a WebAssembly module to use multi-threading, like using Web Workers, this is usually needed when doing IO, and Bevy already handle this.

[anchpop]

The hardest part has been developing an API which isn't Rust biased, since one of the main selling points of wasm is to enable multi-language targeting, modders without knowledge of Rust, must be able to use and develop games with Bevy.

What languages have you been designing for besides Rust? I didn't know there were that many other language that could do an okay job targeting Wasm.

[afonsolage]

Since the goal is to support WebAssembly, pretty much any language that supports it. This include some notable ones:

• C#
• F#
• C++
• Kotlin (Java)
• Go
• Zig
• Pascal
• Swift

And even "wasm native" ones, like AssemblyScript

I don't plan tho to fully support all those languages in wabi project, since this will be a massive effort, instead I'll likely have a proof of concept, like implementing a full game (breakout?) in a popular language, like C# or Kotlin, just to ensure that it is possible to use any wasm module in Bevy.

[afonsolage]

I made a follow-up to those thoughts, narrowing down to a more concrete implementation. There are still many things to tryout, but I'm slowly getting there.

afonsolage/wabi#1