Current state of Android support?

[anderspitman]

#160 mentions that Android wasn't supported at the time. It appears that maybe it is now (tier 3) according to https://docs.wasmtime.dev/stability-tiers.html?

I'm trying to build a cross-platform app in Python (see here for full background). My app needs to run ffmpeg for some audio processing. I'm hoping that I can use a WASI build of ffmpeg with wasmtime-py to avoid cross-platform issues, but I need wasmtime-py to run on Android for that to be viable.

[alexcrichton]

I'm not aware of anyone having done this before with wasmtime-py, but in theory you're right that it should be possible. Getting this actually working will likely require frobbing how wasmtime-py is published to somehow upload the android binaries somewhere. I'm not sure how to best do that with PyPI's tags/wheels, but there's some possibly related information in this thread

Basically it's expected that this won't work today due to the wasmtime artifacts not being republished through this repository, and figuring out exactly that publication process will be necessary to unblock this.

[anderspitman]

Thanks @alexcrichton. I've successfully tested wasmtime-py + ffmpeg-wasi on my Linux/x86_64 machine. The results are promising and I'd like to get this working on Android. Give that wasmtime already has Android binaries, I agree figuring out the correct PYPI incantation is probably the crux. I'm currently working on this and will let you know what I learn.

[anderspitman]

One potential problem is that https://github.com/bytecodealliance/wasmtime-py/blob/main/ci/download-wasmtime.py doesn't appear to have support for android.

Actually, I'm a bit surprised wasmtime has separate builds for Android. I believe generic Linux/musl builds generally work fine on Android, except maybe hostname resolution. IIRC when I was running a Golang executable on Android a few years ago I had to use hard-coded DNS servers. Are there other reasons for the Android-specific build?

[alexcrichton]

I'm no Android expert so I can't answer with specifics, but in my limited historical experience it's been that while Android is linux-like it breaks down in enough ways that an entirely separate new target is warranted. That means that while some Linux binaries work ok not all do.

If it works though you can try it out? The dual of that script you linked is here where somehow at runtime Python will need to determine, on Android, "here's where the library is located". I don't know what that detection currently does on Android but it'll need to be kept in sync with the download script.

[mhsmith]

I believe generic Linux/musl builds generally work fine on Android

Not really, because Android has its own C library which isn't binary compatible with musl or glibc. Static executables may work in theory, but current versions of Android don't allow apps to call their own executable files (chaquo/chaquopy#605). So shared libraries are the only feasible way for Python to call native code.

I'm not sure how to best do that with PyPI's tags/wheels

PyPI recently enabled the Android wheel tag format specified here, and I'm working on adding support for that in cibuildwheel (pypa/cibuildwheel#1960). Meanwhile, the best way to produce these wheels is with the Chaquopy build tool.

[anderspitman]

@alexcrichton that helps, thanks. According to PEP 738, sys.platform should be returning android. I'll clone wasmtime-py and hack around on it locally and see what I can get working.

@mhsmith

Static executables may work in theory, but current versions of Android don't allow apps to call their own executable files

Are you certain on that? I remember circa 2021 (so after Android 29) I was able to run static executables, but it required some hacky workarounds like having the filename start with lib and the file had to be in a specific NDK directory, essentially tricking Android into thinking it was a shared library. I may be remembering wrong. Perhaps I was targeting an earlier Android version even though 29 was out?

[anderspitman]

Unfortunately it looks like sys.platform is returning linux on both the Android emulator and a physical Pixel 7. I may still be able to work around this for my local wasmtime but not sure about the general case.

[anderspitman]

Hmm

import platform
print(platform.system())

is showing Linux 5.10.136-android12-9.... Maybe we could do a string comparision and see if it includes "android". Pretty hacky but might work in the short term.

[alexcrichton]

Yeah I think it's reasonable to soup up the detection logic of which dll to open, and if that goes beyond just looking at sys.platform I think that's fine.

[anderspitman]

Ok I got it working on the Android emulator by downloading wasmtime-v32.0.0-x86_64-android.tar.xz and manually pushing _libwasmtime.so into the location expected by BeeWare/Chaquopy. wasmtime is so cool!

Unfortunately I don't have root on my physical device so I can't just copy files around and haven't tested aarch64 yet. I'll see if I can finagle BeeWare into copying the files for me.

[anderspitman]

I managed to hack it together by putting the wasmtime libraries in the BeeWare resources directory and hard-coding wasmtime-py to pick up the files from there. Interestingly it doesn't allow the original _libwasmtime.so filename. Apparently doesn't like the .so extension.

In any case, it's working in both the x86_64 emulator and a physical aarch64 device. I believe the thing left to do is understand Chaquopy well enough to copy the files where it expects.

@mhsmith can you guide me at all on that part? Do I need to make a Chaquopy package for wasmtime-py?

[alexcrichton]

Oh nice! glad to hear it's working 👍

Happy to review any changes on the wasmtime-py side once you're ready too

[mhsmith]

According to PEP 738, sys.platform should be returning android.

Yes, this was introduced in Python 3.13. Before that, it would return linux, and you could detect Android by the existence of sys.getandroidapilevel.

I remember circa 2021 (so after Android 29) I was able to run static executables, but it required some hacky workarounds like having the filename start with lib and the file had to be in a specific NDK directory, essentially tricking Android into thinking it was a shared library.

I haven't tried it, but that may be an option. In which case, the executable wouldn't even need to be static, as long as it was linked against Android's libc.

I believe the thing left to do is understand Chaquopy well enough to copy the files where it expects. @mhsmith can you guide me at all on that part? Do I need to make a Chaquopy package for wasmtime-py?

So you already have an Android build of the native library, and the Python interface accesses it through ctypes, and there's no other native code involved? In that case, the following code should almost work:

wasmtime-py/wasmtime/_ffi.py

Lines 28 to 31 in 5e89132

	filename = Path(__file__).parent / (sys.platform + '-' + machine) / libname
	if not filename.exists():
	raise RuntimeError("precompiled wasmtime binary not found at `{}`".format(filename))
	dll = cdll.LoadLibrary(str(filename))

The only issue is that to help startup performance, Chaquopy doesn't extract .so files from the APK until they're needed, so the exists check will fail. The code should be changed to simply call LoadLibrary and then check whether it worked.

You'll also need to package it into a .whl file with the correct tag, depending on what Android API level the native library was built against. Since a few months ago, these tags are now accepted on PyPI.

It should then be possible to install the .whl file into an app using Chaquopy or Briefcase, without any manual moving of files.

[alexcrichton]

Wasmtime does indeed fall into the case of having precompiled binaries and only using ctypes with no other native code (to confirm). I think it'd be reasonable to unconditionally LoadLibrary and then catch exceptions and perhaps add more metadata to the exception if the file doesn't exist. (or something like that)

For the wheel tags the definition of Rust targets are here so looks like we'd correspond to android_27_arm64_v8a and android_27_x86_64

[mhsmith]

The 27 in the NDK version number isn't an API level, it's just an arbitrary version number. I believe the minimum API level supported by the current NDK is actually 21.

[anderspitman]

Excellent. I'm happy to make the necessary changes to wasmtime-py and submit a PR.

@mhsmith sounds like the best (only?) way currently to build Android package wheels is using the chaquopy build tool?

[mhsmith]

In this case there's no native code to build, so the only reason to use the Chaquopy build tool is to set the wheel tag. Depending on your build system, there may be an easier way of doing that.

[anderspitman]

Looks like a pretty basic setup.py to me. Does setuptools have this functionality?

[alexcrichton]

Currently wheels are built with tags here

[anderspitman]

Started a PR here: #279

Currently only building and publishing is implemented. Still working on importing.

One problem with the current PR is the Android API version is hard-coded to 26. I came up with that number by running file _libwasmtime.so on the Android build. @mhsmith do you know if there's a good way to determine this from the shared library at build time in CI? I might be able to use the file command but that feels a bit hacky.

[alexcrichton]

Above @mhsmith pointed out that the Rust targets are likely API level 21, but @anderspitman how did you use file? I ran file over the *.so and it just spit out a bland "x86_64 so" without any API version levels in there from what I could see

[anderspitman]

This is what file (version 5.46) gives me:

file wasmtime/android-aarch64/_libwasmtime.so
wasmtime/android-aarch64/_libwasmtime.so: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, for Android 26, built by NDK r27c (12479018), not stripped

[mhsmith]

This must be a recent addition to file; it doesn't appear with version 5.44:

ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, BuildID[sha1]=56fce7cd483cd72d2e88c9e4818de39f81c9a614, with debug_info, not stripped

I don't know any other way of getting this information from the library; in fact, I wasn't expecting it to be in there at all. But I guess it's probably accurate.

[anderspitman]

Sounds like we probably better not rely on that information. @alexcrichton might it be possible to include the API version in the filename when the upstream Android artifacts are built?

[mhsmith]

The relevant code in file looks fine to me, so if you can get this information by running file as part of the build, then I see no reason not to rely on it.

[alexcrichton]

I'd also be ok avoiding auto-detection and instead having a CI job that checks "the API version is 26, right?" and fails. That would be a nudge to update it if the binary artifact ever changes without having to integrate everything into one process. If you can figure out how to integrate it though then that also seems reasonable, whichever you'd prefer