example to build aarch64-linux-android

[aesthetic-factory]

Hi, I've multiple attempts to cross-compile it for Android ARM.
I can cross-compile standalone llama cpp on my Linux desktop following the guide.
https://github.com/ggml-org/llama.cpp/blob/master/docs/android.md

But I cannot find a way to build it with Rust.
These are the commands I used.

cargo build --target aarch64-linux-android --no-default-features
cargo ndk -t arm64-v8a --release

I've also tried different build options and set different env variables but still no luck.

[MarcusDunn]

what errors are you getting?

[aesthetic-factory]

I can see 2 issues so far.
The first one is related to OpenSSL.
I am on Fedora Linux, and I am pretty sure pkgconf-pkg-config and openssl-devel are installed.

  Could not find openssl via pkg-config:
  pkg-config has not been configured to support cross-compilation.

  Install a sysroot for the target platform and configure it via
  PKG_CONFIG_SYSROOT_DIR and PKG_CONFIG_PATH, or install a
  cross-compiling wrapper for pkg-config and set it via
  PKG_CONFIG environment variable.

  cargo:warning=Could not find directory of OpenSSL installation, and this `-sys` crate cannot proceed without this knowledge. If OpenSSL is installed and this crate had trouble finding it,  you can set the `OPENSSL_DIR` environment variable for the compilation process. See stderr section below for further information.

  --- stderr

For the other one, I guess the main problem is that it cannot find the correct compiler.

  TARGET_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
  cargo:warning=Compiler family detection failed due to error: ToolNotFound: failed to find tool "aarch64-linux-android-clang": No such file or directory (os error 2)
  RUSTC_WRAPPER = None
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some(true)
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None
  cargo:rerun-if-env-changed=TARGET_CFLAGS
  TARGET_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS_aarch64_linux_android
  CFLAGS_aarch64_linux_android = None
  cargo:rerun-if-env-changed=CFLAGS_aarch64-linux-android
  CFLAGS_aarch64-linux-android = None
  CARGO_ENCODED_RUSTFLAGS = Some()
  cargo:warning=Compiler family detection failed due to error: ToolNotFound: failed to find tool "aarch64-linux-android-clang": No such file or directory (os error 2)

  --- stderr

  error occurred in cc-rs: failed to find tool "aarch64-linux-android-clang": No such file or directory (os error 2)

I've also tried something like this.

export CMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake
cargo build --target aarch64-linux-android

  --- stderr
  /usr/include/bits/floatn.h:97:9: error: __float128 is not supported on this target

  thread 'main' panicked at llama-cpp-sys-2/build.rs:248:10:
  Failed to generate bindings: ClangDiagnostic("/usr/include/bits/floatn.h:97:9: err

[MarcusDunn]

are you able to build llama.cpp by itself?

[aesthetic-factory]

Yes I can.
I just need to set $ANDROID_NDK and use this command.

cmake \
  -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
  -DANDROID_ABI=arm64-v8a \
  -DANDROID_PLATFORM=android-28 \
  -DCMAKE_C_FLAGS="-march=armv8.7a" \
  -DCMAKE_CXX_FLAGS="-march=armv8.7a" \
  -DGGML_OPENMP=OFF \
  -DGGML_VULKAN=OFF \
  -DLLAMA_CURL=OFF \
  -B build-android
  
  
cmake --build build-android --config Release -j16

[MarcusDunn]

A PR would be welcome with a feature flag to enable the appropriate flags. The build script calls to cmake, so you should be able to reproduce the successful build without too much trouble.

[4j17h]

I’ve successfully resolved the Android build errors (PR - #782), so the library now builds cleanly for all supported ABIs @aesthetic-factory , you can test it out.

However, I’m now blocked by the Uniffi (https://github.com/jhugman/uniffi-bindgen-react-native) Rust bindings on Android, calling LlamaModel::create() still triggers a SIGABRT inside the panic unwind machinery even after applying all C++ mmap and backtrace patches. It turns out Uniffi’s catch_unwind can’t recover from a C++ abort(), so my attempt to use FFI remains unstable on Android