Building llama.cpp for Android as a .so library

[samolego]

Hello there, for the past week I've been trying to make llama.cpp use clblast in my android app (I'm using modified version of java bindings; https://github.com/kherud/java-llama.cpp). I don't really know much about cmake and make, and I'm wondering how would I trigger the build in android studio to get similar results to this: https://github.com/ggerganov/llama.cpp#building-the-project-using-termux-f-droid

So far I've tried:

...
# checkout llama.cpp
include(FetchContent)
FetchContent_Declare(
        llama.cpp
        GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git
        GIT_TAG b1792
)
FetchContent_MakeAvailable(llama.cpp)

# checkout CLBlast
FetchContent_Declare(
        CLBlast
        GIT_REPOSITORY https://github.com/CNugteren/CLBlast.git
        GIT_TAG 1.6.1
)
FetchContent_Populate(CLBlast)

# Copy include/clblast.h from CLBlast to llama.cpp root directory todo
configure_file(${clblast_SOURCE_DIR}/include/clblast.h ${llama.cpp_SOURCE_DIR}/clblast.h COPYONLY)


# Copy headers from OpenBLAS to llama.cpp root directory (header files are currently copied directly from my device)
configure_file(../cblas.h ${llama.cpp_SOURCE_DIR}/cblas.h COPYONLY)
configure_file(../openblas_config.h ${llama.cpp_SOURCE_DIR}/openblas_config.h COPYONLY)

# Set LLAMA_CLBLAST=1 for llama.cpp
set(LLAMA_CLBLAST 1)
add_definitions(-DLLAMA_CLBLAST=1)
#target_compile_definitions(llama PRIVATE LLAMA_CLBLAST=1)

...

I manually built the clblast library on device but it seems it's not using it ...

Does anybody know what am I missing here? Thanks :)

[bullno1]

My experience many moons ago:

• Use the NDK toolchain file: https://developer.android.com/ndk/guides/cmake#file
• Just use git submodule + add_subdirectory, you own your project.
  This also allows you to override defaults you don't want.
• Always use EXCLUDE_FROM_ALL when adding 3rd party code.
  You don't want to build their tests or utilities you don't care about.
• Don't rely Android Studio when it comes to cmake or ndk.
  Build the .so separately, write the JNI/JNA binding manually.
  The support has always been terrible.

I manually built the clblast library on device but it seems it's not using it ...

What does "not using it" mean exactly? Was the .so loaded by JNI?

[danemadsen]

Hi, I'm currently building llama.cpp as a .so for android in my app MAID (Mobile Artificial Intelligence Distribution).

https://github.com/Mobile-Artificial-Intelligence/maid

Im thinking i may stop supporting native llama.cpp though as its takes a lot of my time to keep working.