|
| 1 | + |
| 2 | +# Android |
| 3 | + |
| 4 | +## Build on Android using Termux |
| 5 | +[Termux](https://github.com/termux/termux-app#installation) is a method to execute `llama.cpp` on an Android device (no root required). |
| 6 | +``` |
| 7 | +apt update && apt upgrade -y |
| 8 | +apt install git make cmake |
| 9 | +``` |
| 10 | + |
| 11 | +It's recommended to move your model inside the `~/` directory for best performance: |
| 12 | +``` |
| 13 | +cd storage/downloads |
| 14 | +mv model.gguf ~/ |
| 15 | +``` |
| 16 | + |
| 17 | +[Get the code](https://github.com/ggerganov/llama.cpp#get-the-code) & [follow the Linux build instructions](https://github.com/ggerganov/llama.cpp#build) to build `llama.cpp`. |
| 18 | + |
| 19 | +## Building the Project using Android NDK |
| 20 | +Obtain the [Android NDK](https://developer.android.com/ndk) and then build with CMake. |
| 21 | + |
| 22 | +Execute the following commands on your computer to avoid downloading the NDK to your mobile. Alternatively, you can also do this in Termux: |
| 23 | +``` |
| 24 | +$ mkdir build-android |
| 25 | +$ cd build-android |
| 26 | +$ export NDK=<your_ndk_directory> |
| 27 | +$ cmake -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-23 -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod .. |
| 28 | +$ make |
| 29 | +``` |
| 30 | + |
| 31 | +Install [termux](https://github.com/termux/termux-app#installation) on your device and run `termux-setup-storage` to get access to your SD card (if Android 11+ then run the command twice). |
| 32 | + |
| 33 | +Finally, copy these built `llama` binaries and the model file to your device storage. Because the file permissions in the Android sdcard cannot be changed, you can copy the executable files to the `/data/data/com.termux/files/home/bin` path, and then execute the following commands in Termux to add executable permission: |
| 34 | + |
| 35 | +(Assumed that you have pushed the built executable files to the /sdcard/llama.cpp/bin path using `adb push`) |
| 36 | +``` |
| 37 | +$cp -r /sdcard/llama.cpp/bin /data/data/com.termux/files/home/ |
| 38 | +$cd /data/data/com.termux/files/home/bin |
| 39 | +$chmod +x ./* |
| 40 | +``` |
| 41 | + |
| 42 | +Download model [llama-2-7b-chat.Q4_K_M.gguf](https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/blob/main/llama-2-7b-chat.Q4_K_M.gguf), and push it to `/sdcard/llama.cpp/`, then move it to `/data/data/com.termux/files/home/model/` |
| 43 | + |
| 44 | +``` |
| 45 | +$mv /sdcard/llama.cpp/llama-2-7b-chat.Q4_K_M.gguf /data/data/com.termux/files/home/model/ |
| 46 | +``` |
| 47 | + |
| 48 | +Now, you can start chatting: |
| 49 | +``` |
| 50 | +$cd /data/data/com.termux/files/home/bin |
| 51 | +$./llama-cli -m ../model/llama-2-7b-chat.Q4_K_M.gguf -n 128 -cml |
| 52 | +``` |
| 53 | + |
| 54 | +Here's a demo of an interactive session running on Pixel 5 phone: |
| 55 | + |
| 56 | +https://user-images.githubusercontent.com/271616/225014776-1d567049-ad71-4ef2-b050-55b0b3b9274c.mp4 |
0 commit comments