You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -712,6 +713,8 @@ Building the program with BLAS support may lead to some performance improvements
712
713
713
714
To obtain the official LLaMA 2 weights please see the <a href="#obtaining-and-using-the-facebook-llama-2-model">Obtaining and using the Facebook LLaMA 2 model</a> section. There is also a large selection of pre-quantized `gguf` models available on Hugging Face.
714
715
716
+
Note: `convert.py` does not support LLaMA 3, you can use `convert-hf-to-gguf.py` with LLaMA 3 downloaded from Hugging Face.
717
+
715
718
```bash
716
719
# obtain the official LLaMA model weights and place them in ./models
717
720
ls ./models
@@ -933,25 +936,35 @@ If your issue is with model generation quality, then please at least scan the fo
933
936
934
937
### Android
935
938
936
-
#### Building the Project using Android NDK
937
-
You can easily run `llama.cpp` on Android device with [termux](https://termux.dev/).
939
+
#### Build on Android using Termux
940
+
[Termux](https://github.com/termux/termux-app#installation) is a method to execute `llama.cpp` on an Android device (no root required).
941
+
```
942
+
apt update && apt upgrade -y
943
+
apt install git make cmake
944
+
```
938
945
939
-
First, install the essential packages for termux:
946
+
It's recommended to move your model inside the `~/` directoryforbest performance:
940
947
```
941
-
pkg install clang wget git cmake
948
+
cd storage/downloads
949
+
mv model.gguf ~/
942
950
```
943
-
Second, obtain the [Android NDK](https://developer.android.com/ndk) and then build with CMake:
944
951
945
-
You can execute the following commands on your computer to avoid downloading the NDK to your mobile. Of course, you can also do this in Termux.
952
+
[Get the code](https://github.com/ggerganov/llama.cpp#get-the-code) & [follow the Linux build instructions](https://github.com/ggerganov/llama.cpp#build) to build `llama.cpp`.
946
953
954
+
#### Building the Project using Android NDK
955
+
Obtain the [Android NDK](https://developer.android.com/ndk) and then build with CMake.
956
+
957
+
Execute the following commands on your computer to avoid downloading the NDK to your mobile. Alternatively, you can also do this in Termux:
Install [termux](https://termux.dev/) on your device and run `termux-setup-storage` to get access to your SD card.
965
+
966
+
Install [termux](https://github.com/termux/termux-app#installation) on your device and run `termux-setup-storage` to get access to your SD card (if Android 11+ then run the command twice).
967
+
955
968
Finally, copy these built `llama` binaries and the model file to your device storage. Because the file permissions in the Android sdcard cannot be changed, you can copy the executable files to the `/data/data/com.termux/files/home/bin` path, and then execute the following commands in Termux to add executable permission:
956
969
957
970
(Assumed that you have pushed the built executable files to the /sdcard/llama.cpp/bin path using `adb push`)
Termux from F-Droid offers an alternative route to execute the project on an Android device. This method empowers you to construct the project right from within the terminal, negating the requirement for a rooted device or SD Card.
982
-
983
-
Outlined below are the directives for installing the project using OpenBLAS and CLBlast. This combination is specifically designed to deliver peak performance on recent devices that feature a GPU.
984
-
985
-
If you opt to utilize OpenBLAS, you'll need to install the corresponding package.
986
-
```
987
-
apt install libopenblas
988
-
```
989
-
990
-
Subsequently, if you decide to incorporate CLBlast, you'll first need to install the requisite OpenCL packages:
In order to compile CLBlast, you'll need to first clone the respective Git repository, which can be found at this URL: https://github.com/CNugteren/CLBlast. Alongside this, clone this repository into your home directory. Once this is done, navigate to the CLBlast folder and execute the commands detailed below:
996
-
```
997
-
cmake .
998
-
make
999
-
cp libclblast.so*$PREFIX/lib
1000
-
cp ./include/clblast.h ../llama.cpp
1001
-
```
1002
-
1003
-
Following the previous steps, navigate to the LlamaCpp directory. To compile it with OpenBLAS and CLBlast, execute the command provided below:
make LLAMA_CLBLAST=1 //(sometimes you need to run this command twice)
1008
-
```
1009
-
1010
-
Upon completion of the aforementioned steps, you will have successfully compiled the project. To run it using CLBlast, a slight adjustment is required: a command must be issued to direct the operations towards your device's physical GPU, rather than the virtual one. The necessary command is detailed below:
(Note: some Android devices, like the Zenfone 8, need the following command instead - "export LD_LIBRARY_PATH=/system/vendor/lib64:$LD_LIBRARY_PATH". Source: https://www.reddit.com/r/termux/comments/kc3ynp/opencl_working_in_termux_more_in_comments/ )
1018
-
1019
-
For easy and swift re-execution, consider documenting this final part in a .sh script file. This will enable you to rerun the process with minimal hassle.
1020
-
1021
-
Place your desired model into the `~/llama.cpp/models/` directory and execute the `./main (...)` script.
0 commit comments