Skip to content

Commit be20e7f

Browse files
authored
Reorganize documentation pages (#8325)
* re-organize docs * add link among docs * add link to build docs * fix style * de-duplicate sections
1 parent 7ed03b8 commit be20e7f

File tree

14 files changed

+626
-603
lines changed

14 files changed

+626
-603
lines changed

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@ build*
4747
!build-info.cpp.in
4848
!build-info.sh
4949
!build.zig
50+
!docs/build.md
5051
/libllama.so
5152
/llama-*
5253
android-ndk-*

README.md

Lines changed: 69 additions & 599 deletions
Large diffs are not rendered by default.

docs/android.md

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
2+
# Android
3+
4+
## Build on Android using Termux
5+
[Termux](https://github.com/termux/termux-app#installation) is a method to execute `llama.cpp` on an Android device (no root required).
6+
```
7+
apt update && apt upgrade -y
8+
apt install git make cmake
9+
```
10+
11+
It's recommended to move your model inside the `~/` directory for best performance:
12+
```
13+
cd storage/downloads
14+
mv model.gguf ~/
15+
```
16+
17+
[Get the code](https://github.com/ggerganov/llama.cpp#get-the-code) & [follow the Linux build instructions](https://github.com/ggerganov/llama.cpp#build) to build `llama.cpp`.
18+
19+
## Building the Project using Android NDK
20+
Obtain the [Android NDK](https://developer.android.com/ndk) and then build with CMake.
21+
22+
Execute the following commands on your computer to avoid downloading the NDK to your mobile. Alternatively, you can also do this in Termux:
23+
```
24+
$ mkdir build-android
25+
$ cd build-android
26+
$ export NDK=<your_ndk_directory>
27+
$ cmake -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-23 -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod ..
28+
$ make
29+
```
30+
31+
Install [termux](https://github.com/termux/termux-app#installation) on your device and run `termux-setup-storage` to get access to your SD card (if Android 11+ then run the command twice).
32+
33+
Finally, copy these built `llama` binaries and the model file to your device storage. Because the file permissions in the Android sdcard cannot be changed, you can copy the executable files to the `/data/data/com.termux/files/home/bin` path, and then execute the following commands in Termux to add executable permission:
34+
35+
(Assumed that you have pushed the built executable files to the /sdcard/llama.cpp/bin path using `adb push`)
36+
```
37+
$cp -r /sdcard/llama.cpp/bin /data/data/com.termux/files/home/
38+
$cd /data/data/com.termux/files/home/bin
39+
$chmod +x ./*
40+
```
41+
42+
Download model [llama-2-7b-chat.Q4_K_M.gguf](https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/blob/main/llama-2-7b-chat.Q4_K_M.gguf), and push it to `/sdcard/llama.cpp/`, then move it to `/data/data/com.termux/files/home/model/`
43+
44+
```
45+
$mv /sdcard/llama.cpp/llama-2-7b-chat.Q4_K_M.gguf /data/data/com.termux/files/home/model/
46+
```
47+
48+
Now, you can start chatting:
49+
```
50+
$cd /data/data/com.termux/files/home/bin
51+
$./llama-cli -m ../model/llama-2-7b-chat.Q4_K_M.gguf -n 128 -cml
52+
```
53+
54+
Here's a demo of an interactive session running on Pixel 5 phone:
55+
56+
https://user-images.githubusercontent.com/271616/225014776-1d567049-ad71-4ef2-b050-55b0b3b9274c.mp4
File renamed without changes.
File renamed without changes.

docs/build.md

Lines changed: 288 additions & 0 deletions
Large diffs are not rendered by default.

docs/HOWTO-add-model.md renamed to docs/development/HOWTO-add-model.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
## Add a new model architecture to `llama.cpp`
1+
# Add a new model architecture to `llama.cpp`
22

33
Adding a model requires few steps:
44

File renamed without changes.

0 commit comments

Comments
 (0)