Skip to content

Commit 1db3b58

Browse files
committed
docs: Add docs explaining how to install pre-built wheels.
1 parent c50309e commit 1db3b58

File tree

1 file changed

+45
-0
lines changed

1 file changed

+45
-0
lines changed

README.md

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,15 @@ This will also build `llama.cpp` from source and install it alongside this pytho
4444

4545
If this fails, add `--verbose` to the `pip install` see the full cmake build log.
4646

47+
**Pre-built Wheel (New)**
48+
49+
It is also possible to install a pre-built wheel with basic CPU support.
50+
51+
```bash
52+
pip install llama-cpp-python \
53+
--extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
54+
```
55+
4756
### Installation Configuration
4857

4958
`llama.cpp` supports a number of hardware acceleration backends to speed up inference as well as backend specific options. See the [llama.cpp README](https://github.com/ggerganov/llama.cpp#build) for a full list.
@@ -108,6 +117,30 @@ To install with cuBLAS, set the `LLAMA_CUDA=on` environment variable before inst
108117
CMAKE_ARGS="-DLLAMA_CUDA=on" pip install llama-cpp-python
109118
```
110119

120+
**Pre-built Wheel (New)**
121+
122+
It is also possible to install a pre-built wheel with CUDA support. As long as your system meets some requirements:
123+
124+
- CUDA Version is 12.1, 12.2 or 12.3
125+
- Python Version is 3.10, 3.11 or 3.12
126+
127+
```bash
128+
pip install llama-cpp-python \
129+
--extra-index-url https://abetlen.github.io/llama-cpp-python/whl/<cuda-version>
130+
```
131+
132+
Where `<cuda-version>` is one of the following:
133+
- `cu121`: CUDA 12.1
134+
- `cu122`: CUDA 12.2
135+
- `cu123`: CUDA 12.3
136+
137+
For example, to install the CUDA 12.1 wheel:
138+
139+
```bash
140+
pip install llama-cpp-python \
141+
--extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121
142+
```
143+
111144
</details>
112145

113146
<details>
@@ -119,6 +152,18 @@ To install with Metal (MPS), set the `LLAMA_METAL=on` environment variable befor
119152
CMAKE_ARGS="-DLLAMA_METAL=on" pip install llama-cpp-python
120153
```
121154

155+
**Pre-built Wheel (New)**
156+
157+
It is also possible to install a pre-built wheel with Metal support. As long as your system meets some requirements:
158+
159+
- MacOS Version is 11.0 or later
160+
- Python Version is 3.10, 3.11 or 3.12
161+
162+
```bash
163+
pip install llama-cpp-python \
164+
--extra-index-url https://abetlen.github.io/llama-cpp-python/whl/metal
165+
```
166+
122167
</details>
123168
<details>
124169

0 commit comments

Comments
 (0)