@@ -15,7 +15,7 @@ Build the `llm-tool` scheme in Xcode.
15
15
To run this in Xcode simply press cmd-opt-r to set the scheme arguments. For example:
16
16
17
17
```
18
- --model mlx-community/Mistral-7B-v0.1-hf- 4bit-mlx
18
+ --model mlx-community/Mistral-7B-Instruct- v0.3- 4bit
19
19
--prompt "swift programming language"
20
20
--max-tokens 50
21
21
```
@@ -27,7 +27,7 @@ the Hugging Face HubApi stores the downloaded files.
27
27
28
28
The model should be a path in the Hugging Face repository, e.g.:
29
29
30
- - ` mlx-community/Mistral-7B-v0.1-hf- 4bit-mlx `
30
+ - ` mlx-community/Mistral-7B-Instruct- v0.3- 4bit `
31
31
- ` mlx-community/phi-2-hf-4bit-mlx `
32
32
33
33
See [ LLM] ( ../../Libraries/MLXLLM/README.md ) for more info.
@@ -40,12 +40,15 @@ Use the `mlx-run` script to run the command line tools:
40
40
./mlx-run llm-tool --prompt "swift programming language"
41
41
```
42
42
43
+ Note: ` mlx-run ` is a shell script that uses ` xcode ` command line tools to
44
+ locate the built binaries. It is equivalent to running from Xcode itself.
45
+
43
46
By default this will find and run the tools built in _ Release_ configuration. Specify ` --debug `
44
47
to find and run the tool built in _ Debug_ configuration.
45
48
46
49
See also:
47
50
48
- - [ MLX troubleshooting] ( https://ml-explore.github.io /mlx-swift/MLX /documentation/mlx/troubleshooting )
51
+ - [ MLX troubleshooting] ( https://swiftpackageindex.com/ ml-explore/mlx-swift/main /documentation/mlx/troubleshooting )
49
52
50
53
### Troubleshooting
51
54
@@ -126,7 +129,7 @@ Here is an example run using adapters on the last 4 layers of the model:
126
129
giving output like this:
127
130
128
131
```
129
- Model: mlx-community/Mistral-7B-v0.1-hf- 4bit-mlx
132
+ Model: mlx-community/Mistral-7B-Instruct- v0.3- 4bit
130
133
Total parameters: 1,242M
131
134
Trainable parameters: 0.426M
132
135
Iteration 1: validation loss 2.443872, validation time 3.330629s
@@ -163,7 +166,7 @@ You can test the LoRA adapated model against the `test` dataset using this comma
163
166
164
167
```
165
168
./mlx-run llm-tool lora test \
166
- --model mlx-community/Mistral-7B-v0.1-hf- 4bit-mlx \
169
+ --model mlx-community/Mistral-7B-Instruct- v0.3- 4bit \
167
170
--data Data/lora \
168
171
--adapter /tmp/lora-layers-4.safetensors \
169
172
--batch-size 1 --lora-layers 4 \
@@ -173,7 +176,7 @@ You can test the LoRA adapated model against the `test` dataset using this comma
173
176
This will run all the items (100 in the example data we are using) in the test set and compute the loss:
174
177
175
178
```
176
- Model: mlx-community/Mistral-7B-v0.1-hf- 4bit-mlx
179
+ Model: mlx-community/Mistral-7B-Instruct- v0.3- 4bit
177
180
Total parameters: 1,242M
178
181
Trainable parameters: 0.426M
179
182
Test loss 1.327623, ppl 3.772065
@@ -192,7 +195,7 @@ Given that format you might issue a command like this:
192
195
193
196
```
194
197
./mlx-run llm-tool lora eval \
195
- --model mlx-community/Mistral-7B-v0.1-hf- 4bit-mlx \
198
+ --model mlx-community/Mistral-7B-Instruct- v0.3- 4bit \
196
199
--adapter /tmp/lora-layers-4.safetensors \
197
200
--lora-layers 4 \
198
201
--prompt "table: 1-10015132-16
206
209
You might be treated to a response like this:
207
210
208
211
```
209
- Model: mlx-community/Mistral-7B-v0.1-hf- 4bit-mlx
212
+ Model: mlx-community/Mistral-7B-Instruct- v0.3- 4bit
210
213
Total parameters: 1,242M
211
214
Trainable parameters: 0.426M
212
215
Starting generation ...
@@ -223,7 +226,7 @@ have the adapter weights merged in:
223
226
224
227
```
225
228
./mlx-run llm-tool lora fuse \
226
- --model mlx-community/Mistral-7B-v0.1-hf- 4bit-mlx \
229
+ --model mlx-community/Mistral-7B-Instruct- v0.3- 4bit \
227
230
--adapter /tmp/lora-layers-4.safetensors \
228
231
--output mlx-community/mistral-lora
229
232
```
0 commit comments