Skip to content

Commit f373a49

Browse files
committed
Update llm_inference.md
Signed-off-by: Michael Yuan <michael@secondstate.io>
1 parent 2cf37a8 commit f373a49

File tree

1 file changed

+2
-3
lines changed

1 file changed

+2
-3
lines changed

docs/develop/rust/wasinn/llm_inference.md

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -119,10 +119,9 @@ You can configure the chat inference application through CLI options.
119119

120120
The `--prompt-template` option is perhaps the most interesting. It allows the application to support different open source LLM models beyond llama2. Check out more prompt templates [here](https://github.com/LlamaEdge/LlamaEdge/tree/main/api-server/chat-prompts).
121121

122-
The `--ctx-size` option specifies the context windows size of the application. It is limited by the model's intrinsic context window size. If you increase the `--ctx-size`, make sure that you also
123-
explicitly specify the `--batch-size` to a reasonable value (e.g., `--batch-size 512`).
122+
The `--ctx-size` option specifies the context windows size of the application. It is limited by the model's intrinsic context window size.
124123

125-
The following command tells WasmEdge to print out logs and statistics of the model at runtime.
124+
The `--log-stat` tells WasmEdge to print out logs and statistics of the model at runtime.
126125

127126
```bash
128127
wasmedge --dir .:. --nn-preload default:GGML:AUTO:Meta-Llama-3.1-8B-Instruct-Q5_K_M.gguf \

0 commit comments

Comments
 (0)