You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/develop/rust/wasinn/llm_inference.md
+2-3Lines changed: 2 additions & 3 deletions
Original file line number
Diff line number
Diff line change
@@ -119,10 +119,9 @@ You can configure the chat inference application through CLI options.
119
119
120
120
The `--prompt-template` option is perhaps the most interesting. It allows the application to support different open source LLM models beyond llama2. Check out more prompt templates [here](https://github.com/LlamaEdge/LlamaEdge/tree/main/api-server/chat-prompts).
121
121
122
-
The `--ctx-size` option specifies the context windows size of the application. It is limited by the model's intrinsic context window size. If you increase the `--ctx-size`, make sure that you also
123
-
explicitly specify the `--batch-size` to a reasonable value (e.g., `--batch-size 512`).
122
+
The `--ctx-size` option specifies the context windows size of the application. It is limited by the model's intrinsic context window size.
124
123
125
-
The following command tells WasmEdge to print out logs and statistics of the model at runtime.
124
+
The `--log-stat` tells WasmEdge to print out logs and statistics of the model at runtime.
0 commit comments