You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: examples/server/README.md
+12Lines changed: 12 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -62,6 +62,18 @@ page cache before using this. See https://github.com/ggerganov/llama.cpp/issues/
62
62
-`--chat-template JINJA_TEMPLATE`: Set custom jinja chat template. This parameter accepts a string, not a file name. Default: template taken from model's metadata. We only support [some pre-defined templates](https://github.com/ggerganov/llama.cpp/wiki/Templates-supported-by-llama_chat_apply_template)
63
63
-`--log-disable`: Output logs to stdout only, not to `llama.log`. Default: enabled
64
64
-`--log-format FORMAT`: Define the log output to FORMAT: json or text Default: `json`
65
+
-`--rope-scaling` : RoPE scaling method. Defaults to linear unless otherwise specified by the model. Options are `none`, `linear`, `yarn`
66
+
-`--rope-freq-base N` : RoPE frequency base (default: loaded from model)
67
+
-`--rope-freq-scale N`: RoPE frequency scaling factor, expands context by a factor of 1/N (e.g. 0.25)
-`-ctk TYPE`, `--cache-type-k TYPE` : KV cache data type for K (default: `f16`, options `f32`, `f16`, `q8_0`, `q4_0`, `q4_1`, `iq4_nl`, `q5_0`, or `q5_1`)
76
+
-`-ctv TYPE`, `--cache-type-v TYPE` : KV cache type for V (default `f16`, see `-ctk` for options)
65
77
66
78
**If compiled with `LLAMA_SERVER_SSL=ON`**
67
79
-`--ssl-key-file FNAME`: path to file a PEM-encoded SSL private key
0 commit comments