Skip to content

Commit 260b7c6

Browse files
authored
server : update readme with undocumented options (#7013)
1 parent 53d6c52 commit 260b7c6

File tree

1 file changed

+12
-0
lines changed

1 file changed

+12
-0
lines changed

examples/server/README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,18 @@ page cache before using this. See https://github.com/ggerganov/llama.cpp/issues/
6262
- `--chat-template JINJA_TEMPLATE`: Set custom jinja chat template. This parameter accepts a string, not a file name. Default: template taken from model's metadata. We only support [some pre-defined templates](https://github.com/ggerganov/llama.cpp/wiki/Templates-supported-by-llama_chat_apply_template)
6363
- `--log-disable`: Output logs to stdout only, not to `llama.log`. Default: enabled
6464
- `--log-format FORMAT`: Define the log output to FORMAT: json or text Default: `json`
65+
- `--rope-scaling` : RoPE scaling method. Defaults to linear unless otherwise specified by the model. Options are `none`, `linear`, `yarn`
66+
- `--rope-freq-base N` : RoPE frequency base (default: loaded from model)
67+
- `--rope-freq-scale N`: RoPE frequency scaling factor, expands context by a factor of 1/N (e.g. 0.25)
68+
- `--yarn-ext-factor N` : YaRN: extrapolation mix factor (Default: 1.0, 0.0 = full interpolation)
69+
- `--yarn-attn-factor N` : YaRN: scale sqrt(t) or attention magnitude (default: 1.0)
70+
- `--yarn-beta-slow N`: YaRN: High correction dim or alpha (default: 1.0)
71+
- `--yarn-beta-fast N`: YaRN: low correction dim or beta (default: 32.0)
72+
- `--pooling` : Pooling type for embeddings, use model default if unspecified. Options are `none`, `mean`, `cls`
73+
- `-dt N`, `--defrag-thold N`: KV cache defragmentation threshold (default: -1.0, < 0 = disabled)
74+
- `-fa`, `--flash-attn` : enable flash attention (default: disabled).
75+
- `-ctk TYPE`, `--cache-type-k TYPE` : KV cache data type for K (default: `f16`, options `f32`, `f16`, `q8_0`, `q4_0`, `q4_1`, `iq4_nl`, `q5_0`, or `q5_1`)
76+
- `-ctv TYPE`, `--cache-type-v TYPE` : KV cache type for V (default `f16`, see `-ctk` for options)
6577

6678
**If compiled with `LLAMA_SERVER_SSL=ON`**
6779
- `--ssl-key-file FNAME`: path to file a PEM-encoded SSL private key

0 commit comments

Comments
 (0)