Skip to content

Commit bf9bf37

Browse files
committed
Add typed metadata support in README for GGUF, showcasing how to retrieve metadata with type information.
1 parent 6416140 commit bf9bf37

File tree

1 file changed

+30
-0
lines changed

1 file changed

+30
-0
lines changed

packages/gguf/README.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,36 @@ const { metadata, tensorInfos } = await gguf(
6868
);
6969
```
7070

71+
### Typed metadata
72+
73+
You can get metadata with type information by setting `typedMetadata: true`. This provides both the original value and its GGUF data type:
74+
75+
```ts
76+
import { GGMLQuantizationType, GGUFValueType, gguf } from "@huggingface/gguf";
77+
78+
const URL_LLAMA = "https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/resolve/191239b/llama-2-7b-chat.Q2_K.gguf";
79+
80+
const { metadata, typedMetadata } = await gguf(URL_LLAMA, { typedMetadata: true });
81+
82+
console.log(typedMetadata);
83+
// {
84+
// version: { value: 2, type: GGUFValueType.UINT32 },
85+
// tensor_count: { value: 291n, type: GGUFValueType.UINT64 },
86+
// kv_count: { value: 19n, type: GGUFValueType.UINT64 },
87+
// "general.architecture": { value: "llama", type: GGUFValueType.STRING },
88+
// "general.file_type": { value: 10, type: GGUFValueType.UINT32 },
89+
// "general.name": { value: "LLaMA v2", type: GGUFValueType.STRING },
90+
// "llama.attention.head_count": { value: 32, type: GGUFValueType.UINT32 },
91+
// "llama.attention.layer_norm_rms_epsilon": { value: 9.999999974752427e-7, type: GGUFValueType.FLOAT32 },
92+
// "tokenizer.ggml.tokens": { value: ["<unk>", "<s>", "</s>", ...], type: GGUFValueType.ARRAY },
93+
// ...
94+
// }
95+
96+
// Access both value and type information
97+
console.log(typedMetadata["general.architecture"].value); // "llama"
98+
console.log(typedMetadata["general.architecture"].type); // GGUFValueType.STRING (8)
99+
```
100+
71101
### Strictly typed
72102

73103
By default, known fields in `metadata` are typed. This includes various fields found in [llama.cpp](https://github.com/ggerganov/llama.cpp), [whisper.cpp](https://github.com/ggerganov/whisper.cpp) and [ggml](https://github.com/ggerganov/ggml).

0 commit comments

Comments
 (0)