[GGUF] typed metadata

[mishig25]

Description

Enhance GGUF functionality by adding typedMetadata support

This update introduces typedMetadata to the gguf function, allowing users to request structured metadata alongside the standard output. The implementation includes checks for both V1 and V2 file formats, ensuring compatibility and consistency in metadata retrieval. Additionally, tests have been added to validate the new functionality and ensure that metadata values align correctly between standard and typed formats.

Usage

import { GGMLQuantizationType, GGUFValueType, gguf } from "@huggingface/gguf";

const URL_LLAMA = "https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/resolve/191239b/llama-2-7b-chat.Q2_K.gguf";

const { metadata, typedMetadata } = await gguf(URL_LLAMA, { typedMetadata: true });

console.log(typedMetadata);
// {
//     version: { value: 2, type: GGUFValueType.UINT32 },
//     tensor_count: { value: 291n, type: GGUFValueType.UINT64 },
//     kv_count: { value: 19n, type: GGUFValueType.UINT64 },
//     "general.architecture": { value: "llama", type: GGUFValueType.STRING },
//     "general.file_type": { value: 10, type: GGUFValueType.UINT32 },
//     "general.name": { value: "LLaMA v2", type: GGUFValueType.STRING },
//     "llama.attention.head_count": { value: 32, type: GGUFValueType.UINT32 },
//     "llama.attention.layer_norm_rms_epsilon": { value: 9.999999974752427e-7, type: GGUFValueType.FLOAT32 },
//     "tokenizer.ggml.tokens": { value: ["<unk>", "<s>", "</s>", ...], type: GGUFValueType.ARRAY },
//     ...
// }

// Access both value and type information
console.log(typedMetadata["general.architecture"].value); // "llama"
console.log(typedMetadata["general.architecture"].type);  // GGUFValueType.STRING (8)

[ngxson]

Just wondering, should we now separate GGUFValueType.ARRAY into GGUFValueType.ARRAY_INT32, GGUFValueType.ARRAY_STRING, etc ?

It will come in handy when we want to distinguish among array of uint, int or float

[mishig25]

that would be diverging from the gguf spec, no ?

https://github.com/ggml-org/ggml/blob/master/docs/gguf.md?plain=1#L191

there is no ARRAY_STRING or ARRAY_INT32 in enum gguf_metadata_value_type

[mishig25]

pushed 0ba56b1

if type is array, there will be property subType as well

//     "tokenizer.ggml.tokens": { value: ["<unk>", "<s>", "</s>", ...], type: GGUFValueType.ARRAY, subType: GGUFValueType.STRING },
//     "tokenizer.ggml.scores": { value: [0.0, -1000.0, -1000.0, ...], type: GGUFValueType.ARRAY, subType: GGUFValueType.FLOAT32 },

[julien-c]

i would keep things simple personally but 🤷

[ngxson]

Yes subType would work too!

This is necessary because we need to reconstruct the array with the original type. Otherwise GGUF will fail to load if the element type mismatched (which can be the case for float/uint/int)