Now that we can readily read GGUF files from Python, I made some experimental GGUF tools #4050
KerfuffleV2
started this conversation in
Show and tell
Replies: 1 comment
-
I managed to get tensor to image script generating much more interesting output: It's colored according to standard deviations above and below the mean. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
These may be useful, but please note they are not to the standard of scripts/tools included in the repo here. They may or may not work.
Link: https://github.com/KerfuffleV2/gguf-tools
This is what I have so far:
gguf-checksum
Allows calculating a model's SHA256 without being affected by the exact order of the fields in the file. It's also possible to get checksums of the individual tensors or metadata fields. Note: The overall hash will not match that from tools like
sha256sum
since it is based on a sorted version of the fields. The point of the tool is to allow comparing models in a way where the order of the tensors of fields does not affect the result.Examples:
gguf-checksum.py model.gguf
— Generate a full checksum for the model, including tensor data. At the end, you'll get a SHA256 for the KV metadata, the tensor metadata, the whole metadata section, the tensor data alone and the overall SHA256 with all sections included. As mentioned, this won't match a SHA256 based only on the raw file content.gguf-checksum.py --json model.gguf
— Same as above except the output will be in JSON format. Example:gguf-checksum.py --hash-individual-kvs --hash-individual-tensors model.gguf
— Same as the first, except you will also get a SHA256 for each individual KV and each tensor's data. Example:gguf-frankenstein
You supply an input metadata GGUF file and optionally an input tensor data GGUF file and this utility will stitch the two together into a new GGUF file. When the tensor data file isn't specified, you end up with a vocab-only model that just has the metadata. This could be used for future Frankenstein-ing or training a model with that vocab/metadata as the base.
Examples:
gguf-frankenstein.py --metadata md.gguf --tensor td.gguf --output result.gguf
— Createresult.gguf
with the key/value metadata frommd.gguf
and the tensor data (and tensor metadata) fromtd.gguf
.gguf-frankenstein.py --metadata md.gguf --output result.gguf
— Createresult.gguf
with the key/value metadata frommd.gguf
. This will be a vocab-only model that could be used for training.gguf-tensor-to-image
Saves a tensor or tensors from a GGUF file as an image. See the
CFG_
values near the top. Some tensors are more interesting than others. Check out anattn_q
tensor if you get the chance. Oh baby, there's a lot going on. The script can deal withF32
,F16
andQ8_0
tensors and includes a tinyQ8_0
quantization/dequantization implementation.Examples:
gguf-tensor-to-image.py --output out.png model.gguf output.weight
— Save theoutput.weight
tensor inmodel.gguf
asout.png
gguf-tensor-to-image.py --output out.png model.gguf output.weight token_embd.weight
— Save the specified tensors inmodel.gguf
asoutput.weight.out.png
andtoken_embd.weight.out.png
gguf-tensor-to-image.py --output ./imgs/tensor.png model.gguf '*'
— Save all tensors inmodel.gguf
like./imgs/output.weight.tensor.png
. Note: Be sure to quote or escape*
when specifying it as an option.Beta Was this translation helpful? Give feedback.
All reactions