Skip to content

--quantized-type is not user friendly #78

@danielzgtg

Description

@danielzgtg

Following error messages to specify mandatory parameters, I got to this point:

$ ./quantize --quantized-type F16 --model-path ~/Kokoro_espeak.gguf --quantized-model-path ./out.gguf
ERROR: 0 is not a valid quantization type.

Turns out --quantize-type is an int_param‽

enum ggml_type qtype = static_cast<ggml_type>(*args.get_int_param("--quantized-type"));

I should be able to use human-readable names like I do with llama.cpp

./llama-quantize ./models/mymodel/ggml-model-f16.gguf ./models/mymodel/ggml-model-Q4_K_M.gguf Q4_K_M

The following human-friendly names should be supported, corresponding to those in https://huggingface.co/mmwillet2/Kokoro_GGUF/tree/main :

  • F16
  • Q4
  • Q5
  • Q8

Metadata

Metadata

Assignees

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions