llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'mllama'

[physimo]

Happened when I try to load Llama 3.2 vision-instruct type, such as the 11b vision instruct
Full log:

llama_model_loader: loaded meta data with 26 key-value pairs and 396 tensors from A:\models\Llama-3.2-11B-Vision-Instruct.f16.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = mllama
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                               general.name str              = Model
llama_model_loader: - kv   3:                         general.size_label str              = 10B
llama_model_loader: - kv   4:                         mllama.block_count u32              = 40
llama_model_loader: - kv   5:                      mllama.context_length u32              = 131072
llama_model_loader: - kv   6:                    mllama.embedding_length u32              = 4096
llama_model_loader: - kv   7:                 mllama.feed_forward_length u32              = 14336
llama_model_loader: - kv   8:                mllama.attention.head_count u32              = 32
llama_model_loader: - kv   9:             mllama.attention.head_count_kv u32              = 8
llama_model_loader: - kv  10:                      mllama.rope.freq_base f32              = 500000.000000
llama_model_loader: - kv  11:    mllama.attention.layer_norm_rms_epsilon f32              = 0.000010
llama_model_loader: - kv  12:                          general.file_type u32              = 1
llama_model_loader: - kv  13:                          mllama.vocab_size u32              = 128256
llama_model_loader: - kv  14:                mllama.rope.dimension_count u32              = 128
llama_model_loader: - kv  15:    mllama.attention.cross_attention_layers arr[i32,8]       = [3, 8, 13, 18, 23, 28, 33, 38]
llama_model_loader: - kv  16:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  17:                         tokenizer.ggml.pre str              = smaug-bpe
llama_model_loader: - kv  18:                      tokenizer.ggml.tokens arr[str,128257]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  19:                  tokenizer.ggml.token_type arr[i32,128257]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  20:                      tokenizer.ggml.merges arr[str,280147]  = ["ﾄ ﾄ", "ﾄ ﾄﾄﾄ", "ﾄﾄ ﾄﾄ", "...
llama_model_loader: - kv  21:                tokenizer.ggml.bos_token_id u32              = 128000
llama_model_loader: - kv  22:                tokenizer.ggml.eos_token_id u32              = 128009
llama_model_loader: - kv  23:            tokenizer.ggml.padding_token_id u32              = 128004
llama_model_loader: - kv  24:                    tokenizer.chat_template str              = {{- bos_token }}\n{%- if custom_tools ...
llama_model_loader: - kv  25:               general.quantization_version u32              = 2
llama_model_loader: - type  f32:  114 tensors
llama_model_loader: - type  f16:  282 tensors
llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'mllama'
llama_load_model_from_file: failed to load model
file:///A:/node_modules/node-llama-cpp/dist/evaluator/LlamaModel/LlamaModel.js:479
                throw new Error("Failed to load model");
                      ^

Error: Failed to load model
    at Function._create (file:///A:/node_modules/node-llama-cpp/dist/evaluator/LlamaModel/LlamaModel.js:479:23)
    at Object.<anonymous> (file:///A:/node_modules/node-llama-cpp/dist/bindings/Llama.js:187:24)
    at withLock (file:///A:/node_modules/lifecycle-utils/dist/withLock.js:36:16)
    at Llama.loadModel (file:///A:/node_modules/node-llama-cpp/dist/bindings/Llama.js:183:16)
    at NodeLlamaHandler.startLlama (A:\src\llama\llama.ts:27:18)
    at LlamaHandler.initialize (A:\src\llama\index.ts:25:5)

I know multimodal is not yet supported, but does that necessarily means we can't use the meta llama 3.2 11b model since it only has vision-instruct type?

[giladgd]

The mllama architecture is not supported by llama.cpp.
If you don't need the vision capabilities, then there's no advantage to using Llama 3.2 with vision over Llama 3.1 (reference)

[physimo]

oh, so it's not related to the multimodal support
thanks for clarifying!