Get the total number of layers

[axel7083]

Given a model, I would like to know the number of layers in it, but I am bit lost between all the items in the metadata.

I used https://github.com/abetlen/llama-cpp-python to easily get the metadata of a given model as followed

from llama_cpp import Llama
llm = Llama(model_path="/models/granite-7b-lab-Q4_K_M.gguf")
llm.metadata

However, I am not sure to know which value is the one I should read as there is lot.

In the logs I can see the following

llm_load_print_meta: n_layer          = 32

but in the metadata the values available are

...
'llama.attention.head_count': '32',
'llama.block_count': '32',
'llama.attention.head_count_kv': '32'

but not sure which value I should use

[ngxson]

The number of layer is llama.block_count. One layer can have multiple heads (hence the name "multi-head attention")

[axel7083]

Thanks !