Attention maps?

[jubruckne]

I was wondering if it would be easy to extract attention maps with llama.cpp? I’m not yet very familiar with the code, so I may be completely wrong, but couldn‘t the embedding output functionality be modified so that instead of the embedding layer it could take a parameter specifying which layer’s activations to output?