Support for intfloat/e5-mistral-7b-instruct

[nathanpbell]

In trying to wrap my head around this, I think I've found that llama.cpp would need to support two new features to get this embedding model to work optimally:

1. we need away to probe the values at the last layer, before the LM head (or ideally skip the LM head all together). Does the current embedding endpoint do exactly that? I couldn't fully follow where it grabs its values from

2. We need a way to pass in an attention mask along with the batch of inputs, or calculate one.

Before I explore this more, am I on the right path here that a) llama doesn't currently have these features and b) they are needed and c) they are in theory sufficient (or close to it) to get e5-mistral to work as intended.

[ggerganov]

1. Yes, llama_get_embeddings() will give you the activations before the LM head. See the emebedding example
2. What kind of a masks specifically are needed? llama.cpp support attention masks via the llama_batch parameters - see explanation here: Support for LLMLingua #4823 (comment)