How to get LLM's embedding during batch inference?

[importrayhan]

Reminder

• I have read the above rules and searched the existing issues.

Description

Sometimes the pre-embedding layers or prediction logits are more useful than the prediction tokens.
To achieve the similar features in Guidance-AI or Layer-wise Analysis, is there any possible configuration in batch inference?

For example,
--output-layer: -1
--output-layer-length: 10
to obtain prediction logits of top 10 tokens.

Pull Request

No response