Recommended method for outputing log_probs or top_logprobs

[de-abreu]

Hello there,

I'm looking for a method to use Outlines with Mistral in a text classification task and output the probabilities of each label being picked. Here is my sample experiment where I already implemented Outlines to the text classification task. How could I modify my experiment to implement log_probs?

Thanks!

[rlouf]

The recommended way is to implement a custom logits processor, a simplified version of #1408 and pass it when calling the model, see https://dottxt-ai.github.io/outlines/main/features/advanced/logits_processors/. We would definitely welcome a PR.