Obtain proability of tokens #9689
Unanswered
ChristianHinge
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I wish to implement LLama.cpp for yes/no question answering based on images. In this context I would like to get the token probabilities (so I can use them as a measure of uncertainty).
For the server implementation, it is possible to do
n_probs=N
and obtain the probabilities of N most probable tokens. However, I do not see a similar option forllama-cli
(orllama-minicpmv-cli
, which is the one I use).AFAIK the server implementation does not currently support image-input, so I am left with the CLI's.
Have I missed something or do you have any ideas on how I may address this?
Beta Was this translation helpful? Give feedback.
All reactions