Get probability over vocabulary for each token

[djmMax]

Expected Behavior

I want to get probability over vocabulary for each token while running embedding or normal prediction.

Current Behavior

No mention of probability over vocabulary mentioned in the project

Environment and Context

Ubuntu 22.04

[staviq]

Take a look at this example: https://github.com/ggerganov/llama.cpp/blob/ac43576124a75c2de6e333ac31a3444ff9eb9458/examples/simple/simple.cpp#L98

Token probs are already there.

You can modify that example however you like,

It's the llama_sample_token_X that trims those probs down to a single "winner" token.

Similarly, in main you should be able to extract probs the same way, right before that line: https://github.com/ggerganov/llama.cpp/blob/ac43576124a75c2de6e333ac31a3444ff9eb9458/examples/main/main.cpp#L628