Python Wrapper - Access to output tokens rather than string?

[noamgat]

Hi,
Is there a way to get the output token sequence when generating text with llama-cpp-python? I don't need the logprobs, just the tokens.
If I tokenize the result I don't necessarily get the same tokens that were chosen, as there are potentially many ways to get to the same output string.

[noamgat]

I was able to do this manually by using a combination of llm.eval_tokens after the function returns and testing which suffix of it detokenizes to the output string. Is this a stable approach?

[KerfuffleV2]

Your best bet is probably to ask in the repo for that project. It uses llama.cpp as a backend but it's made by different people as far as I know: https://github.com/abetlen/llama-cpp-python