How to access the outputs before sentencepiece detokenisation?

[pasricha]

Hi,

Is there a way to look at the output tokens before they are detokenised? Right now I am using model.predict() (as shown in the t5-trivia example) to generate the output for a sequence to sequence model, but this saves the detokenized output to a file.

I have tried looking at the result in the decode method, but it also returns output which is already detokenized. I want to see the output token ids before sentencepiece detokenization. How can I do this?

Thanks