When I get The prompt has been truncated from xxx token to 412 tokens, how to shorten prompt?

[demongolem-biz2]

So the general form of a prompt is something like

"Given the context please answer the question. Context: {join(documents)}; Question: "
"{query}; Answer:"

And the Context and Question will be filled in for you. But how do I prevent this expanded prompt from running over the token limit? What I would might want to do is say to have the first 300 tokens of the Context and the first 50 tokens of the Question, but in reality what I see is that the prompt as a whole is truncated when it reaches 412 token (for a 512 token model) whereever that may be.

[demongolem-biz2]

To add, I have used split length of 300 words in the PreProcessor. But words != tokens apparently because what comes out of that is much longer than 300 tokens. Is this so, is there no really correspondence to the definition of word in the PreProcessor and definition of token in the model being used by the PromptNode (google/flan-t5-base in this case I suppose)? I want to use as many of the 412 tokens as I possibly can and be able to tell if I am going to be truncated.

Otherwise maybe I can put my context (which is smaller) at the beginning? I don't know how to rearrange the prompt for maximum utility.