Set temperature of model from huggingface

[ethantrokie]

It looks like there is a fairly simple way to set the temperature of the OpenAI model using the OpenAIAnswerGenerator, but I can't seem to find a way to change the temperature for any models taken from hugging face. How can I to set the temperature?

[ZanSara]

Hey @ethantrokie which node are you using? OpenAIAnswerGenerator supports only OpenAI models.

[ethantrokie]

Hi @ZanSara,

I'm trying to use the Flan-T5-large model using just the standard PromptNode. I'm aware that OpenAIAnswerGenerator supports only OpenAI models, but is there a similar function/parameter that I can set in a normal PromptNode that can set the temperature of the model? Or is there a way to work around the limitations of a PromptNode and set the temperature a different way?

[dfokina]

We added some information about this in our docs. This is only possible for Haystack v1.16 and on, though.
https://docs.haystack.deepset.ai/v1.16.0/docs/prompt_node#using-hugging-face-models