Text Generation w/o repeating the prompt

[wildmanwong]

Is there a parameter to specify if we do NOT want the prompt repeated when generating text?

[AguirreNicolas]

You should modify this line probably.

vllm/vllm/entrypoints/api_server.py

Line 61 in 7b895c5

text_outputs = [prompt + output.text for output in final_output.outputs]

Or on the client side take len(prompt) and remove then from the response.