Batch Generation from a List of Prompts, Multiple Completions per Prompt

[tzipproth]

For creating a small text corpus for some training experiments,
I'm searching for an option to create a large amount outputs from a list of prompts,
with n outputs per prompt with a configurable temperature.

This would be very useful to have I think, for example:

prompt1: Create a fantasy story featuring a fairy and an elf as characters
prompt2: Create a fantasy story featuring an ogre and a knight as characters
... and so on.

For each prompt n outputs which temperature t.
I searched in the code, but couldn't find a simple method to do that.
I can code this too, if someone would guide me to the right place in the code to add this functionality.

[KerfuffleV2]

You might find the simple-inference example from my seqrep sampler pull (#2593) a decent place to start. See this comment: #2593 (comment)

simple-inference has some special features for generating multiple drafts based on a prompt. It also has an extended features in interactive mode where you can enter commands to do stuff like stop a sequence, switch focus, etc. In it's current state it's probably not exactly what you want since it only supports one shared prompt, so you'd have to hack it to fit your use case.

[ggerganov]

Also, batched example is a good starting place - needs to be extended with multiple prompts support, which should be easy