How to use llama.cpp to generate answers in NLI

[ClaudiuCreanga]

I am looking to use llama.cpp to generate hypothesis from premise (NLI).
Say I have a premise "man is sitting on the couch". I want the model to generate a neutral hypothesis like "the keyboard is in english". Or a contradictory hypothesis: "the man is standing". I want it to do that for 10k examples.
I have llama.cpp working in interactive mode using:

./main -m ./models/llama-2-7b-chat.gguf.bin \                 
  --color \       
  --ctx_size 2048 \
  -n -1 \
  -ins -b 256 \
  --top_k 10000 \
  --temp 0.2 \
  --repeat_penalty 1.1 \
  -t 8

But I don't think that's the best way for this task.

[staviq]

You can use server and feed it prompts with curl, this way you don't have to reload the model every example, and you can call it as many times as you need.