Generating questions from a prompt and a given context

[sancelot]

I am trying to obtain questions relative to a given context. But I obtain hallucinating answers.

I am not sure , if the problem is related to the prompt or not

here is my program

from llama_cpp import Llama

# define n_ctx manually to permit larger contexts
LLM = Llama(model_path="../../../../llama-2-7b.Q5_K_M.gguf", n_ctx=512)

# create a text prompt
context = """
Nous sommes dans les Yvelines
Il fait beau
Nous sommes a quelques km d'un joli lac
le boulanger est sympatique
Il y a également une boucherie
"""

prompt = """
You are a Teacher/ Professor. Your task is to setup
4 questions for an upcoming
    quiz/examination. The questions should be diverse in nature 
   across the document. Restrict the questions to the 
    context  provided.
 Context is provided below :                                                                                                                                                                                                                                                                                                                                             
   %s                                                                                                                                                                                                                                                                                                                  
""" % context


print(prompt)
# set max_tokens to 0 to remove the response size limit
output = LLM(prompt, max_tokens=0)

# display the response
print(output["choices"][0]["text"])

print("---------------")
print(output)

[staviq]

Judging by the model filename, it seems like you are using base llama2 model, which isn't trained for QnA or instruct type inference.

If you are using chat model or instruct model, you should use a prompt format appropriate for given model.

Otherwise, the model has no way of understanding what you want from it, and will just produce text roughly similar to the prompt.

[sancelot]

You were right , I am now using an instruct model ! Thanks
I add to adopt [INST] ... [/INST] for the model I used