Question Answer Generation Pipeline - Adjust dataset for errors

[Nick7900]

I'm trying to generate new QA-dataset for a specific domain using the QuestionAnswerGenerationPipeline from this tutorial.
https://haystack.deepset.ai/tutorials/13_question_generation

This pipeline is fantastic and will save me a lot of time, but I've noticed that some of the questions and answers are incorrect.

For instance, I tried to use this text about Einstein as a demo and run the pipeline

text4 = "Albert Einstein, (born March 14, 1879, Ulm, Württemberg, Germany—died April 18, 1955, Princeton, New Jersey, U.S.), German-born physicist who developed the special and general theories of relativity and won the Nobel Prize for Physics in 1921 for his explanation of the photoelectric effect. Einstein is generally considered the most influential physicist of the 20th century."

Out of 6 QA the pipeline got 2 of them wrong.
Visualize wrong output
``
Output:

• Q: Who is generally considered to be the most influential? (miss physicist )
  A: Albert Einstein
• Q: What is the photoelectric effect?
  A: Albert Einstein (Wrong answer)
  ``

What is the most efficient method for handling and adjusting these QA-datasets once they have been generated?
Is there an interface where I can easily insert the QA-dataset and manually adjust them, or what do people usually do if there is an error in a QA-dataset?

Cheers,
Nick

[ZanSara]

Hey @Nick7900, I think one way could be to use the Annotation Tool. Have you tried that?

[Nick7900]

@ZanSara But the only document types it can accept are txt files, right?
I considered creating QA for thousands of documents using the pipeline, because it is not always easy to ask questions on different documents. After that, I'd like to double check the QA-dataset, which would be a JSON file, look for any errors, and then fix them.
Is that possible with the Annotation tool?

[ZanSara]

I've rarely used the annotation tool myself, but I believe you could load an existing dataset... I'll check with someone more knowledgeable and let you know 👍

[Nick7900]

Hi @ZanSara! Have you heard anything yet? :)

[Nick7900]

Hi @ZanSara are there any updates? :)

[ZanSara]

Hey I'm really sorry for the delay, I totally lost sight of this thread. Seems like it's not currently possible with the Annotation tool, so unfortunately it seems like there's no good way to do what you ask with it or with Haystack in general. If you find some other tool that can do the job, though, feel free to share so that other users can benefit from it too!