Haystack Annotation Tool Question Ordering

[atastats]

Hello,

I am using the hosted version of the Annotation tool to label question/answer pairs to fine tune my own LLM. The issue I am facing is that when I batch upload document specific questions, it seems to not respect ordering. I have the questions ordered so that questions with answers near each other in the text are back to back making it easier for my labelers. But upon uploading my csv, the question ordering changes in the annotation tool. I have even tried prepending my questions with alphabetical characters to force appropriate indexing should some form of sort occur. Does anyone know a solution to this?

Thanks in advance!

[vblagoje]

Were you able to solve your issue @atastats ? We are not officially supporting the annotation tool :-(

[atastats]

No, it seems to be just an issue with uploading document specific questions. My solution was just to upload my full list of questions and not have document specific ones. Wastes a bit of labelers time on questions the model was already confident about, but the ordering being off would have wasted more time.