How do I find the source document of the answer from the answer

[mc112611]

When the pipeline outputs the final answer, return the file name or metadata of the document where the answer originated.

[anakin87]

Hey @mc112611...
In order to help, we need some more details. Please provide a minimal code example...

[mc112611]

This is an Answer, and the source of this answer comes from a small document, which comes from dividing a large document into multiple copies. So how can I find the large document based on the answer.

<Answer {'answer': 'Robb Stark', 'type': 'extractive', 'score': 0.090131476521492, 'context': "* '''Roslin Frey''' (season 3) portrayed by Alexandra Dowling. Roslin Tully (née Frey) is Lord Walder Frey's daughter and considered the most beautiful one. She was previously the intended bride for Robb Stark, who was unaware of her beauty and went on to marry Talisa Maegyr instead. She is married to Lord Edmure Tully as compensation at what becomes known as the Red Wedding. In the sixth season, it is mentioned that Roslin has given birth to Edmure's child.\n\n===House Tyrell===\n", 'offsets_in_document': [{'start': 199, 'end': 209}], 'offsets_in_context': [{'start': 199, 'end': 209}], 'document_ids': ['9e72eebc37a70cdd1b7db1bad85c4d1e'], 'meta': {'_split_id': 122}}>,

Can we add a unique identifier for each document during the preprocessing stage. And search for the source document in the document collection based on its name or unique identifier

[anakin87]

This old discussion might help you.

(If not, please provide the code you used).

[mc112611]

Could you please tell me about meta and how to pass in custom dictionary type data.

For example,
I have three documents, namely A.txt B.txt C.txt.
When I use the following code from the official tutorial to write three txt files to the document_store：
doc_dir = "../data/tutorial8"
all_docs = convert_files_to_docs(dir_path=doc_dir)
preprocessor = PreProcessor( clean_empty_lines=True, clean_whitespace=True, clean_header_footer=False, split_by="word", split_length=100, split_respect_sentence_boundary=True,language='zh' )
docs = preprocessor.process(all_docs)

How can I customize meta_ Dict.

I hope its document_ The information stored in the store is

{'content_type ':' text ',' score ': None,' meta ': {'name': 'A.txt', 'split_id': 2, 'bat_man': 'super_man'}, 'id Hash_ Keys': ['content '],' embedding ': None,' id ':' 8c106b7e48b030d6fed707069bbdda40 '}

[anakin87]

I think convert_files_to_docs is a bit too magic and does not allow the flexibility you need.

Instead, I suggest looking at this example: https://github.com/deepset-ai/haystack/blob/main/examples/basic_qa_pipeline.py

Particularly these lines can be useful for you:

haystack/examples/basic_qa_pipeline.py

Lines 25 to 26 in fd25106

	file_paths = [p for p in Path(doc_dir).glob("**/*")]
	files_metadata = [{"name": path.name} for path in file_paths]

haystack/examples/basic_qa_pipeline.py

Line 53 in fd25106

indexing_pipeline.run(file_paths=file_paths, meta=files_metadata)

Please let me know if you find a way to solve this problem!

[mc112611]

I think I know what to do next. Your response was very helpful. Thank you, sir or madam