QA based on previously saved data

[wawaks]

I use Qdrant (in Docker) to store vectors. I am able to store vectors and ask questions based on them in one script. But I can't find any example in docs where the dataset is searched based on a previously created collection. How to load the data? I found that Pinecone has "from_existing_index" function, which probably does the thing. But Qdrant doesn't have such a function. I created my own solution using qdrant_client, but I would like to use Langchain to simplify the script. How to do it?

The second question refers to vector creation. I have a large corpus of documents. All examples I found import data to Qdrant in one go. Is it possible to add one document at a time? I tried "from_documents", but this function recreates an empty collection every time. How to add without recreating the collection?

[VpkPrasanna]

@hwchase17 any thoughts on this ?

[wawaks]

Well, it was long time ago and I forgot details. But I had to change something in the code to make add_documents work. My functions are:

``
def create_collection(docs, collection_name):
embeddings = HuggingFaceEmbeddings(model_name="all-mpnet-base-v2")
qdrant = Qdrant.from_documents(docs, embeddings, collection_name=collection_name, timeout=60)

def update_collection(docs, collection_name):
client = QdrantClient("localhost", port=6333)
embeddings = HuggingFaceEmbeddings(model_name="all-mpnet-base-v2")
qdrant = Qdrant(client=client, collection_name=collection_name, embeddings=embeddings)
result = qdrant.add_documents(docs)
``

Regarding the first question:

``
def query_collection(client, collection_name, question, number_of_results):
embeddings = HuggingFaceEmbeddings(model_name="all-mpnet-base-v2")
query_vector = embeddings.embed_query(question)

result = client.search(
    collection_name=collection_name,
    query_vector=query_vector,
    limit=number_of_results)
answer_data = list()
for scoredpoint in result:
    idnumber, version, score, payload, vector = scoredpoint
    metadata = payload[1]['metadata']
    page_content = payload[1]['page_content']
    data = [score[1], metadata, page_content]
    answer_data.append(data)
return answer_data        

``

I hope this will help. :-)

[VpkPrasanna]

Hi @wawaks this method works but is there any way i can use this with in langchain for RAG based implementations ?

[PrzeMazur95]

HI @VpkPrasanna, have you figured out if you can use already created collection for RAG purposes? It's been a long time, but I think that I have the same problem. You can create collection on the fly, with methods like from_text, or from_document, and then use it, but I am not able to create the collection first, and only once, and then use it like from_existing_collection(collection_name="xxx")

[wawaks]

I have no idea. :-(