Parallelization

[Sumanth-er]

I am trying to insert text documents but using the insert() method takes up lot of time. Is there any way to run this parallel to speed up?

I'm using ollama model on windows.

[ntsarb]

I haven't looked into the code, yet, and I have only tested the default config. I observed that most of the time the GPU is ~40% utilised in my case, which suggests there's space for improvement. Performance may be entirely different with postgres+age or another DB. I would like to test it. If the LightRAG implementation doesn't allow for multiple chunks to be processed in parallel, there is certainly potential for optimisation.

[ntsarb]

Update: It was a configuration issue. I'm now reaching 90% GPU utilisation, which is perfecty fine with me.

[BireleyX]

I also want to speed it up myself, but haven't got time to experiment yet.

Have you tried playing with the following environment parameters (Numbers in parenthesis are the default values):
MAX_PARALLEL_INSERT (2)
EMBEDDING_BATCH_NUM (32)
EMBEDDING_FUNC_MAX_ASYNC (16)
MAX_ASYNC (4)

if you just want to avoid execution blocking, you can using the async functions instead... ie. lightrag.ainsert() instead of lightrag.insert().
you can also try batch insert by passing a list of filenames to lightrag.insert()

please do update if you have findings...

[BireleyX]

I also tried Lightrag with ollama before, with my paltry 8GB 3070 GPU. I recall the embedding process took the GPU to 100%.
and I didn't see the ollama logs updating anymore for several minutes... and so i gave up.
the bottleneck might by your system and not primarily lightrag.

[Sumanth-er]

I have already tried with those parameters but when I measured the time taken for each step I found that the embedding were fast to create.The main problem is final response by llm which takes lot of time.