RuntimeWarning: Enable tracemalloc to get the object allocation traceback

[wuyang910]

I have deployed a model locally using Ollama, and when I run it in vscode with python=3.12, I encountered the following Runtime error, which seems to be related to synchronization or asynchronization. I have checked that the local model runs normally and can be invoked properly. But I'm not sure where the problem occurred. Thank you all for your guidance.

error as follows:

My code is

import os
os.environ["OLLAMA_NUM_PARALLEL"] = "4"  
os.environ["OLLAMA_MAX_LOADED_MODELS"] = "4" 
from paperqa import Docs, Settings

# valid extensions include .pdf, .txt, and .html
doc_paths = ("/home/python_project/RagControl/data/metagpt.pdf", "/homepython_project/RagControl/data/Spheroid culture.pdf")
model_name = "ollama/deepseek-r1:8b"
embedding_model_name= "ollama/ordis/gte-Qwen2-7B-instruct-Q5_K_M-GGUF-8k:latest"

local_llm_config = {
    "model_list": [
        {
            "model_name": model_name,
            "litellm_params": {
                "model": model_name,
                "api_base": "http://localhost:11434",
                "api_type" : "ollama",
            }
        }
    ]
}

embedding_llm_config = {
    "model_list": [
       {
            "model_name": embedding_model_name,
            "litellm_params": {
                "model": embedding_model_name,
                "api_base": "http://localhost:11434",
                "api_type" : "ollama",
            }
        }
    ]
}

settings = Settings(llm=model_name,
        llm_config=local_llm_config,
        summary_llm=model_name,
        summary_llm_config=local_llm_config,
        embedding=embedding_model_name,
        embedding_config= embedding_llm_config
        )
settings.answer.answer_max_sources = 3 
settings.agent.agent_llm=model_name  
settings.agent.agent_llm_config=local_llm_config

docs = Docs()

for doc in doc_paths:
    print(doc)
    docs.add(doc, settings=settings) 

answer = docs.query(
    "What is metagpt?",
    settings=settings
)

print(answer.formatted_answer)

[dosubot[bot]]

There are two similar unsolved discussions related to your issue:

1. RuntimeWarning: Enable tracemalloc to get the object allocation traceback ^[1].
2. RuntimeError: This event loop is already running ^[2].

Your issue seems to be related to the asyncio event loop and the anyio library, particularly when using Python 3.12. Here are a few suggestions to help resolve the problem:

1. Check Event Loop Compatibility: Ensure that the asyncio event loop is correctly initialized and compatible with the libraries you are using. Python 3.12 may have introduced changes that affect how event loops are managed.

2. Use async Methods: If your code involves asynchronous operations, consider using the asynchronous methods provided by the paperqa module, such as aadd and aquery, to ensure proper handling of async tasks.

3. Isolate Event Loops: If you are running multiple asynchronous tasks, make sure each task is using the correct event loop. You might need to explicitly set the event loop for each task.

4. Update Libraries: Ensure that all libraries (anyio, asyncio, litellm, etc.) are up to date, as updates may include fixes for compatibility issues with Python 3.12.

5. Debugging: Use tracemalloc to get more detailed memory allocation traces, which might help identify where the issue originates.

These steps should help you troubleshoot and potentially resolve the issue with the asyncio event loop in your setup.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[maykcaldas]

Hello @wuyang910 !

Have you seen this discussion.
I usually see these asyncio event loop errors when I'm running a jupyter notebook. If that's your case, try to add nast_asyncio to the beginning of your notebook:

import nest_asyncio
nest_asyncio.apply()

[wuyang910]

Thank you @maykcaldas
I run the code in vscode with the format of ".py". I try to add nast_asyncio to the beginning of my ".py". But the Runtime error still exists. I follow your advice, I try to run the code in a jupyter notebook with adding the code

import nest_asyncio
nest_asyncio.apply() 

to the beginning of my notebook. It works well. But I still don‘t understand what's the problem of runnming Doc synchronously in the format of ".py"
So I read the document of usage of paperqa in Pypi again carefully. This is the link https://pypi.org/project/paper-qa/ . I try to use Doc asynchronously in “.py”，it then works well.
My code as follows:

import asyncio
from paperqa import Docs

async def main() -> None:
    docs = Docs()
    # valid extensions include .pdf, .txt, .md, and .html
    for doc in ("/home/python_project/RagControl/data/metagpt.pdf", "/home/python_project/RagControl/data/Spheroid culture.pdf"):
        await docs.aadd(doc,settings=settings)
    session = await docs.aquery(
    "What are metagpt?",settings=settings
    ) 
    print(session.formatted_answer)

asyncio.run(main())

Above all, it seems maybe there is something wrong when running Doc synchronously in the format of ".py, but I can't understand.

[wuyang910]

Furthermore ，I try run the code in vscode with the format of ".ipynb" without the code

import nest_asyncio
nest_asyncio.apply() 

adding to the beginning, it still works well

[maykcaldas]

Hello @wuyang910 ,

That's very weird. I am able to reproduce your error. However, the error doesn't exist when using other providers. I think the issue is on how litellm handles ollama async calls. I can't give you a definitive diagnostic, but I believe using asyncio.run() is the correct approach here. I'll set it as answer for now and close the discussion. Feel free to reopen it for any additional questions!