Excessive Size

[blunweon]

Hi there, saw your Reddit post and it piqued my interest. Attempted to run it on my Ollama server. However, I notice it has a huge increase in memory size when compared to when the LLM is being ran in natively via ollama run <model>. Is this an intended function? I have attached a screenshot showing the difference in memory size and CPU/GPU utilization between the two. I noticed it didn't utilize my GPU as well although Ollama is served with GPU passed to it (as you can see in the native ollama run portion, it is able to utilize my GPU).

I also hit this rate limiting message when attempting to run a nonsense research topic, is that normal?

[blunweon]

Managed to get GPU to work by adding 'num_gpu': 1 to the llm_wrapper.py under the generate_ollama function. Attached a screenshot below for example. I believe it is able to make use of nvidia GPUs as well by setting the variable to point to the nvidia GPU but am not able to test it out as I do not have an nvidia GPU available to test.

The other variables included after 'num_gpu': 1 are not needed for GPU usage upon testing. When 'num_gpu' is set to 0, purely CPU will be used without offloading to GPU.

Changes to the llm_config.py and llm_wrapper.py can be viewed in my repo for a possible solution: https://github.com/dave-yap/Automated-AI-Web-Researcher-Ollama

I have an issue where I am unable to get a summary of the research done in the terminal, but I can see all the websites and texts in the .txt file. The .txt file generated does not have a research summary as well?