Best Ollama model for local LLM?

[GreaseMonkey88]

First of all, this is an awesome tool!

What would be the best Ollama model to choose for this project? In my case, I´m running it locally on my M4 Max with 64GB of RAM.

Thanks!

[maxwelljmckee]

The answer is, it's complicated. It depends the types of capabilities you want your model to have such as tools and vision.

As far as your computer, it sounds like a powerhouse. You should be able to comfortably run medium-size models up to ≈30-40B parameters without any issues. Larger than this and you might start to see performance degradation. If you're curious, you can always pop open the Activity Monitor app and watch your memory utilization spike as the model processes a request. This should give you a pretty good idea of your machine's limitations.

Last, the choice among open sources models such as llama, mistral, qwen, or gemma generally comes down to the types of tasks you're going to ask it to perform, as well as your preferred "writing style" generated by the model. Each will have a bit of a different character, and the best way to choose is to experiment with the options until you feel satisfied with the model that best meets the demands of your project.

[BradKML]

Any good defaults for people with 8GB/16GB/32GB/64GB RAM space? Let's say the topics are STEM, humanities, and basic life skills, and that access to internet is possible through proper tool use.