Windows: Noob questions about how to set this up.

[RookieRacoon]

In step 4 I have Ollama already installed and I have phi3:14b installed, though I could not add phi3:14b-medium-128k-instruct since it is not available on Ollama and on huggingface there was no Ollama option, only transformers or vLLM.

The next instruction is "Create a file named modelfile with these exact contents:" My question is how do I create a file? Pasting the contents to the command prompt directly definitely didn't work so what am I missing.

Sorry if this is extremely basic stuff.

[electronicjude]

I might be able to help you with that. I use Notepad++ , CRTL + N to create a new file, save it just as 'Modelfile' without extension. You can Notepad or any text editor to do the same. And you know what to do from there. I think that bit addresses your question, but here's some additional stuff that may help if you're getting no love:

I generally save the 'Modelfile' without extension in the folder where my .gguf model file is located - its easy - but you can save it anywhere really because you specify the Path to the model in the Modelfile.

The contents I've seen vary quite a bit - I created one just today specifically for the deepseek-r1 model which I'll paste below - the PARAMETERs for which I got off the Ollama website even though the distilled model I downloaded via LMStudio is not the one strictly supported via Ollama - it still works. Same for the TEMPLATE. Contents of my Modelfile:

FROM full\path\to\your\gguf\model_name.gguf
PARAMETER num_ctx 38000
PARAMETER stop "<｜begin▁of▁sentence｜>"
PARAMETER stop "<｜end▁of▁sentence｜>"
PARAMETER stop "<｜User｜>"
PARAMETER stop "<｜Assistant｜>"
TEMPLATE """{{- if .System }}{{ .System }}{{ end }}
{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1}}
{{- if eq .Role "user" }}<｜User｜>{{ .Content }}
{{- else if eq .Role "assistant" }}<｜Assistant｜>{{ .Content }}{{- if not $last }}<｜end▁of▁sentence｜>{{- end }}
{{- end }}
{{- if and $last (ne .Role "assistant") }}<｜Assistant｜>{{- end }}
{{- end }}"""

hope it helps bro